2024-05-03T14:22:21.326 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-03T14:22:21.326 DEBUG:teuthology.run:Teuthology command: teuthology --name rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi --archive /home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687343 --owner scheduled_rishabh@teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/suites/pjd}} --verbose -- /home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687343/orig.config.yaml 2024-05-03T14:22:21.385 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-03T14:22:21.484 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687343 branch: wip-rishabh-testing-20240501.193033 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/suites/pjd}} email: null first_in_suite: false job_id: '7687343' kernel: client: branch: testing flavor: default kdb: true sha1: 806ec36b21129f113ce356c007dd59694d340b37 ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240501.193033 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 fuse default permissions: false fuse set user groups: true rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: 9a047fe7599a549042d06a3fb5b6813a203eabea subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported install: ceph: flavor: default sha1: 9a047fe7599a549042d06a3fb5b6813a203eabea kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - nowsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240501.193033 sha1: 9a047fe7599a549042d06a3fb5b6813a203eabea owner: scheduled_rishabh@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 1698 sha1: 9a047fe7599a549042d06a3fb5b6813a203eabea sleep_before_teardown: 0 subset: 434/512 suite: fs suite_branch: wip-rishabh-testing-20240501.193033 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_9a047fe7599a549042d06a3fb5b6813a203eabea/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 9a047fe7599a549042d06a3fb5b6813a203eabea syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi060.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN60ohXqyQ4xTGpuJWX9HxonnjrmxRBmiqHg5iN26cwI/KCpNJ5Sc1kSgjcB2p3dpSe7oz9UzLlYPp5bTS+Am+E= smithi090.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOlmViT/vmNVrEbE6Z+ZbKbEiQbLdqSvx1HK3HuBrCj8yAimQ9rfBovQ4tx6cxlcm+K1iqPhg3qtIQz6dLl8CRo= smithi149.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB+gfbFMrK4+ZHV2UxCa21jLg5KDjBEEOh1D2EqBf8EtG7hwlZl2V6EHmyJ7kJDQVC1ELMmTGpXlQKZHTKZbu98= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/pjd.sh timeout: 6h teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-03_13:56:19 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-03T14:22:21.485 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_9a047fe7599a549042d06a3fb5b6813a203eabea/qa; will attempt to use it 2024-05-03T14:22:21.485 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_9a047fe7599a549042d06a3fb5b6813a203eabea/qa/tasks 2024-05-03T14:22:21.486 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-03T14:22:21.487 INFO:teuthology.task.internal:Checking packages... 2024-05-03T14:22:21.510 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '9a047fe7599a549042d06a3fb5b6813a203eabea' 2024-05-03T14:22:21.510 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-03T14:22:21.510 INFO:teuthology.packaging:ref: None 2024-05-03T14:22:21.511 INFO:teuthology.packaging:tag: None 2024-05-03T14:22:21.511 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240501.193033 2024-05-03T14:22:21.511 INFO:teuthology.packaging:sha1: 9a047fe7599a549042d06a3fb5b6813a203eabea 2024-05-03T14:22:21.511 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240501.193033 2024-05-03T14:22:21.652 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3450.g9a047fe7 2024-05-03T14:22:21.654 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-03T14:22:21.663 INFO:teuthology.task.internal:no buildpackages task found 2024-05-03T14:22:21.663 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-03T14:22:21.684 INFO:teuthology.task.internal:Saving configuration 2024-05-03T14:22:21.704 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-03T14:22:21.713 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-03T14:22:21.738 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi060.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687343', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-03 14:16:48.122341', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN60ohXqyQ4xTGpuJWX9HxonnjrmxRBmiqHg5iN26cwI/KCpNJ5Sc1kSgjcB2p3dpSe7oz9UzLlYPp5bTS+Am+E='} 2024-05-03T14:22:21.757 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi090.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687343', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-03 14:16:48.120985', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOlmViT/vmNVrEbE6Z+ZbKbEiQbLdqSvx1HK3HuBrCj8yAimQ9rfBovQ4tx6cxlcm+K1iqPhg3qtIQz6dLl8CRo='} 2024-05-03T14:22:21.775 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi149.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687343', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-03 14:16:48.119511', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB+gfbFMrK4+ZHV2UxCa21jLg5KDjBEEOh1D2EqBf8EtG7hwlZl2V6EHmyJ7kJDQVC1ELMmTGpXlQKZHTKZbu98='} 2024-05-03T14:22:21.775 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-03T14:22:21.787 INFO:teuthology.task.internal:roles: ubuntu@smithi060.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-05-03T14:22:21.787 INFO:teuthology.task.internal:roles: ubuntu@smithi090.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-05-03T14:22:21.787 INFO:teuthology.task.internal:roles: ubuntu@smithi149.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-05-03T14:22:21.787 INFO:teuthology.run_tasks:Running task console_log... 2024-05-03T14:22:21.925 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f0ab80ff160>, signals=[15]) 2024-05-03T14:22:21.925 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-03T14:22:21.934 INFO:teuthology.task.internal:Opening connections... 2024-05-03T14:22:21.934 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi060.front.sepia.ceph.com 2024-05-03T14:22:21.936 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-03T14:22:22.013 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi090.front.sepia.ceph.com 2024-05-03T14:22:22.013 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-03T14:22:22.089 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi149.front.sepia.ceph.com 2024-05-03T14:22:22.090 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-03T14:22:22.165 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-03T14:22:22.176 DEBUG:teuthology.orchestra.run.smithi060:> uname -m 2024-05-03T14:22:22.195 INFO:teuthology.orchestra.run.smithi060.stdout:x86_64 2024-05-03T14:22:22.195 DEBUG:teuthology.orchestra.run.smithi060:> cat /etc/os-release 2024-05-03T14:22:22.253 INFO:teuthology.orchestra.run.smithi060.stdout:NAME="CentOS Stream" 2024-05-03T14:22:22.254 INFO:teuthology.orchestra.run.smithi060.stdout:VERSION="9" 2024-05-03T14:22:22.254 INFO:teuthology.orchestra.run.smithi060.stdout:ID="centos" 2024-05-03T14:22:22.254 INFO:teuthology.orchestra.run.smithi060.stdout:ID_LIKE="rhel fedora" 2024-05-03T14:22:22.254 INFO:teuthology.orchestra.run.smithi060.stdout:VERSION_ID="9" 2024-05-03T14:22:22.254 INFO:teuthology.orchestra.run.smithi060.stdout:PLATFORM_ID="platform:el9" 2024-05-03T14:22:22.254 INFO:teuthology.orchestra.run.smithi060.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-03T14:22:22.254 INFO:teuthology.orchestra.run.smithi060.stdout:ANSI_COLOR="0;31" 2024-05-03T14:22:22.254 INFO:teuthology.orchestra.run.smithi060.stdout:LOGO="fedora-logo-icon" 2024-05-03T14:22:22.254 INFO:teuthology.orchestra.run.smithi060.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-03T14:22:22.254 INFO:teuthology.orchestra.run.smithi060.stdout:HOME_URL="https://centos.org/" 2024-05-03T14:22:22.254 INFO:teuthology.orchestra.run.smithi060.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-03T14:22:22.254 INFO:teuthology.orchestra.run.smithi060.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-03T14:22:22.255 INFO:teuthology.orchestra.run.smithi060.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-03T14:22:22.255 INFO:teuthology.lock.ops:Updating smithi060.front.sepia.ceph.com on lock server 2024-05-03T14:22:22.273 DEBUG:teuthology.orchestra.run.smithi090:> uname -m 2024-05-03T14:22:22.292 INFO:teuthology.orchestra.run.smithi090.stdout:x86_64 2024-05-03T14:22:22.292 DEBUG:teuthology.orchestra.run.smithi090:> cat /etc/os-release 2024-05-03T14:22:22.349 INFO:teuthology.orchestra.run.smithi090.stdout:NAME="CentOS Stream" 2024-05-03T14:22:22.350 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION="9" 2024-05-03T14:22:22.350 INFO:teuthology.orchestra.run.smithi090.stdout:ID="centos" 2024-05-03T14:22:22.350 INFO:teuthology.orchestra.run.smithi090.stdout:ID_LIKE="rhel fedora" 2024-05-03T14:22:22.350 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION_ID="9" 2024-05-03T14:22:22.350 INFO:teuthology.orchestra.run.smithi090.stdout:PLATFORM_ID="platform:el9" 2024-05-03T14:22:22.350 INFO:teuthology.orchestra.run.smithi090.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-03T14:22:22.350 INFO:teuthology.orchestra.run.smithi090.stdout:ANSI_COLOR="0;31" 2024-05-03T14:22:22.350 INFO:teuthology.orchestra.run.smithi090.stdout:LOGO="fedora-logo-icon" 2024-05-03T14:22:22.351 INFO:teuthology.orchestra.run.smithi090.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-03T14:22:22.351 INFO:teuthology.orchestra.run.smithi090.stdout:HOME_URL="https://centos.org/" 2024-05-03T14:22:22.351 INFO:teuthology.orchestra.run.smithi090.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-03T14:22:22.351 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-03T14:22:22.351 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-03T14:22:22.351 INFO:teuthology.lock.ops:Updating smithi090.front.sepia.ceph.com on lock server 2024-05-03T14:22:22.373 DEBUG:teuthology.orchestra.run.smithi149:> uname -m 2024-05-03T14:22:22.390 INFO:teuthology.orchestra.run.smithi149.stdout:x86_64 2024-05-03T14:22:22.390 DEBUG:teuthology.orchestra.run.smithi149:> cat /etc/os-release 2024-05-03T14:22:22.447 INFO:teuthology.orchestra.run.smithi149.stdout:NAME="CentOS Stream" 2024-05-03T14:22:22.447 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION="9" 2024-05-03T14:22:22.447 INFO:teuthology.orchestra.run.smithi149.stdout:ID="centos" 2024-05-03T14:22:22.447 INFO:teuthology.orchestra.run.smithi149.stdout:ID_LIKE="rhel fedora" 2024-05-03T14:22:22.447 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION_ID="9" 2024-05-03T14:22:22.447 INFO:teuthology.orchestra.run.smithi149.stdout:PLATFORM_ID="platform:el9" 2024-05-03T14:22:22.447 INFO:teuthology.orchestra.run.smithi149.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-03T14:22:22.447 INFO:teuthology.orchestra.run.smithi149.stdout:ANSI_COLOR="0;31" 2024-05-03T14:22:22.448 INFO:teuthology.orchestra.run.smithi149.stdout:LOGO="fedora-logo-icon" 2024-05-03T14:22:22.448 INFO:teuthology.orchestra.run.smithi149.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-03T14:22:22.448 INFO:teuthology.orchestra.run.smithi149.stdout:HOME_URL="https://centos.org/" 2024-05-03T14:22:22.448 INFO:teuthology.orchestra.run.smithi149.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-03T14:22:22.448 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-03T14:22:22.448 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-03T14:22:22.448 INFO:teuthology.lock.ops:Updating smithi149.front.sepia.ceph.com on lock server 2024-05-03T14:22:22.465 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-03T14:22:22.481 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-03T14:22:22.492 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-03T14:22:22.492 DEBUG:teuthology.orchestra.run.smithi060:> test '!' -e /home/ubuntu/cephtest 2024-05-03T14:22:22.495 DEBUG:teuthology.orchestra.run.smithi090:> test '!' -e /home/ubuntu/cephtest 2024-05-03T14:22:22.497 DEBUG:teuthology.orchestra.run.smithi149:> test '!' -e /home/ubuntu/cephtest 2024-05-03T14:22:22.512 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-03T14:22:22.524 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-03T14:22:22.524 DEBUG:teuthology.orchestra.run.smithi060:> test -z $(ls -A /var/lib/ceph) 2024-05-03T14:22:22.553 DEBUG:teuthology.orchestra.run.smithi090:> test -z $(ls -A /var/lib/ceph) 2024-05-03T14:22:22.555 DEBUG:teuthology.orchestra.run.smithi149:> test -z $(ls -A /var/lib/ceph) 2024-05-03T14:22:22.797 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-03T14:22:22.870 INFO:teuthology.run_tasks:Running task kernel... 2024-05-03T14:22:22.891 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'} 2024-05-03T14:22:22.891 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-05-03T14:22:22.891 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-05-03T14:22:22.891 DEBUG:teuthology.orchestra.run.smithi060:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-03T14:22:22.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T14:22:22.909 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-05-03T14:22:23.074 DEBUG:teuthology.task.kernel:sha1 for client.0 is 806ec36b21129f113ce356c007dd59694d340b37 2024-05-03T14:22:23.074 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-05-03T14:22:23.074 DEBUG:teuthology.orchestra.run.smithi060:> uname -r 2024-05-03T14:22:23.092 INFO:teuthology.orchestra.run.smithi060.stdout:5.14.0-437.el9.x86_64 2024-05-03T14:22:23.092 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-05-03T14:22:23.092 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-05-03T14:22:23.093 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum upgrade -y linux-firmware 2024-05-03T14:22:25.818 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - BaseOS 8.0 MB/s | 8.0 MB 00:01 2024-05-03T14:22:27.968 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - AppStream 18 MB/s | 19 MB 00:01 2024-05-03T14:22:32.194 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-05-03T14:22:33.878 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 16 kB 00:00 2024-05-03T14:22:38.057 INFO:teuthology.orchestra.run.smithi060.stdout:Extra Packages for Enterprise Linux 5.2 MB/s | 21 MB 00:04 2024-05-03T14:22:43.330 INFO:teuthology.orchestra.run.smithi060.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-05-03T14:22:45.380 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-05-03T14:22:45.381 INFO:teuthology.orchestra.run.smithi060.stdout:Nothing to do. 2024-05-03T14:22:45.381 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-05-03T14:22:45.497 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-05-03T14:22:45.497 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-05-03T14:22:45.647 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-05-03T14:22:45.647 DEBUG:teuthology.orchestra.run.smithi060:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-05-03T14:22:45.650 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi060.front.sepia.ceph.com to complete... 2024-05-03T14:22:57.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03 14:22:57 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-05-03T14:22:57.764 INFO:teuthology.orchestra.run.smithi060.stderr:FINISHED --2024-05-03 14:22:57-- 2024-05-03T14:22:57.764 INFO:teuthology.orchestra.run.smithi060.stderr:Total wall clock time: 12s 2024-05-03T14:22:57.764 INFO:teuthology.orchestra.run.smithi060.stderr:Downloaded: 1 files, 341M in 12s (28.9 MB/s) 2024-05-03T14:22:57.767 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-05-03T14:22:57.767 DEBUG:teuthology.orchestra.run.smithi060:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-05-03T14:22:59.996 INFO:teuthology.orchestra.run.smithi060.stdout:Verifying... ######################################## 2024-05-03T14:23:00.113 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing... ######################################## 2024-05-03T14:23:00.236 INFO:teuthology.orchestra.run.smithi060.stdout:Updating / installing... 2024-05-03T14:23:08.520 INFO:teuthology.orchestra.run.smithi060.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-05-03T14:23:48.823 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi060.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-05-03T14:23:48.823 DEBUG:teuthology.orchestra.run.smithi060:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-05-03T14:23:48.907 INFO:teuthology.orchestra.run.smithi060.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-05-03T14:23:48.907 INFO:teuthology.orchestra.run.smithi060.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-05-03T14:23:48.907 INFO:teuthology.orchestra.run.smithi060.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-05-03T14:23:48.907 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-05-03T14:23:48.907 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-05-03T14:23:48.907 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-05-03T14:23:48.907 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-05-03T14:23:48.908 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-05-03T14:23:48.908 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-05-03T14:23:48.908 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-05-03T14:23:48.908 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-05-03T14:23:48.908 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-05-03T14:23:48.908 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-05-03T14:23:48.908 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-05-03T14:23:48.908 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-05-03T14:23:48.908 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-05-03T14:23:48.908 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-05-03T14:23:48.908 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-05-03T14:23:48.908 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-05-03T14:23:48.908 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-05-03T14:23:48.908 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-05-03T14:23:48.909 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-05-03T14:23:48.909 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-05-03T14:23:48.909 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-05-03T14:23:48.909 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-05-03T14:23:48.909 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-05-03T14:23:48.909 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-05-03T14:23:48.909 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-05-03T14:23:48.909 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-05-03T14:23:48.909 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-05-03T14:23:48.909 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-05-03T14:23:48.909 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-05-03T14:23:48.909 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-05-03T14:23:48.910 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-05-03T14:23:48.910 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-05-03T14:23:48.910 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-05-03T14:23:48.910 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-05-03T14:23:48.910 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-05-03T14:23:48.910 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-05-03T14:23:48.910 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-05-03T14:23:48.910 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-05-03T14:23:48.910 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-05-03T14:23:48.910 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-05-03T14:23:48.910 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-05-03T14:23:48.910 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-05-03T14:23:48.910 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-05-03T14:23:48.911 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-05-03T14:23:48.911 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-05-03T14:23:48.911 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-05-03T14:23:48.911 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-05-03T14:23:48.911 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-05-03T14:23:48.911 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-05-03T14:23:48.911 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-05-03T14:23:48.911 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-05-03T14:23:48.911 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-05-03T14:23:48.911 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-05-03T14:23:48.911 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-05-03T14:23:48.911 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-05-03T14:23:48.911 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-05-03T14:23:48.912 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-05-03T14:23:48.912 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-05-03T14:23:48.912 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-05-03T14:23:48.912 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-05-03T14:23:48.912 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-05-03T14:23:48.912 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-05-03T14:23:48.912 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-05-03T14:23:48.912 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-05-03T14:23:48.912 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-05-03T14:23:48.912 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-05-03T14:23:48.912 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-05-03T14:23:48.912 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-05-03T14:23:48.912 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-05-03T14:23:48.912 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-05-03T14:23:48.913 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-05-03T14:23:48.913 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-05-03T14:23:48.913 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-05-03T14:23:48.913 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-05-03T14:23:48.913 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-05-03T14:23:48.913 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-05-03T14:23:48.913 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-05-03T14:23:48.913 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-05-03T14:23:48.913 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-05-03T14:23:48.913 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-05-03T14:23:48.913 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-05-03T14:23:48.913 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-05-03T14:23:48.913 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-05-03T14:23:48.914 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-05-03T14:23:48.914 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-05-03T14:23:48.914 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-05-03T14:23:48.914 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-05-03T14:23:48.914 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-05-03T14:23:48.914 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-05-03T14:23:48.914 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-05-03T14:23:48.914 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-05-03T14:23:48.914 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-05-03T14:23:48.914 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-05-03T14:23:48.914 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-05-03T14:23:48.914 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-05-03T14:23:48.914 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-05-03T14:23:48.915 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-05-03T14:23:48.915 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-05-03T14:23:48.915 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-05-03T14:23:48.915 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-05-03T14:23:48.915 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-05-03T14:23:48.915 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-05-03T14:23:48.915 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-05-03T14:23:48.915 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-05-03T14:23:48.915 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-05-03T14:23:48.915 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-05-03T14:23:48.915 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-05-03T14:23:48.915 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-05-03T14:23:48.915 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-05-03T14:23:48.915 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-05-03T14:23:48.916 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-05-03T14:23:48.916 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-05-03T14:23:48.916 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-05-03T14:23:48.916 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-05-03T14:23:48.916 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-05-03T14:23:48.916 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-05-03T14:23:48.916 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-05-03T14:23:48.916 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-05-03T14:23:48.916 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-05-03T14:23:48.916 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-05-03T14:23:48.916 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-05-03T14:23:48.916 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-05-03T14:23:48.916 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-05-03T14:23:48.917 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-05-03T14:23:48.917 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-05-03T14:23:48.917 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-05-03T14:23:48.917 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-05-03T14:23:48.917 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-05-03T14:23:48.917 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-05-03T14:23:48.917 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-05-03T14:23:48.917 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-05-03T14:23:48.917 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-05-03T14:23:48.917 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-05-03T14:23:48.917 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-05-03T14:23:48.917 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-05-03T14:23:48.917 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-05-03T14:23:48.918 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-05-03T14:23:48.918 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-05-03T14:23:48.918 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-05-03T14:23:48.918 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-05-03T14:23:48.918 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-05-03T14:23:48.918 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-05-03T14:23:48.918 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-05-03T14:23:48.918 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-05-03T14:23:48.918 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-05-03T14:23:48.918 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-05-03T14:23:48.918 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-05-03T14:23:48.918 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-05-03T14:23:48.918 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-05-03T14:23:48.919 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-05-03T14:23:48.919 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-05-03T14:23:48.919 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-05-03T14:23:48.919 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-05-03T14:23:48.919 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-05-03T14:23:48.919 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-05-03T14:23:48.919 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-05-03T14:23:48.919 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-05-03T14:23:48.919 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-05-03T14:23:48.919 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-05-03T14:23:48.919 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-05-03T14:23:48.919 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-05-03T14:23:48.919 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-05-03T14:23:48.919 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-05-03T14:23:48.920 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-05-03T14:23:48.920 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-05-03T14:23:48.920 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-05-03T14:23:48.920 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-05-03T14:23:48.920 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-05-03T14:23:48.920 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-05-03T14:23:48.920 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-05-03T14:23:48.920 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-05-03T14:23:48.920 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-05-03T14:23:48.920 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-05-03T14:23:48.920 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-05-03T14:23:48.920 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-05-03T14:23:48.920 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-05-03T14:23:48.920 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-05-03T14:23:48.920 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-05-03T14:23:48.920 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-05-03T14:23:48.921 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-05-03T14:23:48.921 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-05-03T14:23:48.921 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-05-03T14:23:48.921 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-05-03T14:23:48.921 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-05-03T14:23:48.921 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-05-03T14:23:48.921 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-05-03T14:23:48.921 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-05-03T14:23:48.921 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-05-03T14:23:48.921 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-05-03T14:23:48.921 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-05-03T14:23:48.921 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-05-03T14:23:48.921 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-05-03T14:23:48.921 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-05-03T14:23:48.921 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-05-03T14:23:48.922 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-05-03T14:23:48.922 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-05-03T14:23:48.922 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-05-03T14:23:48.922 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-05-03T14:23:48.922 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-05-03T14:23:48.922 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-05-03T14:23:48.922 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-05-03T14:23:48.922 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-05-03T14:23:48.922 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-05-03T14:23:48.922 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-05-03T14:23:48.922 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-05-03T14:23:48.922 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-05-03T14:23:48.922 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-05-03T14:23:48.923 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-05-03T14:23:48.923 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-05-03T14:23:48.923 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-05-03T14:23:48.923 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-05-03T14:23:48.923 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-05-03T14:23:48.923 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-05-03T14:23:48.923 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-05-03T14:23:48.923 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-05-03T14:23:48.923 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-05-03T14:23:48.923 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-05-03T14:23:48.923 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-05-03T14:23:48.923 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-05-03T14:23:48.923 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-05-03T14:23:48.923 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-05-03T14:23:48.923 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-05-03T14:23:48.924 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-05-03T14:23:48.924 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-05-03T14:23:48.924 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-05-03T14:23:48.924 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-05-03T14:23:48.924 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-05-03T14:23:48.924 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-05-03T14:23:48.924 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-05-03T14:23:48.924 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-05-03T14:23:48.924 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-05-03T14:23:48.924 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-05-03T14:23:48.924 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-05-03T14:23:48.924 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-05-03T14:23:48.924 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-05-03T14:23:48.925 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-05-03T14:23:48.925 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-05-03T14:23:48.925 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-05-03T14:23:48.925 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-05-03T14:23:48.925 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-05-03T14:23:48.925 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-05-03T14:23:48.925 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-05-03T14:23:48.925 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-05-03T14:23:48.925 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-05-03T14:23:48.925 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-05-03T14:23:48.925 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-05-03T14:23:48.925 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-05-03T14:23:48.926 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-05-03T14:23:48.926 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-05-03T14:23:48.926 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-05-03T14:23:48.926 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-05-03T14:23:48.926 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-05-03T14:23:48.926 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-05-03T14:23:48.926 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-05-03T14:23:48.926 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-05-03T14:23:48.926 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-05-03T14:23:48.926 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-05-03T14:23:48.926 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-05-03T14:23:48.926 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-05-03T14:23:48.927 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-05-03T14:23:48.927 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-05-03T14:23:48.927 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-05-03T14:23:48.927 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-05-03T14:23:48.927 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-05-03T14:23:48.927 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-05-03T14:23:48.927 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-05-03T14:23:48.927 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-05-03T14:23:48.927 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-05-03T14:23:48.927 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-05-03T14:23:48.927 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-05-03T14:23:48.927 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-05-03T14:23:48.927 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-05-03T14:23:48.927 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-05-03T14:23:48.928 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-05-03T14:23:48.928 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-05-03T14:23:48.928 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-05-03T14:23:48.928 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-05-03T14:23:48.928 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-05-03T14:23:48.928 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-05-03T14:23:48.928 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-05-03T14:23:48.928 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-05-03T14:23:48.928 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-05-03T14:23:48.928 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-05-03T14:23:48.928 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-05-03T14:23:48.928 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-05-03T14:23:48.928 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-05-03T14:23:48.929 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-05-03T14:23:48.929 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-05-03T14:23:48.929 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-05-03T14:23:48.929 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-05-03T14:23:48.929 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-05-03T14:23:48.929 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-05-03T14:23:48.929 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-05-03T14:23:48.930 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-05-03T14:23:48.930 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-05-03T14:23:48.930 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-05-03T14:23:48.930 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-05-03T14:23:48.930 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-05-03T14:23:48.930 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-05-03T14:23:48.930 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-05-03T14:23:48.930 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-05-03T14:23:48.930 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-05-03T14:23:48.930 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-05-03T14:23:48.930 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-05-03T14:23:48.930 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-05-03T14:23:48.930 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-05-03T14:23:48.931 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-05-03T14:23:48.931 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-05-03T14:23:48.931 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-05-03T14:23:48.931 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-05-03T14:23:48.931 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-05-03T14:23:48.931 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-05-03T14:23:48.931 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-05-03T14:23:48.931 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-05-03T14:23:48.931 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-05-03T14:23:48.931 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-05-03T14:23:48.932 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-05-03T14:23:48.932 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-05-03T14:23:48.932 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-05-03T14:23:48.932 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-05-03T14:23:48.932 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-05-03T14:23:48.933 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-05-03T14:23:48.933 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-05-03T14:23:48.933 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-05-03T14:23:48.933 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-05-03T14:23:48.933 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-05-03T14:23:48.933 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-05-03T14:23:48.934 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-05-03T14:23:48.934 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-05-03T14:23:48.934 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-05-03T14:23:48.934 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-05-03T14:23:48.934 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-05-03T14:23:48.934 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-05-03T14:23:48.934 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-05-03T14:23:48.934 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-05-03T14:23:48.935 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-05-03T14:23:48.935 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-05-03T14:23:48.935 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-05-03T14:23:48.935 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-05-03T14:23:48.935 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-05-03T14:23:48.935 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-05-03T14:23:48.935 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-05-03T14:23:48.935 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-05-03T14:23:48.935 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-05-03T14:23:48.935 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-05-03T14:23:48.935 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-05-03T14:23:48.936 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-05-03T14:23:48.936 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-05-03T14:23:48.936 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-05-03T14:23:48.936 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-05-03T14:23:48.936 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-05-03T14:23:48.936 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-05-03T14:23:48.936 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-05-03T14:23:48.936 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-05-03T14:23:48.936 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-05-03T14:23:48.936 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-05-03T14:23:48.936 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-05-03T14:23:48.936 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-05-03T14:23:48.936 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-05-03T14:23:48.937 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-05-03T14:23:48.937 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-05-03T14:23:48.937 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-05-03T14:23:48.937 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-05-03T14:23:48.937 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-05-03T14:23:48.937 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-05-03T14:23:48.937 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-05-03T14:23:48.937 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-05-03T14:23:48.937 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-05-03T14:23:48.937 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-05-03T14:23:48.937 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-05-03T14:23:48.937 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-05-03T14:23:48.937 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-05-03T14:23:48.937 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-05-03T14:23:48.937 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-05-03T14:23:48.938 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-05-03T14:23:48.938 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-05-03T14:23:48.938 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-05-03T14:23:48.938 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-05-03T14:23:48.938 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-05-03T14:23:48.938 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-05-03T14:23:48.938 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-05-03T14:23:48.938 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-05-03T14:23:48.938 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-05-03T14:23:48.938 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-05-03T14:23:48.938 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-05-03T14:23:48.938 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-05-03T14:23:48.938 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-05-03T14:23:48.939 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-05-03T14:23:48.939 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-05-03T14:23:48.939 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-05-03T14:23:48.939 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-05-03T14:23:48.939 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-05-03T14:23:48.939 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-05-03T14:23:48.939 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-05-03T14:23:48.939 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-05-03T14:23:48.939 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-05-03T14:23:48.939 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-05-03T14:23:48.939 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-05-03T14:23:48.940 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-05-03T14:23:48.940 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-05-03T14:23:48.940 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-05-03T14:23:48.940 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-05-03T14:23:48.940 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-05-03T14:23:48.940 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-05-03T14:23:48.940 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-05-03T14:23:48.940 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-05-03T14:23:48.940 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-05-03T14:23:48.940 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-05-03T14:23:48.940 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-05-03T14:23:48.940 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-05-03T14:23:48.940 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-05-03T14:23:48.941 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-05-03T14:23:48.941 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-05-03T14:23:48.941 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-05-03T14:23:48.941 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-05-03T14:23:48.941 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-05-03T14:23:48.941 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-05-03T14:23:48.941 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-05-03T14:23:48.941 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-05-03T14:23:48.941 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-05-03T14:23:48.941 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-05-03T14:23:48.941 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-05-03T14:23:48.941 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-05-03T14:23:48.941 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-05-03T14:23:48.941 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-05-03T14:23:48.942 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-05-03T14:23:48.942 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-05-03T14:23:48.942 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-05-03T14:23:48.942 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-05-03T14:23:48.942 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-05-03T14:23:48.942 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-05-03T14:23:48.942 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-05-03T14:23:48.942 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-05-03T14:23:48.942 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-05-03T14:23:48.942 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-05-03T14:23:48.943 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-05-03T14:23:48.943 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-05-03T14:23:48.943 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-05-03T14:23:48.943 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-05-03T14:23:48.943 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-05-03T14:23:48.943 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-05-03T14:23:48.943 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-05-03T14:23:48.943 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-05-03T14:23:48.943 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-05-03T14:23:48.943 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-05-03T14:23:48.943 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-05-03T14:23:48.943 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-05-03T14:23:48.943 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-05-03T14:23:48.943 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-05-03T14:23:48.944 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-05-03T14:23:48.944 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-05-03T14:23:48.944 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-05-03T14:23:48.944 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-05-03T14:23:48.944 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-05-03T14:23:48.944 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-05-03T14:23:48.944 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-05-03T14:23:48.944 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-05-03T14:23:48.944 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-05-03T14:23:48.944 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-05-03T14:23:48.944 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-05-03T14:23:48.944 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-05-03T14:23:48.944 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-05-03T14:23:48.944 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-05-03T14:23:48.944 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-05-03T14:23:48.945 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-05-03T14:23:48.945 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-05-03T14:23:48.945 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-05-03T14:23:48.945 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-05-03T14:23:48.945 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-05-03T14:23:48.945 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-05-03T14:23:48.945 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-05-03T14:23:48.945 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-05-03T14:23:48.945 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-05-03T14:23:48.945 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-05-03T14:23:48.945 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-05-03T14:23:48.945 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-05-03T14:23:48.945 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-05-03T14:23:48.946 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-05-03T14:23:48.946 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-05-03T14:23:48.946 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-05-03T14:23:48.946 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-05-03T14:23:48.946 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-05-03T14:23:48.946 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-05-03T14:23:48.946 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-05-03T14:23:48.946 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-05-03T14:23:48.946 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-05-03T14:23:48.946 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-05-03T14:23:48.946 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-05-03T14:23:48.946 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-05-03T14:23:48.947 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-05-03T14:23:48.947 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-05-03T14:23:48.947 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-05-03T14:23:48.947 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-05-03T14:23:48.947 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-05-03T14:23:48.947 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-05-03T14:23:48.947 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-05-03T14:23:48.947 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-05-03T14:23:48.947 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-05-03T14:23:48.947 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-05-03T14:23:48.947 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-05-03T14:23:48.947 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-05-03T14:23:48.947 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-05-03T14:23:48.947 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-05-03T14:23:48.948 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-05-03T14:23:48.948 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-05-03T14:23:48.948 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-05-03T14:23:48.948 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-05-03T14:23:48.948 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-05-03T14:23:48.948 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-05-03T14:23:48.948 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-05-03T14:23:48.948 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-05-03T14:23:48.948 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-05-03T14:23:48.948 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-05-03T14:23:48.948 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-05-03T14:23:48.948 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-05-03T14:23:48.948 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-05-03T14:23:48.948 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-05-03T14:23:48.949 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-05-03T14:23:48.949 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-05-03T14:23:48.949 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-05-03T14:23:48.949 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-05-03T14:23:48.949 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-05-03T14:23:48.949 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-05-03T14:23:48.949 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-05-03T14:23:48.949 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-05-03T14:23:48.949 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-05-03T14:23:48.949 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-05-03T14:23:48.949 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-05-03T14:23:48.949 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-05-03T14:23:48.949 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-05-03T14:23:48.950 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-05-03T14:23:48.950 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-05-03T14:23:48.950 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-05-03T14:23:48.950 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-05-03T14:23:48.950 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-05-03T14:23:48.950 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-05-03T14:23:48.950 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-05-03T14:23:48.950 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-05-03T14:23:48.950 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-05-03T14:23:48.950 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-05-03T14:23:48.950 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-05-03T14:23:48.950 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-05-03T14:23:48.950 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-05-03T14:23:48.951 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-05-03T14:23:48.951 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-05-03T14:23:48.951 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-05-03T14:23:48.951 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-05-03T14:23:48.951 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-05-03T14:23:48.951 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-05-03T14:23:48.951 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-05-03T14:23:48.951 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-05-03T14:23:48.951 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-05-03T14:23:48.951 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-05-03T14:23:48.951 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-05-03T14:23:48.951 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-05-03T14:23:48.951 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-05-03T14:23:48.951 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-05-03T14:23:48.952 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-05-03T14:23:48.952 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-05-03T14:23:48.952 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-05-03T14:23:48.952 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-05-03T14:23:48.952 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-05-03T14:23:48.952 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-05-03T14:23:48.952 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-05-03T14:23:48.952 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-05-03T14:23:48.952 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-05-03T14:23:48.952 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-05-03T14:23:48.952 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-05-03T14:23:48.952 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-05-03T14:23:48.952 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-05-03T14:23:48.952 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-05-03T14:23:48.952 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-05-03T14:23:48.953 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-05-03T14:23:48.953 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-05-03T14:23:48.953 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-05-03T14:23:48.953 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-05-03T14:23:48.953 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-05-03T14:23:48.953 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-05-03T14:23:48.953 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-05-03T14:23:48.953 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-05-03T14:23:48.953 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-05-03T14:23:48.953 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-05-03T14:23:48.953 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-05-03T14:23:48.953 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-05-03T14:23:48.954 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-05-03T14:23:48.954 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-05-03T14:23:48.954 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-05-03T14:23:48.954 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-05-03T14:23:48.954 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-05-03T14:23:48.954 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-05-03T14:23:48.954 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-05-03T14:23:48.954 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-05-03T14:23:48.954 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-05-03T14:23:48.954 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-05-03T14:23:48.954 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-05-03T14:23:48.954 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-05-03T14:23:48.954 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-05-03T14:23:48.954 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-05-03T14:23:48.955 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-05-03T14:23:48.955 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-05-03T14:23:48.955 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-05-03T14:23:48.955 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-05-03T14:23:48.955 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-05-03T14:23:48.955 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-05-03T14:23:48.955 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-05-03T14:23:48.955 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-05-03T14:23:48.955 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-05-03T14:23:48.955 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-05-03T14:23:48.955 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-05-03T14:23:48.955 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-05-03T14:23:48.955 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-05-03T14:23:48.955 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-05-03T14:23:48.955 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-05-03T14:23:48.956 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-05-03T14:23:48.956 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-05-03T14:23:48.956 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-05-03T14:23:48.956 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-05-03T14:23:48.956 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-05-03T14:23:48.956 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-05-03T14:23:48.956 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-05-03T14:23:48.956 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-05-03T14:23:48.956 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-05-03T14:23:48.956 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-05-03T14:23:48.956 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-05-03T14:23:48.956 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-05-03T14:23:48.956 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-05-03T14:23:48.956 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-05-03T14:23:48.956 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-05-03T14:23:48.957 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-05-03T14:23:48.957 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-05-03T14:23:48.957 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-05-03T14:23:48.957 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-05-03T14:23:48.957 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-05-03T14:23:48.957 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-05-03T14:23:48.957 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-05-03T14:23:48.957 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-05-03T14:23:48.957 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-05-03T14:23:48.957 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-05-03T14:23:48.957 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-05-03T14:23:48.957 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-05-03T14:23:48.957 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-05-03T14:23:48.957 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-05-03T14:23:48.958 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-05-03T14:23:48.958 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-05-03T14:23:48.958 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-05-03T14:23:48.958 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-05-03T14:23:48.958 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-05-03T14:23:48.958 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-05-03T14:23:48.958 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-05-03T14:23:48.958 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-05-03T14:23:48.958 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-05-03T14:23:48.959 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-05-03T14:23:48.959 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-05-03T14:23:48.959 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-05-03T14:23:48.959 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-05-03T14:23:48.959 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-05-03T14:23:48.959 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-05-03T14:23:48.959 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-05-03T14:23:48.959 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-05-03T14:23:48.959 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-05-03T14:23:48.959 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-05-03T14:23:48.959 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-05-03T14:23:48.959 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-05-03T14:23:48.960 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-05-03T14:23:48.960 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-05-03T14:23:48.960 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-05-03T14:23:48.960 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-05-03T14:23:48.960 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-05-03T14:23:48.960 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-05-03T14:23:48.960 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-05-03T14:23:48.960 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-05-03T14:23:48.960 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-05-03T14:23:48.960 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-05-03T14:23:48.960 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-05-03T14:23:48.960 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-05-03T14:23:48.960 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-05-03T14:23:48.960 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-05-03T14:23:48.960 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-05-03T14:23:48.961 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-05-03T14:23:48.961 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-05-03T14:23:48.961 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-05-03T14:23:48.961 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-05-03T14:23:48.961 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-05-03T14:23:48.961 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-05-03T14:23:48.961 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-05-03T14:23:48.961 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-05-03T14:23:48.961 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-05-03T14:23:48.961 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-05-03T14:23:48.961 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-05-03T14:23:48.961 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-05-03T14:23:48.962 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-05-03T14:23:48.962 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-05-03T14:23:48.962 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-05-03T14:23:48.962 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-05-03T14:23:48.962 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-05-03T14:23:48.962 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-05-03T14:23:48.962 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-05-03T14:23:48.962 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-05-03T14:23:48.962 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-05-03T14:23:48.962 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-05-03T14:23:48.962 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-05-03T14:23:48.962 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-05-03T14:23:48.962 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-05-03T14:23:48.962 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-05-03T14:23:48.963 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-05-03T14:23:48.963 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-05-03T14:23:48.963 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-05-03T14:23:48.963 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-05-03T14:23:48.963 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-05-03T14:23:48.963 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-05-03T14:23:48.963 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-05-03T14:23:48.963 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-05-03T14:23:48.963 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-05-03T14:23:48.963 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-05-03T14:23:48.963 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-05-03T14:23:48.963 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-05-03T14:23:48.963 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-05-03T14:23:48.963 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-05-03T14:23:48.964 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-05-03T14:23:48.964 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-05-03T14:23:48.964 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-05-03T14:23:48.964 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-05-03T14:23:48.964 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-05-03T14:23:48.964 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-05-03T14:23:48.964 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-05-03T14:23:48.964 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-05-03T14:23:48.964 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-05-03T14:23:48.964 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-05-03T14:23:48.964 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-05-03T14:23:48.964 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-05-03T14:23:48.964 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-05-03T14:23:48.964 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-05-03T14:23:48.964 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-05-03T14:23:48.965 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-05-03T14:23:48.965 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-05-03T14:23:48.965 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-05-03T14:23:48.965 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-05-03T14:23:48.965 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-05-03T14:23:48.965 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-05-03T14:23:48.965 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-05-03T14:23:48.965 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-05-03T14:23:48.965 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-05-03T14:23:48.965 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-05-03T14:23:48.965 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-05-03T14:23:48.966 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-05-03T14:23:48.966 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-05-03T14:23:48.966 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-05-03T14:23:48.966 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-05-03T14:23:48.966 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-05-03T14:23:48.966 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-05-03T14:23:48.966 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-05-03T14:23:48.966 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-05-03T14:23:48.966 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-05-03T14:23:48.966 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-05-03T14:23:48.966 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-05-03T14:23:48.966 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-05-03T14:23:48.966 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-05-03T14:23:48.966 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-05-03T14:23:48.967 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-05-03T14:23:48.967 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-05-03T14:23:48.967 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-05-03T14:23:48.967 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-05-03T14:23:48.967 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-05-03T14:23:48.967 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-05-03T14:23:48.967 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-05-03T14:23:48.967 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-05-03T14:23:48.967 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-05-03T14:23:48.967 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-05-03T14:23:48.967 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-05-03T14:23:48.967 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-05-03T14:23:48.967 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-05-03T14:23:48.967 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-05-03T14:23:48.968 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-05-03T14:23:48.968 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-05-03T14:23:48.968 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-05-03T14:23:48.968 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-05-03T14:23:48.968 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-05-03T14:23:48.968 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-05-03T14:23:48.968 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-05-03T14:23:48.968 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-05-03T14:23:48.968 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-05-03T14:23:48.968 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-05-03T14:23:48.968 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-05-03T14:23:48.968 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-05-03T14:23:48.968 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-05-03T14:23:48.968 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-05-03T14:23:48.968 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-05-03T14:23:48.969 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-05-03T14:23:48.969 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-05-03T14:23:48.969 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-05-03T14:23:48.969 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-05-03T14:23:48.969 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-05-03T14:23:48.969 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-05-03T14:23:48.969 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-05-03T14:23:48.969 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-05-03T14:23:48.969 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-05-03T14:23:48.969 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-05-03T14:23:48.969 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-05-03T14:23:48.969 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-05-03T14:23:48.969 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-05-03T14:23:48.969 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-05-03T14:23:48.970 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-05-03T14:23:48.970 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-05-03T14:23:48.970 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-05-03T14:23:48.970 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-05-03T14:23:48.970 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-05-03T14:23:48.970 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-05-03T14:23:48.970 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-05-03T14:23:48.970 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-05-03T14:23:48.970 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-05-03T14:23:48.970 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-05-03T14:23:48.970 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-05-03T14:23:48.970 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-05-03T14:23:48.970 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-05-03T14:23:48.970 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-05-03T14:23:48.971 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-05-03T14:23:48.971 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-05-03T14:23:48.971 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-05-03T14:23:48.971 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-05-03T14:23:48.971 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-05-03T14:23:48.971 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-05-03T14:23:48.971 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-05-03T14:23:48.971 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-05-03T14:23:48.971 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-05-03T14:23:48.971 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-05-03T14:23:48.971 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-05-03T14:23:48.971 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-05-03T14:23:48.971 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-05-03T14:23:48.971 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-05-03T14:23:48.971 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-05-03T14:23:48.972 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-05-03T14:23:48.972 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-05-03T14:23:48.972 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-05-03T14:23:48.972 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-05-03T14:23:48.972 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-05-03T14:23:48.972 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-05-03T14:23:48.972 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-05-03T14:23:48.972 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-05-03T14:23:48.972 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-05-03T14:23:48.972 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-05-03T14:23:48.972 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-05-03T14:23:48.972 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-05-03T14:23:48.972 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-05-03T14:23:48.972 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-05-03T14:23:48.972 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-05-03T14:23:48.973 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-05-03T14:23:48.973 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-05-03T14:23:48.973 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-05-03T14:23:48.973 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-05-03T14:23:48.973 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-05-03T14:23:48.973 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-05-03T14:23:48.973 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-05-03T14:23:48.973 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-05-03T14:23:48.973 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-05-03T14:23:48.973 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-05-03T14:23:48.973 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-05-03T14:23:48.973 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-05-03T14:23:48.973 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-05-03T14:23:48.973 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-05-03T14:23:48.973 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-05-03T14:23:48.974 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-05-03T14:23:48.974 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-05-03T14:23:48.974 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-05-03T14:23:48.974 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-05-03T14:23:48.974 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-05-03T14:23:48.974 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-05-03T14:23:48.974 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-05-03T14:23:48.974 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-05-03T14:23:48.974 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-05-03T14:23:48.974 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-05-03T14:23:48.974 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-05-03T14:23:48.974 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-05-03T14:23:48.974 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-05-03T14:23:48.975 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-05-03T14:23:48.975 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-05-03T14:23:48.975 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-05-03T14:23:48.975 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-05-03T14:23:48.975 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-05-03T14:23:48.975 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-05-03T14:23:48.975 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-05-03T14:23:48.975 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-05-03T14:23:48.975 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-05-03T14:23:48.975 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-05-03T14:23:48.975 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-05-03T14:23:48.975 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-05-03T14:23:48.975 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-05-03T14:23:48.975 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-05-03T14:23:48.976 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-05-03T14:23:48.976 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-05-03T14:23:48.976 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-05-03T14:23:48.976 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-05-03T14:23:48.976 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-05-03T14:23:48.976 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-05-03T14:23:48.976 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-05-03T14:23:48.976 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-05-03T14:23:48.976 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-05-03T14:23:48.976 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-05-03T14:23:48.976 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-05-03T14:23:48.976 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-05-03T14:23:48.976 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-05-03T14:23:48.976 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-05-03T14:23:48.976 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-05-03T14:23:48.977 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-05-03T14:23:48.977 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-05-03T14:23:48.977 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-05-03T14:23:48.977 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-05-03T14:23:48.977 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-05-03T14:23:48.977 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-05-03T14:23:48.977 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-05-03T14:23:48.977 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-05-03T14:23:48.977 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-05-03T14:23:48.977 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-05-03T14:23:48.977 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-05-03T14:23:48.977 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-05-03T14:23:48.977 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-05-03T14:23:48.977 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-05-03T14:23:48.978 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-05-03T14:23:48.978 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-05-03T14:23:48.978 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-05-03T14:23:48.978 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-05-03T14:23:48.978 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-05-03T14:23:48.978 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-05-03T14:23:48.978 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-05-03T14:23:48.978 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-05-03T14:23:48.978 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-05-03T14:23:48.978 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-05-03T14:23:48.979 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-05-03T14:23:48.979 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-05-03T14:23:48.979 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-05-03T14:23:48.979 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-05-03T14:23:48.979 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-05-03T14:23:48.979 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-05-03T14:23:48.979 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-05-03T14:23:48.979 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-05-03T14:23:48.979 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-05-03T14:23:48.979 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-05-03T14:23:48.979 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-05-03T14:23:48.979 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-05-03T14:23:48.979 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-05-03T14:23:48.980 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-05-03T14:23:48.980 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-05-03T14:23:48.980 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-05-03T14:23:48.980 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-05-03T14:23:48.980 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-05-03T14:23:48.980 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-05-03T14:23:48.980 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-05-03T14:23:48.980 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-05-03T14:23:48.980 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-05-03T14:23:48.980 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-05-03T14:23:48.980 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-05-03T14:23:48.980 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-05-03T14:23:48.980 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-05-03T14:23:48.980 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-05-03T14:23:48.981 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-05-03T14:23:48.981 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-05-03T14:23:48.981 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-05-03T14:23:48.981 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-05-03T14:23:48.981 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-05-03T14:23:48.981 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-05-03T14:23:48.981 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-05-03T14:23:48.981 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-05-03T14:23:48.981 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-05-03T14:23:48.981 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-05-03T14:23:48.981 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-05-03T14:23:48.981 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-05-03T14:23:48.981 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-05-03T14:23:48.981 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-05-03T14:23:48.981 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-05-03T14:23:48.982 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-05-03T14:23:48.982 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-05-03T14:23:48.982 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-05-03T14:23:48.982 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-05-03T14:23:48.982 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-05-03T14:23:48.982 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-05-03T14:23:48.982 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-05-03T14:23:48.982 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-05-03T14:23:48.982 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-05-03T14:23:48.982 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-05-03T14:23:48.982 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-05-03T14:23:48.982 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-05-03T14:23:48.982 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-05-03T14:23:48.983 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-05-03T14:23:48.983 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-05-03T14:23:48.983 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-05-03T14:23:48.983 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-05-03T14:23:48.983 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-05-03T14:23:48.983 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-05-03T14:23:48.983 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-05-03T14:23:48.983 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-05-03T14:23:48.983 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-05-03T14:23:48.983 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-05-03T14:23:48.983 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-05-03T14:23:48.983 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-05-03T14:23:48.984 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-05-03T14:23:48.984 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-05-03T14:23:48.984 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-05-03T14:23:48.984 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-05-03T14:23:48.984 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-05-03T14:23:48.984 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-05-03T14:23:48.984 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-05-03T14:23:48.984 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-05-03T14:23:48.984 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-05-03T14:23:48.984 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-05-03T14:23:48.984 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-05-03T14:23:48.984 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-05-03T14:23:48.984 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-05-03T14:23:48.984 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-05-03T14:23:48.985 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-05-03T14:23:48.985 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-05-03T14:23:48.985 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-05-03T14:23:48.985 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-05-03T14:23:48.985 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-05-03T14:23:48.985 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-05-03T14:23:48.985 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-05-03T14:23:48.985 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-05-03T14:23:48.985 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-05-03T14:23:48.985 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-05-03T14:23:48.985 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-05-03T14:23:48.985 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-05-03T14:23:48.985 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-05-03T14:23:48.985 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-05-03T14:23:48.986 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-05-03T14:23:48.986 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-05-03T14:23:48.986 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-05-03T14:23:48.986 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-05-03T14:23:48.986 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-05-03T14:23:48.986 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-05-03T14:23:48.986 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-05-03T14:23:48.986 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-05-03T14:23:48.986 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-05-03T14:23:48.986 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-05-03T14:23:48.986 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-05-03T14:23:48.986 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-05-03T14:23:48.986 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-05-03T14:23:48.987 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-05-03T14:23:48.987 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-05-03T14:23:48.987 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-05-03T14:23:48.987 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-05-03T14:23:48.987 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-05-03T14:23:48.987 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-05-03T14:23:48.987 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-05-03T14:23:48.987 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-05-03T14:23:48.987 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-05-03T14:23:48.987 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-05-03T14:23:48.987 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-05-03T14:23:48.987 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-05-03T14:23:48.987 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-05-03T14:23:48.987 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-05-03T14:23:48.987 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-05-03T14:23:48.988 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-05-03T14:23:48.988 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-05-03T14:23:48.988 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-05-03T14:23:48.988 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-05-03T14:23:48.988 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-05-03T14:23:48.988 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-05-03T14:23:48.988 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-05-03T14:23:48.988 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-05-03T14:23:48.988 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-05-03T14:23:48.988 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-05-03T14:23:48.988 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-05-03T14:23:48.988 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-05-03T14:23:48.988 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-05-03T14:23:48.988 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-05-03T14:23:48.989 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-05-03T14:23:48.989 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-05-03T14:23:48.989 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-05-03T14:23:48.989 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-05-03T14:23:48.989 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-05-03T14:23:48.989 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-05-03T14:23:48.989 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-05-03T14:23:48.989 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-05-03T14:23:48.989 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-05-03T14:23:48.989 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-05-03T14:23:48.989 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-05-03T14:23:48.989 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-05-03T14:23:48.989 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-05-03T14:23:48.989 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-05-03T14:23:48.989 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-05-03T14:23:48.990 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-05-03T14:23:48.990 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-05-03T14:23:48.990 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-05-03T14:23:48.990 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-05-03T14:23:48.990 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-05-03T14:23:48.990 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-05-03T14:23:48.990 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-05-03T14:23:48.990 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-05-03T14:23:48.990 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-05-03T14:23:48.990 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-05-03T14:23:48.990 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-05-03T14:23:48.990 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-05-03T14:23:48.990 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-05-03T14:23:48.991 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-05-03T14:23:48.991 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-05-03T14:23:48.991 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-05-03T14:23:48.991 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-05-03T14:23:48.991 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-05-03T14:23:48.991 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-05-03T14:23:48.991 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-05-03T14:23:48.991 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-05-03T14:23:48.991 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-05-03T14:23:48.991 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-05-03T14:23:48.991 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-05-03T14:23:48.991 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-05-03T14:23:48.991 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-05-03T14:23:48.991 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-05-03T14:23:48.992 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-05-03T14:23:48.992 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-05-03T14:23:48.992 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-05-03T14:23:48.992 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-05-03T14:23:48.992 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-05-03T14:23:48.992 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-05-03T14:23:48.992 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-05-03T14:23:48.992 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-05-03T14:23:48.992 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-05-03T14:23:48.992 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-05-03T14:23:48.992 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-05-03T14:23:48.992 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-05-03T14:23:48.992 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-05-03T14:23:48.993 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-05-03T14:23:48.993 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-05-03T14:23:48.993 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-05-03T14:23:48.993 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-05-03T14:23:48.993 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-05-03T14:23:48.993 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-05-03T14:23:48.993 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-05-03T14:23:48.993 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-05-03T14:23:48.993 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-05-03T14:23:48.993 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-05-03T14:23:48.993 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-05-03T14:23:48.993 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-05-03T14:23:48.993 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-05-03T14:23:48.993 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-05-03T14:23:48.994 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-05-03T14:23:48.994 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-05-03T14:23:48.994 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-05-03T14:23:48.994 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-05-03T14:23:48.994 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-05-03T14:23:48.994 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-05-03T14:23:48.994 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-05-03T14:23:48.994 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-05-03T14:23:48.994 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-05-03T14:23:48.994 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-05-03T14:23:48.994 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-05-03T14:23:48.994 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-05-03T14:23:48.994 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-05-03T14:23:48.995 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-05-03T14:23:48.995 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-05-03T14:23:48.995 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-05-03T14:23:48.995 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-05-03T14:23:48.995 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-05-03T14:23:48.995 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-05-03T14:23:48.995 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-05-03T14:23:48.995 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-05-03T14:23:48.995 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-05-03T14:23:48.995 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-05-03T14:23:48.995 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-05-03T14:23:48.995 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-05-03T14:23:48.995 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-05-03T14:23:48.995 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-05-03T14:23:48.995 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-05-03T14:23:48.995 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-05-03T14:23:48.996 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-05-03T14:23:48.996 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-05-03T14:23:48.996 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-05-03T14:23:48.996 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-05-03T14:23:48.996 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-05-03T14:23:48.996 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-05-03T14:23:48.996 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-05-03T14:23:48.996 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-05-03T14:23:48.996 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-05-03T14:23:48.996 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-05-03T14:23:48.996 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-05-03T14:23:48.996 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-05-03T14:23:48.996 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-05-03T14:23:48.996 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-05-03T14:23:48.996 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-05-03T14:23:48.997 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-05-03T14:23:48.997 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-05-03T14:23:48.997 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-05-03T14:23:48.997 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-05-03T14:23:48.997 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-05-03T14:23:48.997 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-05-03T14:23:48.997 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-05-03T14:23:48.997 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-05-03T14:23:48.997 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-05-03T14:23:48.997 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-05-03T14:23:48.997 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-05-03T14:23:48.997 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-05-03T14:23:48.997 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-05-03T14:23:48.997 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-05-03T14:23:48.998 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-05-03T14:23:48.998 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-05-03T14:23:48.998 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-05-03T14:23:48.998 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-05-03T14:23:48.998 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-05-03T14:23:48.998 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-05-03T14:23:48.998 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-05-03T14:23:48.998 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-05-03T14:23:48.998 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-05-03T14:23:48.998 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-05-03T14:23:48.998 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-05-03T14:23:48.998 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-05-03T14:23:48.998 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-05-03T14:23:48.999 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-05-03T14:23:48.999 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-05-03T14:23:48.999 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-05-03T14:23:48.999 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-05-03T14:23:48.999 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-05-03T14:23:48.999 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-05-03T14:23:48.999 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-05-03T14:23:48.999 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-05-03T14:23:48.999 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-05-03T14:23:48.999 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-05-03T14:23:48.999 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-05-03T14:23:48.999 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-05-03T14:23:48.999 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-05-03T14:23:48.999 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-05-03T14:23:48.999 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-05-03T14:23:49.000 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-05-03T14:23:49.000 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-05-03T14:23:49.000 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-05-03T14:23:49.000 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-05-03T14:23:49.000 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-05-03T14:23:49.000 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-05-03T14:23:49.000 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-05-03T14:23:49.000 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-05-03T14:23:49.000 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-05-03T14:23:49.000 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-05-03T14:23:49.000 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-05-03T14:23:49.000 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-05-03T14:23:49.000 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-05-03T14:23:49.000 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-05-03T14:23:49.000 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-05-03T14:23:49.001 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-05-03T14:23:49.001 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-05-03T14:23:49.001 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-05-03T14:23:49.001 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-05-03T14:23:49.001 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-05-03T14:23:49.001 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-05-03T14:23:49.001 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-05-03T14:23:49.001 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-05-03T14:23:49.001 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-05-03T14:23:49.001 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-05-03T14:23:49.001 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-05-03T14:23:49.001 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-05-03T14:23:49.001 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-05-03T14:23:49.001 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-05-03T14:23:49.002 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-05-03T14:23:49.002 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-05-03T14:23:49.002 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-05-03T14:23:49.002 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-05-03T14:23:49.002 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-05-03T14:23:49.002 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-05-03T14:23:49.002 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-05-03T14:23:49.002 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-05-03T14:23:49.002 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-05-03T14:23:49.002 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-05-03T14:23:49.002 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-05-03T14:23:49.002 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-05-03T14:23:49.002 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-05-03T14:23:49.003 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-05-03T14:23:49.003 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-05-03T14:23:49.003 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-05-03T14:23:49.003 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-05-03T14:23:49.003 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-05-03T14:23:49.003 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-05-03T14:23:49.003 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-05-03T14:23:49.003 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-05-03T14:23:49.003 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-05-03T14:23:49.003 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-05-03T14:23:49.003 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-05-03T14:23:49.003 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-05-03T14:23:49.003 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-05-03T14:23:49.003 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-05-03T14:23:49.003 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-05-03T14:23:49.003 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-05-03T14:23:49.004 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-05-03T14:23:49.004 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-05-03T14:23:49.004 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-05-03T14:23:49.004 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-05-03T14:23:49.004 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-05-03T14:23:49.004 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-05-03T14:23:49.004 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-05-03T14:23:49.004 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-05-03T14:23:49.004 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-05-03T14:23:49.004 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-05-03T14:23:49.004 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-05-03T14:23:49.004 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-05-03T14:23:49.004 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-05-03T14:23:49.004 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-05-03T14:23:49.004 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-05-03T14:23:49.005 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-05-03T14:23:49.005 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-05-03T14:23:49.005 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-05-03T14:23:49.005 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-05-03T14:23:49.005 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-05-03T14:23:49.005 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-05-03T14:23:49.005 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-05-03T14:23:49.005 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-05-03T14:23:49.005 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-05-03T14:23:49.005 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-05-03T14:23:49.005 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-05-03T14:23:49.005 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-05-03T14:23:49.005 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-05-03T14:23:49.005 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-05-03T14:23:49.006 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-05-03T14:23:49.006 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-05-03T14:23:49.006 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-05-03T14:23:49.006 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-05-03T14:23:49.006 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-05-03T14:23:49.006 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-05-03T14:23:49.006 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-05-03T14:23:49.006 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-05-03T14:23:49.006 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-05-03T14:23:49.006 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-05-03T14:23:49.006 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-05-03T14:23:49.006 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-05-03T14:23:49.006 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-05-03T14:23:49.006 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-05-03T14:23:49.007 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-05-03T14:23:49.007 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-05-03T14:23:49.007 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-05-03T14:23:49.007 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-05-03T14:23:49.007 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-05-03T14:23:49.007 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-05-03T14:23:49.007 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-05-03T14:23:49.007 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-05-03T14:23:49.007 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-05-03T14:23:49.007 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-05-03T14:23:49.007 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-05-03T14:23:49.007 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-05-03T14:23:49.007 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-05-03T14:23:49.007 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-05-03T14:23:49.008 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-05-03T14:23:49.008 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-05-03T14:23:49.008 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-05-03T14:23:49.008 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-05-03T14:23:49.008 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-05-03T14:23:49.008 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-05-03T14:23:49.008 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-05-03T14:23:49.008 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-05-03T14:23:49.008 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-05-03T14:23:49.008 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-05-03T14:23:49.008 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-05-03T14:23:49.008 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-05-03T14:23:49.008 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-05-03T14:23:49.008 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-05-03T14:23:49.008 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-05-03T14:23:49.009 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-05-03T14:23:49.009 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-05-03T14:23:49.009 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-05-03T14:23:49.009 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-05-03T14:23:49.009 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-05-03T14:23:49.009 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-05-03T14:23:49.009 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-05-03T14:23:49.009 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-05-03T14:23:49.009 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-05-03T14:23:49.009 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-05-03T14:23:49.009 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-05-03T14:23:49.009 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-05-03T14:23:49.009 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-05-03T14:23:49.009 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-05-03T14:23:49.010 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-05-03T14:23:49.010 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-05-03T14:23:49.010 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-05-03T14:23:49.010 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-05-03T14:23:49.010 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-05-03T14:23:49.010 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-05-03T14:23:49.010 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-05-03T14:23:49.010 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-05-03T14:23:49.010 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-05-03T14:23:49.010 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-05-03T14:23:49.010 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-05-03T14:23:49.010 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-05-03T14:23:49.010 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-05-03T14:23:49.010 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-05-03T14:23:49.011 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-05-03T14:23:49.011 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-05-03T14:23:49.011 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-05-03T14:23:49.011 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-05-03T14:23:49.011 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-05-03T14:23:49.011 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-05-03T14:23:49.011 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-05-03T14:23:49.011 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-05-03T14:23:49.011 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-05-03T14:23:49.011 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-05-03T14:23:49.011 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-05-03T14:23:49.011 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-05-03T14:23:49.011 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-05-03T14:23:49.011 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-05-03T14:23:49.012 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-05-03T14:23:49.012 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-05-03T14:23:49.012 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-05-03T14:23:49.012 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-05-03T14:23:49.012 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-05-03T14:23:49.012 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-05-03T14:23:49.012 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-05-03T14:23:49.012 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-05-03T14:23:49.012 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-05-03T14:23:49.012 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-05-03T14:23:49.012 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-05-03T14:23:49.012 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-05-03T14:23:49.012 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-05-03T14:23:49.012 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-05-03T14:23:49.013 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-05-03T14:23:49.013 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-05-03T14:23:49.013 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-05-03T14:23:49.013 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-05-03T14:23:49.013 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-05-03T14:23:49.013 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-05-03T14:23:49.013 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-05-03T14:23:49.013 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-05-03T14:23:49.013 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-05-03T14:23:49.013 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-05-03T14:23:49.013 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-05-03T14:23:49.013 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-05-03T14:23:49.013 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-05-03T14:23:49.013 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-05-03T14:23:49.014 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-05-03T14:23:49.014 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-05-03T14:23:49.014 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-05-03T14:23:49.014 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-05-03T14:23:49.014 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-05-03T14:23:49.014 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-05-03T14:23:49.014 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-05-03T14:23:49.014 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-05-03T14:23:49.014 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-05-03T14:23:49.014 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-05-03T14:23:49.014 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-05-03T14:23:49.014 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-05-03T14:23:49.014 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-05-03T14:23:49.015 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-05-03T14:23:49.015 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-05-03T14:23:49.015 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-05-03T14:23:49.015 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-05-03T14:23:49.015 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-05-03T14:23:49.015 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-05-03T14:23:49.015 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-05-03T14:23:49.015 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-05-03T14:23:49.015 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-05-03T14:23:49.015 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-05-03T14:23:49.015 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-05-03T14:23:49.015 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-05-03T14:23:49.015 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-05-03T14:23:49.015 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-05-03T14:23:49.016 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-05-03T14:23:49.016 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-05-03T14:23:49.016 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-05-03T14:23:49.016 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-05-03T14:23:49.016 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-05-03T14:23:49.016 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-05-03T14:23:49.016 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-05-03T14:23:49.016 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-05-03T14:23:49.016 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-05-03T14:23:49.016 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-05-03T14:23:49.016 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-05-03T14:23:49.016 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-05-03T14:23:49.016 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-05-03T14:23:49.016 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-05-03T14:23:49.017 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-05-03T14:23:49.017 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-05-03T14:23:49.017 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-05-03T14:23:49.017 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-05-03T14:23:49.017 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-05-03T14:23:49.017 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-05-03T14:23:49.017 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-05-03T14:23:49.017 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-05-03T14:23:49.017 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-05-03T14:23:49.017 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-05-03T14:23:49.017 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-05-03T14:23:49.017 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-05-03T14:23:49.017 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-05-03T14:23:49.017 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-05-03T14:23:49.018 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-05-03T14:23:49.018 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-05-03T14:23:49.018 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-05-03T14:23:49.018 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-05-03T14:23:49.018 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-05-03T14:23:49.018 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-05-03T14:23:49.018 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-05-03T14:23:49.018 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-05-03T14:23:49.018 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-05-03T14:23:49.018 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-05-03T14:23:49.018 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-05-03T14:23:49.018 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-05-03T14:23:49.018 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-05-03T14:23:49.018 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-05-03T14:23:49.019 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-05-03T14:23:49.019 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-05-03T14:23:49.019 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-05-03T14:23:49.019 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-05-03T14:23:49.019 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-05-03T14:23:49.019 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-05-03T14:23:49.019 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-05-03T14:23:49.019 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-05-03T14:23:49.019 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-05-03T14:23:49.019 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-05-03T14:23:49.019 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-05-03T14:23:49.019 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-05-03T14:23:49.019 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-05-03T14:23:49.019 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-05-03T14:23:49.020 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-05-03T14:23:49.020 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-05-03T14:23:49.020 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-05-03T14:23:49.020 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-05-03T14:23:49.020 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-05-03T14:23:49.020 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-05-03T14:23:49.020 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-05-03T14:23:49.020 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-05-03T14:23:49.020 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-05-03T14:23:49.020 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-05-03T14:23:49.020 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-05-03T14:23:49.020 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-05-03T14:23:49.020 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-05-03T14:23:49.020 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-05-03T14:23:49.021 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-05-03T14:23:49.021 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-05-03T14:23:49.021 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-05-03T14:23:49.021 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-05-03T14:23:49.021 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-05-03T14:23:49.021 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-05-03T14:23:49.021 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-05-03T14:23:49.021 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-05-03T14:23:49.021 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-05-03T14:23:49.021 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-05-03T14:23:49.021 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-05-03T14:23:49.021 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-05-03T14:23:49.021 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-05-03T14:23:49.022 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-05-03T14:23:49.022 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-05-03T14:23:49.022 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-05-03T14:23:49.022 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-05-03T14:23:49.022 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-05-03T14:23:49.022 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-05-03T14:23:49.022 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-05-03T14:23:49.022 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-05-03T14:23:49.022 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-05-03T14:23:49.022 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-05-03T14:23:49.022 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-05-03T14:23:49.022 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-05-03T14:23:49.023 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-05-03T14:23:49.023 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-05-03T14:23:49.023 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-05-03T14:23:49.023 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-05-03T14:23:49.023 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-05-03T14:23:49.023 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-05-03T14:23:49.023 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-05-03T14:23:49.023 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-05-03T14:23:49.023 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-05-03T14:23:49.023 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-05-03T14:23:49.023 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-05-03T14:23:49.023 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-05-03T14:23:49.023 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-05-03T14:23:49.023 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-05-03T14:23:49.024 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-05-03T14:23:49.024 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-05-03T14:23:49.024 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-05-03T14:23:49.024 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-05-03T14:23:49.024 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-05-03T14:23:49.024 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-05-03T14:23:49.024 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-05-03T14:23:49.024 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-05-03T14:23:49.024 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-05-03T14:23:49.024 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-05-03T14:23:49.024 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-05-03T14:23:49.024 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-05-03T14:23:49.024 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-05-03T14:23:49.025 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-05-03T14:23:49.025 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-05-03T14:23:49.025 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-05-03T14:23:49.025 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-05-03T14:23:49.025 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-05-03T14:23:49.025 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-05-03T14:23:49.025 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-05-03T14:23:49.025 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-05-03T14:23:49.025 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-05-03T14:23:49.025 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-05-03T14:23:49.025 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-05-03T14:23:49.025 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-05-03T14:23:49.026 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-05-03T14:23:49.026 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-05-03T14:23:49.026 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-05-03T14:23:49.026 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-05-03T14:23:49.026 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-05-03T14:23:49.026 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-05-03T14:23:49.026 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-05-03T14:23:49.026 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-05-03T14:23:49.026 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-05-03T14:23:49.026 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-05-03T14:23:49.026 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-05-03T14:23:49.026 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-05-03T14:23:49.026 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-05-03T14:23:49.027 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-05-03T14:23:49.027 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-05-03T14:23:49.027 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-05-03T14:23:49.027 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-05-03T14:23:49.027 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-05-03T14:23:49.027 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-05-03T14:23:49.027 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-05-03T14:23:49.027 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-05-03T14:23:49.027 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-05-03T14:23:49.027 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-05-03T14:23:49.027 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-05-03T14:23:49.027 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-05-03T14:23:49.027 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-05-03T14:23:49.027 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-05-03T14:23:49.028 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-05-03T14:23:49.028 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-05-03T14:23:49.028 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-05-03T14:23:49.028 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-05-03T14:23:49.028 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-05-03T14:23:49.028 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-05-03T14:23:49.028 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-05-03T14:23:49.028 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-05-03T14:23:49.028 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-05-03T14:23:49.028 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-05-03T14:23:49.028 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-05-03T14:23:49.028 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-05-03T14:23:49.028 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-05-03T14:23:49.029 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-05-03T14:23:49.029 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-05-03T14:23:49.029 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-05-03T14:23:49.029 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-05-03T14:23:49.029 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-05-03T14:23:49.029 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-05-03T14:23:49.029 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-05-03T14:23:49.029 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-05-03T14:23:49.030 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-05-03T14:23:49.030 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-05-03T14:23:49.030 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-05-03T14:23:49.030 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-05-03T14:23:49.030 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-05-03T14:23:49.031 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-05-03T14:23:49.031 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-05-03T14:23:49.031 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-05-03T14:23:49.031 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-05-03T14:23:49.031 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-05-03T14:23:49.031 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-05-03T14:23:49.032 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-05-03T14:23:49.032 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-05-03T14:23:49.032 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-05-03T14:23:49.032 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-05-03T14:23:49.032 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-05-03T14:23:49.032 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-05-03T14:23:49.032 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-05-03T14:23:49.032 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-05-03T14:23:49.032 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-05-03T14:23:49.032 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-05-03T14:23:49.033 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-05-03T14:23:49.033 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-05-03T14:23:49.033 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-05-03T14:23:49.033 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-05-03T14:23:49.033 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-05-03T14:23:49.033 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-05-03T14:23:49.033 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-05-03T14:23:49.033 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-05-03T14:23:49.033 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-05-03T14:23:49.034 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-05-03T14:23:49.034 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-05-03T14:23:49.034 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-05-03T14:23:49.034 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-05-03T14:23:49.034 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-05-03T14:23:49.034 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-05-03T14:23:49.034 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-05-03T14:23:49.034 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-05-03T14:23:49.035 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-05-03T14:23:49.035 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-05-03T14:23:49.035 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-05-03T14:23:49.035 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-05-03T14:23:49.035 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-05-03T14:23:49.035 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-05-03T14:23:49.035 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-05-03T14:23:49.035 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-05-03T14:23:49.035 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-05-03T14:23:49.035 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-05-03T14:23:49.035 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-05-03T14:23:49.035 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-05-03T14:23:49.035 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-05-03T14:23:49.035 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-05-03T14:23:49.036 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-05-03T14:23:49.036 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-05-03T14:23:49.036 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-05-03T14:23:49.036 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-05-03T14:23:49.036 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-05-03T14:23:49.036 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-05-03T14:23:49.036 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-05-03T14:23:49.036 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-05-03T14:23:49.036 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-05-03T14:23:49.036 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-05-03T14:23:49.036 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-05-03T14:23:49.036 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-05-03T14:23:49.037 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-05-03T14:23:49.037 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-05-03T14:23:49.037 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-05-03T14:23:49.037 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-05-03T14:23:49.037 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-05-03T14:23:49.037 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-05-03T14:23:49.037 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-05-03T14:23:49.037 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-05-03T14:23:49.037 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-05-03T14:23:49.037 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-05-03T14:23:49.037 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-05-03T14:23:49.037 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-05-03T14:23:49.037 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-05-03T14:23:49.037 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-05-03T14:23:49.038 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-05-03T14:23:49.038 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-05-03T14:23:49.038 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-05-03T14:23:49.038 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-05-03T14:23:49.038 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-05-03T14:23:49.038 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-05-03T14:23:49.038 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-05-03T14:23:49.038 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-05-03T14:23:49.038 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-05-03T14:23:49.038 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-05-03T14:23:49.038 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-05-03T14:23:49.039 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-05-03T14:23:49.039 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-05-03T14:23:49.039 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-05-03T14:23:49.039 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-05-03T14:23:49.039 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-05-03T14:23:49.039 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-05-03T14:23:49.039 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-05-03T14:23:49.039 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-05-03T14:23:49.039 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-05-03T14:23:49.039 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-05-03T14:23:49.039 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-05-03T14:23:49.039 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-05-03T14:23:49.039 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-05-03T14:23:49.040 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-05-03T14:23:49.040 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-05-03T14:23:49.040 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-05-03T14:23:49.040 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-05-03T14:23:49.040 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-05-03T14:23:49.040 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-05-03T14:23:49.040 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-05-03T14:23:49.040 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-05-03T14:23:49.040 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-05-03T14:23:49.040 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-05-03T14:23:49.040 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-05-03T14:23:49.040 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-05-03T14:23:49.040 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-05-03T14:23:49.041 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-05-03T14:23:49.041 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-05-03T14:23:49.041 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-05-03T14:23:49.041 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-05-03T14:23:49.041 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-05-03T14:23:49.041 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-05-03T14:23:49.041 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-05-03T14:23:49.041 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-05-03T14:23:49.041 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-05-03T14:23:49.041 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-05-03T14:23:49.041 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-05-03T14:23:49.041 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-05-03T14:23:49.041 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-05-03T14:23:49.041 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-05-03T14:23:49.041 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-05-03T14:23:49.042 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-05-03T14:23:49.042 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-05-03T14:23:49.042 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-05-03T14:23:49.042 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-05-03T14:23:49.042 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-05-03T14:23:49.042 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-05-03T14:23:49.042 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-05-03T14:23:49.042 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-05-03T14:23:49.042 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-05-03T14:23:49.042 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-05-03T14:23:49.042 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-05-03T14:23:49.043 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-05-03T14:23:49.043 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-05-03T14:23:49.043 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-05-03T14:23:49.043 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-05-03T14:23:49.043 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-05-03T14:23:49.043 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-05-03T14:23:49.043 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-05-03T14:23:49.043 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-05-03T14:23:49.043 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-05-03T14:23:49.043 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-05-03T14:23:49.043 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-05-03T14:23:49.043 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-05-03T14:23:49.043 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-05-03T14:23:49.043 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-05-03T14:23:49.044 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-05-03T14:23:49.044 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-05-03T14:23:49.044 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-05-03T14:23:49.044 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-05-03T14:23:49.044 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-05-03T14:23:49.044 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-05-03T14:23:49.044 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-05-03T14:23:49.044 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-05-03T14:23:49.044 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-05-03T14:23:49.044 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-05-03T14:23:49.044 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-05-03T14:23:49.044 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-05-03T14:23:49.044 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-05-03T14:23:49.044 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-05-03T14:23:49.045 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-05-03T14:23:49.045 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-05-03T14:23:49.045 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-05-03T14:23:49.045 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-05-03T14:23:49.045 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-05-03T14:23:49.045 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-05-03T14:23:49.045 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-05-03T14:23:49.045 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-05-03T14:23:49.045 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-05-03T14:23:49.045 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-05-03T14:23:49.045 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-05-03T14:23:49.045 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-05-03T14:23:49.045 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-05-03T14:23:49.045 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-05-03T14:23:49.046 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-05-03T14:23:49.046 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-05-03T14:23:49.046 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-05-03T14:23:49.046 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-05-03T14:23:49.046 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-05-03T14:23:49.046 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-05-03T14:23:49.046 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-05-03T14:23:49.046 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-05-03T14:23:49.046 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-05-03T14:23:49.046 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-05-03T14:23:49.046 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-05-03T14:23:49.047 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-05-03T14:23:49.047 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-05-03T14:23:49.047 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-05-03T14:23:49.047 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-05-03T14:23:49.047 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-05-03T14:23:49.047 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-05-03T14:23:49.047 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-05-03T14:23:49.047 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-05-03T14:23:49.047 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-05-03T14:23:49.047 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-05-03T14:23:49.047 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-05-03T14:23:49.047 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-05-03T14:23:49.047 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-05-03T14:23:49.047 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-05-03T14:23:49.047 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-05-03T14:23:49.048 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-05-03T14:23:49.048 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-05-03T14:23:49.048 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-05-03T14:23:49.048 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-05-03T14:23:49.048 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-05-03T14:23:49.048 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-05-03T14:23:49.048 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-05-03T14:23:49.048 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-05-03T14:23:49.048 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-05-03T14:23:49.048 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-05-03T14:23:49.048 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-05-03T14:23:49.048 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-05-03T14:23:49.048 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-05-03T14:23:49.048 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-05-03T14:23:49.049 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-05-03T14:23:49.049 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-05-03T14:23:49.049 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-05-03T14:23:49.049 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-05-03T14:23:49.049 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-05-03T14:23:49.049 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-05-03T14:23:49.049 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-05-03T14:23:49.049 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-05-03T14:23:49.049 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-05-03T14:23:49.049 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-05-03T14:23:49.049 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-05-03T14:23:49.049 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-05-03T14:23:49.049 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-05-03T14:23:49.049 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-05-03T14:23:49.050 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-05-03T14:23:49.050 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-05-03T14:23:49.050 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-05-03T14:23:49.050 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-05-03T14:23:49.050 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-05-03T14:23:49.050 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-05-03T14:23:49.050 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-05-03T14:23:49.050 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-05-03T14:23:49.050 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-05-03T14:23:49.050 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-05-03T14:23:49.050 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-05-03T14:23:49.050 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-05-03T14:23:49.050 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-05-03T14:23:49.051 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-05-03T14:23:49.051 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-05-03T14:23:49.051 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-05-03T14:23:49.051 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-05-03T14:23:49.051 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-05-03T14:23:49.051 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-05-03T14:23:49.051 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-05-03T14:23:49.051 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-05-03T14:23:49.051 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-05-03T14:23:49.051 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-05-03T14:23:49.051 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-05-03T14:23:49.051 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-05-03T14:23:49.051 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-05-03T14:23:49.051 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-05-03T14:23:49.051 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-05-03T14:23:49.052 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-05-03T14:23:49.052 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-05-03T14:23:49.052 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-05-03T14:23:49.052 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-05-03T14:23:49.052 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-05-03T14:23:49.052 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-05-03T14:23:49.052 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-05-03T14:23:49.052 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-05-03T14:23:49.052 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-05-03T14:23:49.052 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-05-03T14:23:49.052 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-05-03T14:23:49.052 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-05-03T14:23:49.052 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-05-03T14:23:49.052 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-05-03T14:23:49.053 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-05-03T14:23:49.053 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-05-03T14:23:49.053 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-05-03T14:23:49.053 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-05-03T14:23:49.053 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-05-03T14:23:49.053 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-05-03T14:23:49.053 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-05-03T14:23:49.053 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-05-03T14:23:49.053 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-05-03T14:23:49.053 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-05-03T14:23:49.053 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-05-03T14:23:49.053 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-05-03T14:23:49.053 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-05-03T14:23:49.053 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-05-03T14:23:49.054 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-05-03T14:23:49.054 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-05-03T14:23:49.054 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-05-03T14:23:49.054 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-05-03T14:23:49.054 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-05-03T14:23:49.054 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-05-03T14:23:49.054 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-05-03T14:23:49.054 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-05-03T14:23:49.054 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-05-03T14:23:49.054 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-05-03T14:23:49.054 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-05-03T14:23:49.055 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-05-03T14:23:49.055 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-05-03T14:23:49.055 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-05-03T14:23:49.055 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-05-03T14:23:49.055 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-05-03T14:23:49.055 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-05-03T14:23:49.056 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-05-03T14:23:49.056 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-05-03T14:23:49.056 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-05-03T14:23:49.056 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-05-03T14:23:49.056 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-05-03T14:23:49.056 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-05-03T14:23:49.057 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-05-03T14:23:49.061 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-05-03T14:23:49.061 DEBUG:teuthology.orchestra.run.smithi060:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-05-03T14:23:49.085 INFO:teuthology.orchestra.run.smithi060.stdout:postinstall scriptlet (using /bin/sh): 2024-05-03T14:23:49.085 INFO:teuthology.orchestra.run.smithi060.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-03T14:23:49.085 INFO:teuthology.orchestra.run.smithi060.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-05-03T14:23:49.085 INFO:teuthology.orchestra.run.smithi060.stdout:fi 2024-05-03T14:23:49.085 INFO:teuthology.orchestra.run.smithi060.stdout:for file in vmlinuz System.map config; do 2024-05-03T14:23:49.085 INFO:teuthology.orchestra.run.smithi060.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-05-03T14:23:49.085 INFO:teuthology.orchestra.run.smithi060.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-05-03T14:23:49.085 INFO:teuthology.orchestra.run.smithi060.stdout: fi 2024-05-03T14:23:49.085 INFO:teuthology.orchestra.run.smithi060.stdout:done 2024-05-03T14:23:49.085 INFO:teuthology.orchestra.run.smithi060.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-05-03T14:23:49.085 INFO:teuthology.orchestra.run.smithi060.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-05-03T14:23:49.086 INFO:teuthology.orchestra.run.smithi060.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-05-03T14:23:49.086 INFO:teuthology.orchestra.run.smithi060.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-05-03T14:23:49.086 INFO:teuthology.orchestra.run.smithi060.stdout:fi 2024-05-03T14:23:49.086 INFO:teuthology.orchestra.run.smithi060.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-05-03T14:23:49.086 INFO:teuthology.orchestra.run.smithi060.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-05-03T14:23:49.086 INFO:teuthology.orchestra.run.smithi060.stdout:fi 2024-05-03T14:23:49.086 INFO:teuthology.orchestra.run.smithi060.stdout:preuninstall scriptlet (using /bin/sh): 2024-05-03T14:23:49.086 INFO:teuthology.orchestra.run.smithi060.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-03T14:23:49.086 INFO:teuthology.orchestra.run.smithi060.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-05-03T14:23:49.086 INFO:teuthology.orchestra.run.smithi060.stdout:fi 2024-05-03T14:23:49.087 DEBUG:teuthology.orchestra.run.smithi060:> sudo rpm -qi grub2-tools 2024-05-03T14:23:49.169 INFO:teuthology.orchestra.run.smithi060.stdout:Name : grub2-tools 2024-05-03T14:23:49.169 INFO:teuthology.orchestra.run.smithi060.stdout:Epoch : 1 2024-05-03T14:23:49.170 INFO:teuthology.orchestra.run.smithi060.stdout:Version : 2.06 2024-05-03T14:23:49.170 INFO:teuthology.orchestra.run.smithi060.stdout:Release : 68.el9 2024-05-03T14:23:49.170 INFO:teuthology.orchestra.run.smithi060.stdout:Architecture: x86_64 2024-05-03T14:23:49.170 INFO:teuthology.orchestra.run.smithi060.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-03T14:23:49.170 INFO:teuthology.orchestra.run.smithi060.stdout:Group : Unspecified 2024-05-03T14:23:49.170 INFO:teuthology.orchestra.run.smithi060.stdout:Size : 8264254 2024-05-03T14:23:49.170 INFO:teuthology.orchestra.run.smithi060.stdout:License : GPLv3+ 2024-05-03T14:23:49.170 INFO:teuthology.orchestra.run.smithi060.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-03T14:23:49.170 INFO:teuthology.orchestra.run.smithi060.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-03T14:23:49.170 INFO:teuthology.orchestra.run.smithi060.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-03T14:23:49.170 INFO:teuthology.orchestra.run.smithi060.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-03T14:23:49.170 INFO:teuthology.orchestra.run.smithi060.stdout:Packager : builder@centos.org 2024-05-03T14:23:49.170 INFO:teuthology.orchestra.run.smithi060.stdout:Vendor : CentOS 2024-05-03T14:23:49.170 INFO:teuthology.orchestra.run.smithi060.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-03T14:23:49.171 INFO:teuthology.orchestra.run.smithi060.stdout:Summary : Support tools for GRUB. 2024-05-03T14:23:49.171 INFO:teuthology.orchestra.run.smithi060.stdout:Description : 2024-05-03T14:23:49.171 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:23:49.171 INFO:teuthology.orchestra.run.smithi060.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-03T14:23:49.171 INFO:teuthology.orchestra.run.smithi060.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-03T14:23:49.171 INFO:teuthology.orchestra.run.smithi060.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-03T14:23:49.171 INFO:teuthology.orchestra.run.smithi060.stdout:hardware devices. 2024-05-03T14:23:49.171 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:23:49.171 INFO:teuthology.orchestra.run.smithi060.stdout:This subpackage provides tools for support of all platforms. 2024-05-03T14:23:49.172 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-03T14:23:49.172 INFO:teuthology.task.kernel:Updating grub on smithi060 to boot 6.9.0-rc3-g806ec36b2112 2024-05-03T14:23:49.172 DEBUG:teuthology.orchestra.run.smithi060:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-03T14:23:49.764 INFO:teuthology.orchestra.run.smithi060.stderr:Generating grub configuration file ... 2024-05-03T14:23:51.109 INFO:teuthology.orchestra.run.smithi060.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-03T14:23:51.140 INFO:teuthology.orchestra.run.smithi060.stderr:done 2024-05-03T14:23:51.142 DEBUG:teuthology.orchestra.run.smithi060:> mktemp 2024-05-03T14:23:51.176 INFO:teuthology.orchestra.run.smithi060.stdout:/tmp/tmp.imAU6ctZka 2024-05-03T14:23:51.176 DEBUG:teuthology.orchestra.run.smithi060:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.imAU6ctZka 2024-05-03T14:23:51.242 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 0666 /tmp/tmp.imAU6ctZka 2024-05-03T14:23:51.388 DEBUG:teuthology.orchestra.remote:smithi060:/tmp/tmp.imAU6ctZka is 6KB 2024-05-03T14:23:51.410 DEBUG:teuthology.orchestra.run.smithi060:> rm -fr /tmp/tmp.imAU6ctZka 2024-05-03T14:23:51.425 DEBUG:teuthology.orchestra.run.smithi060:> sudo /bin/ls /boot/loader/entries || true 2024-05-03T14:23:51.490 INFO:teuthology.orchestra.run.smithi060.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-03T14:23:51.490 INFO:teuthology.orchestra.run.smithi060.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-03T14:23:51.490 INFO:teuthology.orchestra.run.smithi060.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-03T14:23:51.490 INFO:teuthology.orchestra.run.smithi060.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-05-03T14:23:51.491 DEBUG:teuthology.orchestra.run.smithi060:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-05-03T14:23:51.647 DEBUG:teuthology.orchestra.run.smithi060:> sudo shutdown -r now 2024-05-03T14:24:21.651 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-05-03T14:24:21.651 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi060.front.sepia.ceph.com' 2024-05-03T14:24:21.652 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-03T14:24:39.966 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.60 2024-05-03T14:24:48.968 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi060.front.sepia.ceph.com' 2024-05-03T14:24:48.969 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-03T14:25:48.971 DEBUG:teuthology.orchestra.remote:timed out 2024-05-03T14:26:00.972 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi060.front.sepia.ceph.com' 2024-05-03T14:26:00.972 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-03T14:26:01.339 DEBUG:teuthology.orchestra.run.smithi060:> true 2024-05-03T14:26:01.946 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi060.front.sepia.ceph.com' 2024-05-03T14:26:01.947 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-g806ec36b2112"... 2024-05-03T14:26:01.947 DEBUG:teuthology.orchestra.run.smithi060:> uname -r 2024-05-03T14:26:01.997 INFO:teuthology.orchestra.run.smithi060.stdout:6.9.0-rc3-g806ec36b2112 2024-05-03T14:26:01.998 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-05-03T14:26:01.998 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-03T14:26:01.998 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-03T14:26:02.999 DEBUG:teuthology.parallel:result is None 2024-05-03T14:26:02.999 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-03T14:26:03.006 INFO:teuthology.task.internal:Creating test directory... 2024-05-03T14:26:03.007 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-03T14:26:03.014 DEBUG:teuthology.orchestra.run.smithi090:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-03T14:26:03.017 DEBUG:teuthology.orchestra.run.smithi149:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-03T14:26:03.067 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-03T14:26:03.073 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-03T14:26:03.082 INFO:teuthology.task.internal:Creating archive directory... 2024-05-03T14:26:03.082 DEBUG:teuthology.orchestra.run.smithi060:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-03T14:26:03.087 DEBUG:teuthology.orchestra.run.smithi090:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-03T14:26:03.089 DEBUG:teuthology.orchestra.run.smithi149:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-03T14:26:03.190 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-03T14:26:03.196 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-03T14:26:03.196 DEBUG:teuthology.orchestra.run.smithi090:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-03T14:26:03.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T14:26:03.212 DEBUG:teuthology.orchestra.run.smithi149:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-03T14:26:03.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T14:26:03.229 DEBUG:teuthology.orchestra.run.smithi060:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-03T14:26:03.237 DEBUG:teuthology.orchestra.run.smithi090:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-03T14:26:03.255 DEBUG:teuthology.orchestra.run.smithi149:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-03T14:26:03.283 INFO:teuthology.orchestra.run.smithi090.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-03T14:26:03.319 INFO:teuthology.orchestra.run.smithi149.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-03T14:26:03.320 INFO:teuthology.orchestra.run.smithi090.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-03T14:26:03.337 INFO:teuthology.orchestra.run.smithi149.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-03T14:26:03.358 INFO:teuthology.orchestra.run.smithi060.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-03T14:26:03.415 INFO:teuthology.orchestra.run.smithi060.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-03T14:26:03.420 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-03T14:26:03.428 INFO:teuthology.task.internal:Configuring sudo... 2024-05-03T14:26:03.428 DEBUG:teuthology.orchestra.run.smithi060:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-03T14:26:03.435 DEBUG:teuthology.orchestra.run.smithi090:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-03T14:26:03.437 DEBUG:teuthology.orchestra.run.smithi149:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-03T14:26:03.519 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-03T14:26:03.527 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-03T14:26:03.527 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-03T14:26:03.534 DEBUG:teuthology.orchestra.run.smithi090:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-03T14:26:03.537 DEBUG:teuthology.orchestra.run.smithi149:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-03T14:26:03.585 DEBUG:teuthology.orchestra.run.smithi060:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-03T14:26:03.672 DEBUG:teuthology.orchestra.run.smithi060:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-03T14:26:03.777 DEBUG:teuthology.orchestra.run.smithi060:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-03T14:26:03.864 DEBUG:teuthology.orchestra.run.smithi060:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-03T14:26:03.955 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-05-03T14:26:03.955 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-03T14:26:04.042 DEBUG:teuthology.orchestra.run.smithi090:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-03T14:26:04.086 DEBUG:teuthology.orchestra.run.smithi090:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-03T14:26:04.138 DEBUG:teuthology.orchestra.run.smithi090:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-03T14:26:04.221 DEBUG:teuthology.orchestra.run.smithi090:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-03T14:26:04.263 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-05-03T14:26:04.263 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-03T14:26:04.329 DEBUG:teuthology.orchestra.run.smithi149:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-03T14:26:04.373 DEBUG:teuthology.orchestra.run.smithi149:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-03T14:26:04.430 DEBUG:teuthology.orchestra.run.smithi149:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-03T14:26:04.514 DEBUG:teuthology.orchestra.run.smithi149:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-03T14:26:04.557 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-03T14:26:04.557 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-03T14:26:04.623 DEBUG:teuthology.orchestra.run.smithi060:> sudo service rsyslog restart 2024-05-03T14:26:04.631 DEBUG:teuthology.orchestra.run.smithi090:> sudo service rsyslog restart 2024-05-03T14:26:04.633 DEBUG:teuthology.orchestra.run.smithi149:> sudo service rsyslog restart 2024-05-03T14:26:04.670 INFO:teuthology.orchestra.run.smithi090.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-03T14:26:04.702 INFO:teuthology.orchestra.run.smithi149.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-03T14:26:04.738 INFO:teuthology.orchestra.run.smithi060.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-03T14:26:05.218 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-03T14:26:05.225 INFO:teuthology.task.internal:Starting timer... 2024-05-03T14:26:05.225 INFO:teuthology.run_tasks:Running task pcp... 2024-05-03T14:26:05.250 INFO:teuthology.run_tasks:Running task selinux... 2024-05-03T14:26:05.257 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-05-03T14:26:05.258 DEBUG:teuthology.orchestra.run.smithi060:> sudo service auditd rotate 2024-05-03T14:26:05.381 INFO:teuthology.orchestra.run.smithi060.stdout:Rotating logs: 2024-05-03T14:26:05.386 DEBUG:teuthology.orchestra.run.smithi090:> sudo service auditd rotate 2024-05-03T14:26:05.474 INFO:teuthology.orchestra.run.smithi090.stdout:Rotating logs: 2024-05-03T14:26:05.475 DEBUG:teuthology.orchestra.run.smithi149:> sudo service auditd rotate 2024-05-03T14:26:05.534 INFO:teuthology.orchestra.run.smithi149.stdout:Rotating logs: 2024-05-03T14:26:05.536 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-03T14:26:05.536 DEBUG:teuthology.orchestra.run.smithi060:> /usr/sbin/getenforce 2024-05-03T14:26:05.594 INFO:teuthology.orchestra.run.smithi060.stdout:Permissive 2024-05-03T14:26:05.594 DEBUG:teuthology.orchestra.run.smithi090:> /usr/sbin/getenforce 2024-05-03T14:26:05.620 INFO:teuthology.orchestra.run.smithi090.stdout:Permissive 2024-05-03T14:26:05.620 DEBUG:teuthology.orchestra.run.smithi149:> /usr/sbin/getenforce 2024-05-03T14:26:05.654 INFO:teuthology.orchestra.run.smithi149.stdout:Permissive 2024-05-03T14:26:05.654 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi060.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi090.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi149.front.sepia.ceph.com': 'permissive'} 2024-05-03T14:26:05.654 DEBUG:teuthology.orchestra.run.smithi060:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-03T14:26:05.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T14:26:05.737 DEBUG:teuthology.orchestra.run.smithi090:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-03T14:26:05.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T14:26:05.765 DEBUG:teuthology.orchestra.run.smithi149:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-03T14:26:05.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T14:26:05.791 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-03T14:26:05.791 DEBUG:teuthology.orchestra.run.smithi060:> sudo /usr/sbin/setenforce permissive 2024-05-03T14:26:05.872 DEBUG:teuthology.orchestra.run.smithi090:> sudo /usr/sbin/setenforce permissive 2024-05-03T14:26:05.906 DEBUG:teuthology.orchestra.run.smithi149:> sudo /usr/sbin/setenforce permissive 2024-05-03T14:26:05.940 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-03T14:26:05.949 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-05-03T14:26:05.955 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-05-03T14:26:06.090 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-03T14:26:06.106 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-03T14:26:06.107 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi060.front.sepia.ceph.com,smithi090.front.sepia.ceph.com,smithi149.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-03T14:31:47.502 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi060.front.sepia.ceph.com'), Remote(name='ubuntu@smithi090.front.sepia.ceph.com'), Remote(name='ubuntu@smithi149.front.sepia.ceph.com')] 2024-05-03T14:31:47.503 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi060.front.sepia.ceph.com' 2024-05-03T14:31:47.504 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-03T14:31:47.615 DEBUG:teuthology.orchestra.run.smithi060:> true 2024-05-03T14:31:47.774 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi060.front.sepia.ceph.com' 2024-05-03T14:31:47.775 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-05-03T14:31:47.775 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-03T14:31:47.851 DEBUG:teuthology.orchestra.run.smithi090:> true 2024-05-03T14:31:47.925 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-05-03T14:31:47.925 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-05-03T14:31:47.926 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-03T14:31:47.998 DEBUG:teuthology.orchestra.run.smithi149:> true 2024-05-03T14:31:48.077 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-05-03T14:31:48.077 INFO:teuthology.run_tasks:Running task clock... 2024-05-03T14:31:48.089 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-03T14:31:48.089 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-03T14:31:48.090 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-03T14:31:48.097 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-03T14:31:48.097 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-03T14:31:48.099 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-03T14:31:48.099 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-03T14:31:48.126 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-03T14:31:48.139 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-03T14:31:48.146 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-03T14:31:48.159 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-03T14:31:48.163 INFO:teuthology.orchestra.run.smithi090.stderr:sudo: ntpd: command not found 2024-05-03T14:31:48.173 INFO:teuthology.orchestra.run.smithi090.stdout:506 Cannot talk to daemon 2024-05-03T14:31:48.183 INFO:teuthology.orchestra.run.smithi149.stderr:sudo: ntpd: command not found 2024-05-03T14:31:48.186 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-03T14:31:48.190 INFO:teuthology.orchestra.run.smithi060.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-03T14:31:48.193 INFO:teuthology.orchestra.run.smithi149.stdout:506 Cannot talk to daemon 2024-05-03T14:31:48.198 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-03T14:31:48.206 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-03T14:31:48.220 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-03T14:31:48.236 INFO:teuthology.orchestra.run.smithi060.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-03T14:31:48.250 INFO:teuthology.orchestra.run.smithi090.stderr:bash: line 1: ntpq: command not found 2024-05-03T14:31:48.253 INFO:teuthology.orchestra.run.smithi090.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-03T14:31:48.253 INFO:teuthology.orchestra.run.smithi090.stdout:=============================================================================== 2024-05-03T14:31:48.253 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T14:31:48.253 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T14:31:48.253 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T14:31:48.254 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T14:31:48.280 INFO:teuthology.orchestra.run.smithi149.stderr:bash: line 1: ntpq: command not found 2024-05-03T14:31:48.284 INFO:teuthology.orchestra.run.smithi149.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-03T14:31:48.284 INFO:teuthology.orchestra.run.smithi149.stdout:=============================================================================== 2024-05-03T14:31:48.284 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T14:31:48.284 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T14:31:48.284 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T14:31:48.284 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T14:31:48.312 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: ntpd: command not found 2024-05-03T14:31:48.348 INFO:teuthology.orchestra.run.smithi060.stdout:506 Cannot talk to daemon 2024-05-03T14:31:48.393 INFO:teuthology.orchestra.run.smithi060.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-03T14:31:48.438 INFO:teuthology.orchestra.run.smithi060.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-03T14:31:48.585 INFO:teuthology.orchestra.run.smithi060.stderr:bash: line 1: ntpq: command not found 2024-05-03T14:31:48.593 INFO:teuthology.orchestra.run.smithi060.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-03T14:31:48.593 INFO:teuthology.orchestra.run.smithi060.stdout:=============================================================================== 2024-05-03T14:31:48.593 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T14:31:48.593 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T14:31:48.594 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T14:31:48.594 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T14:31:48.595 INFO:teuthology.run_tasks:Running task install... 2024-05-03T14:31:48.604 DEBUG:teuthology.task.install:project ceph 2024-05-03T14:31:48.604 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '9a047fe7599a549042d06a3fb5b6813a203eabea'}} 2024-05-03T14:31:48.604 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '9a047fe7599a549042d06a3fb5b6813a203eabea'} 2024-05-03T14:31:48.604 INFO:teuthology.task.install:Using flavor: default 2024-05-03T14:31:48.610 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-03T14:31:48.610 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-03T14:31:48.611 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9a047fe7599a549042d06a3fb5b6813a203eabea', 'tag': None, 'wait_for_package': False} 2024-05-03T14:31:48.611 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9a047fe7599a549042d06a3fb5b6813a203eabea 2024-05-03T14:31:48.613 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9a047fe7599a549042d06a3fb5b6813a203eabea', 'tag': None, 'wait_for_package': False} 2024-05-03T14:31:48.614 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9a047fe7599a549042d06a3fb5b6813a203eabea 2024-05-03T14:31:48.615 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9a047fe7599a549042d06a3fb5b6813a203eabea', 'tag': None, 'wait_for_package': False} 2024-05-03T14:31:48.615 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9a047fe7599a549042d06a3fb5b6813a203eabea 2024-05-03T14:31:48.766 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/ 2024-05-03T14:31:48.766 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3450.g9a047fe7 2024-05-03T14:31:48.769 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/ 2024-05-03T14:31:48.769 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3450.g9a047fe7 2024-05-03T14:31:48.840 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/ 2024-05-03T14:31:48.841 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3450.g9a047fe7 2024-05-03T14:31:48.880 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-03T14:31:48.880 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-05-03T14:31:48.880 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-03T14:31:48.946 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-03T14:31:48.946 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-05-03T14:31:48.946 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-03T14:31:48.951 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-03T14:31:48.951 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-03T14:31:48.951 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-03T14:31:48.973 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-03T14:31:48.973 DEBUG:teuthology.orchestra.run.smithi090:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9a047fe7599a549042d06a3fb5b6813a203eabea/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-03T14:31:48.975 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-03T14:31:48.975 DEBUG:teuthology.orchestra.run.smithi060:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9a047fe7599a549042d06a3fb5b6813a203eabea/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-03T14:31:48.978 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-03T14:31:48.978 DEBUG:teuthology.orchestra.run.smithi149:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9a047fe7599a549042d06a3fb5b6813a203eabea/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-03T14:31:49.039 DEBUG:teuthology.orchestra.run.smithi090:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-03T14:31:49.043 DEBUG:teuthology.orchestra.run.smithi149:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-03T14:31:49.108 DEBUG:teuthology.orchestra.run.smithi060:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-03T14:31:49.113 DEBUG:teuthology.orchestra.run.smithi090:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-03T14:31:49.118 DEBUG:teuthology.orchestra.run.smithi149:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-03T14:31:49.176 INFO:teuthology.orchestra.run.smithi090.stdout:check_obsoletes = 1 2024-05-03T14:31:49.178 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum clean all 2024-05-03T14:31:49.182 INFO:teuthology.orchestra.run.smithi149.stdout:check_obsoletes = 1 2024-05-03T14:31:49.183 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum clean all 2024-05-03T14:31:49.224 DEBUG:teuthology.orchestra.run.smithi060:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-03T14:31:49.308 INFO:teuthology.orchestra.run.smithi060.stdout:check_obsoletes = 1 2024-05-03T14:31:49.313 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum clean all 2024-05-03T14:31:49.404 INFO:teuthology.orchestra.run.smithi090.stdout:50 files removed 2024-05-03T14:31:49.412 INFO:teuthology.orchestra.run.smithi149.stdout:50 files removed 2024-05-03T14:31:49.431 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-03T14:31:49.440 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-03T14:31:49.642 INFO:teuthology.orchestra.run.smithi060.stdout:50 files removed 2024-05-03T14:31:49.682 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-03T14:31:50.049 INFO:teuthology.orchestra.run.smithi149.stdout:ceph packages for x86_64 241 kB/s | 84 kB 00:00 2024-05-03T14:31:50.188 INFO:teuthology.orchestra.run.smithi090.stdout:ceph packages for x86_64 167 kB/s | 84 kB 00:00 2024-05-03T14:31:50.614 INFO:teuthology.orchestra.run.smithi090.stdout:ceph noarch packages 31 kB/s | 12 kB 00:00 2024-05-03T14:31:50.710 INFO:teuthology.orchestra.run.smithi060.stdout:ceph packages for x86_64 147 kB/s | 84 kB 00:00 2024-05-03T14:31:51.032 INFO:teuthology.orchestra.run.smithi090.stdout:ceph source packages 4.9 kB/s | 1.9 kB 00:00 2024-05-03T14:31:51.287 INFO:teuthology.orchestra.run.smithi060.stdout:ceph noarch packages 25 kB/s | 12 kB 00:00 2024-05-03T14:31:51.777 INFO:teuthology.orchestra.run.smithi149.stdout:ceph noarch packages 7.0 kB/s | 12 kB 00:01 2024-05-03T14:31:51.896 INFO:teuthology.orchestra.run.smithi060.stdout:ceph source packages 3.6 kB/s | 1.9 kB 00:00 2024-05-03T14:31:52.219 INFO:teuthology.orchestra.run.smithi149.stdout:ceph source packages 4.7 kB/s | 1.9 kB 00:00 2024-05-03T14:31:52.699 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - BaseOS 4.9 MB/s | 8.0 MB 00:01 2024-05-03T14:31:52.810 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - BaseOS 9.7 MB/s | 8.0 MB 00:00 2024-05-03T14:31:54.302 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - BaseOS 3.9 MB/s | 8.0 MB 00:02 2024-05-03T14:31:54.687 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-05-03T14:31:56.704 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - AppStream 7.2 MB/s | 19 MB 00:02 2024-05-03T14:31:59.094 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - CRB 7.0 MB/s | 6.2 MB 00:00 2024-05-03T14:32:00.628 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - Extras packages 41 kB/s | 16 kB 00:00 2024-05-03T14:32:01.875 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - CRB 5.3 MB/s | 6.2 MB 00:01 2024-05-03T14:32:02.634 INFO:teuthology.orchestra.run.smithi090.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-05-03T14:32:04.588 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - Extras packages 12 kB/s | 16 kB 00:01 2024-05-03T14:32:05.535 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - AppStream 1.9 MB/s | 19 MB 00:10 2024-05-03T14:32:06.115 INFO:teuthology.orchestra.run.smithi060.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-05-03T14:32:07.582 INFO:teuthology.orchestra.run.smithi090.stdout:lab-extras 36 kB/s | 1.7 kB 00:00 2024-05-03T14:32:09.709 INFO:teuthology.orchestra.run.smithi090.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-03T14:32:09.710 INFO:teuthology.orchestra.run.smithi090.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-03T14:32:09.721 INFO:teuthology.orchestra.run.smithi090.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-03T14:32:09.722 INFO:teuthology.orchestra.run.smithi090.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-03T14:32:09.723 INFO:teuthology.orchestra.run.smithi090.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-03T14:32:09.724 INFO:teuthology.orchestra.run.smithi090.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-03T14:32:09.725 INFO:teuthology.orchestra.run.smithi090.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-03T14:32:09.730 INFO:teuthology.orchestra.run.smithi090.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-03T14:32:09.730 INFO:teuthology.orchestra.run.smithi090.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-03T14:32:09.789 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-05-03T14:32:09.815 INFO:teuthology.orchestra.run.smithi090.stdout:======================================================================================= 2024-05-03T14:32:09.815 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repository Size 2024-05-03T14:32:09.815 INFO:teuthology.orchestra.run.smithi090.stdout:======================================================================================= 2024-05-03T14:32:09.815 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-05-03T14:32:09.815 INFO:teuthology.orchestra.run.smithi090.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-03T14:32:09.815 INFO:teuthology.orchestra.run.smithi090.stdout: ceph x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 6.6 k 2024-05-03T14:32:09.816 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-base x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 5.4 M 2024-05-03T14:32:09.816 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-fuse x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 891 k 2024-05-03T14:32:09.816 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 145 k 2024-05-03T14:32:09.816 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 1.5 M 2024-05-03T14:32:09.816 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 142 k 2024-05-03T14:32:09.816 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 3.5 M 2024-05-03T14:32:09.816 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 7.4 M 2024-05-03T14:32:09.816 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-rook noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 49 k 2024-05-03T14:32:09.816 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-radosgw x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 11 M 2024-05-03T14:32:09.816 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-test x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 49 M 2024-05-03T14:32:09.816 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-volume noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 273 k 2024-05-03T14:32:09.816 INFO:teuthology.orchestra.run.smithi090.stdout: cephadm noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 763 k 2024-05-03T14:32:09.816 INFO:teuthology.orchestra.run.smithi090.stdout: cephfs-mirror x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 225 k 2024-05-03T14:32:09.816 INFO:teuthology.orchestra.run.smithi090.stdout: cephfs-shell x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 27 k 2024-05-03T14:32:09.817 INFO:teuthology.orchestra.run.smithi090.stdout: cephfs-top noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 18 k 2024-05-03T14:32:09.817 INFO:teuthology.orchestra.run.smithi090.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-03T14:32:09.817 INFO:teuthology.orchestra.run.smithi090.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-03T14:32:09.817 INFO:teuthology.orchestra.run.smithi090.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-03T14:32:09.817 INFO:teuthology.orchestra.run.smithi090.stdout: libcephfs-devel x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 32 k 2024-05-03T14:32:09.817 INFO:teuthology.orchestra.run.smithi090.stdout: libcephfs2 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 747 k 2024-05-03T14:32:09.817 INFO:teuthology.orchestra.run.smithi090.stdout: librados-devel x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 127 k 2024-05-03T14:32:09.817 INFO:teuthology.orchestra.run.smithi090.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-03T14:32:09.817 INFO:teuthology.orchestra.run.smithi090.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-03T14:32:09.817 INFO:teuthology.orchestra.run.smithi090.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-03T14:32:09.817 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-03T14:32:09.817 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-03T14:32:09.817 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-03T14:32:09.817 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cephfs x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 161 k 2024-05-03T14:32:09.818 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rados x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 320 k 2024-05-03T14:32:09.818 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rbd x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 298 k 2024-05-03T14:32:09.818 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rgw x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 100 k 2024-05-03T14:32:09.818 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-fuse x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 85 k 2024-05-03T14:32:09.818 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-mirror x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 3.1 M 2024-05-03T14:32:09.818 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-nbd x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 170 k 2024-05-03T14:32:09.818 INFO:teuthology.orchestra.run.smithi090.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-03T14:32:09.818 INFO:teuthology.orchestra.run.smithi090.stdout:Upgrading: 2024-05-03T14:32:09.818 INFO:teuthology.orchestra.run.smithi090.stdout: librados2 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 3.4 M 2024-05-03T14:32:09.818 INFO:teuthology.orchestra.run.smithi090.stdout: librbd1 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 3.1 M 2024-05-03T14:32:09.818 INFO:teuthology.orchestra.run.smithi090.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-03T14:32:09.818 INFO:teuthology.orchestra.run.smithi090.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-03T14:32:09.819 INFO:teuthology.orchestra.run.smithi090.stdout:Installing dependencies: 2024-05-03T14:32:09.819 INFO:teuthology.orchestra.run.smithi090.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-03T14:32:09.819 INFO:teuthology.orchestra.run.smithi090.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-03T14:32:09.819 INFO:teuthology.orchestra.run.smithi090.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-03T14:32:09.819 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-common x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 21 M 2024-05-03T14:32:09.819 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 34 k 2024-05-03T14:32:09.819 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mds x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 2.4 M 2024-05-03T14:32:09.819 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 274 k 2024-05-03T14:32:09.819 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mon x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 4.7 M 2024-05-03T14:32:09.819 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-osd x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 17 M 2024-05-03T14:32:09.819 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 16 k 2024-05-03T14:32:09.819 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-selinux x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 25 k 2024-05-03T14:32:09.819 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-03T14:32:09.819 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-03T14:32:09.819 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-03T14:32:09.819 INFO:teuthology.orchestra.run.smithi090.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-03T14:32:09.820 INFO:teuthology.orchestra.run.smithi090.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-03T14:32:09.820 INFO:teuthology.orchestra.run.smithi090.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-03T14:32:09.820 INFO:teuthology.orchestra.run.smithi090.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-03T14:32:09.820 INFO:teuthology.orchestra.run.smithi090.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-03T14:32:09.820 INFO:teuthology.orchestra.run.smithi090.stdout: libcephsqlite x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 163 k 2024-05-03T14:32:09.820 INFO:teuthology.orchestra.run.smithi090.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-03T14:32:09.820 INFO:teuthology.orchestra.run.smithi090.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-03T14:32:09.820 INFO:teuthology.orchestra.run.smithi090.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-03T14:32:09.820 INFO:teuthology.orchestra.run.smithi090.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-03T14:32:09.820 INFO:teuthology.orchestra.run.smithi090.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-03T14:32:09.820 INFO:teuthology.orchestra.run.smithi090.stdout: libradosstriper1 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 500 k 2024-05-03T14:32:09.820 INFO:teuthology.orchestra.run.smithi090.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-03T14:32:09.820 INFO:teuthology.orchestra.run.smithi090.stdout: librgw2 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 5.4 M 2024-05-03T14:32:09.820 INFO:teuthology.orchestra.run.smithi090.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-03T14:32:09.821 INFO:teuthology.orchestra.run.smithi090.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-03T14:32:09.821 INFO:teuthology.orchestra.run.smithi090.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-03T14:32:09.821 INFO:teuthology.orchestra.run.smithi090.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-03T14:32:09.821 INFO:teuthology.orchestra.run.smithi090.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-03T14:32:09.821 INFO:teuthology.orchestra.run.smithi090.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-03T14:32:09.821 INFO:teuthology.orchestra.run.smithi090.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-03T14:32:09.821 INFO:teuthology.orchestra.run.smithi090.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-03T14:32:09.821 INFO:teuthology.orchestra.run.smithi090.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-03T14:32:09.821 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-03T14:32:09.821 INFO:teuthology.orchestra.run.smithi090.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-03T14:32:09.821 INFO:teuthology.orchestra.run.smithi090.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-03T14:32:09.821 INFO:teuthology.orchestra.run.smithi090.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-03T14:32:09.821 INFO:teuthology.orchestra.run.smithi090.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-03T14:32:09.821 INFO:teuthology.orchestra.run.smithi090.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-03T14:32:09.821 INFO:teuthology.orchestra.run.smithi090.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-03T14:32:09.822 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-03T14:32:09.822 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-argparse x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 45 k 2024-05-03T14:32:09.822 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-common x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 135 k 2024-05-03T14:32:09.822 INFO:teuthology.orchestra.run.smithi090.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-03T14:32:09.822 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-03T14:32:09.822 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-03T14:32:09.822 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-03T14:32:09.822 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-03T14:32:09.822 INFO:teuthology.orchestra.run.smithi090.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-03T14:32:09.822 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-03T14:32:09.822 INFO:teuthology.orchestra.run.smithi090.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-03T14:32:09.822 INFO:teuthology.orchestra.run.smithi090.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-03T14:32:09.822 INFO:teuthology.orchestra.run.smithi090.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-03T14:32:09.822 INFO:teuthology.orchestra.run.smithi090.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-03T14:32:09.822 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-03T14:32:09.822 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-03T14:32:09.823 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-03T14:32:09.823 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-03T14:32:09.823 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-03T14:32:09.823 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-03T14:32:09.823 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-03T14:32:09.823 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-03T14:32:09.823 INFO:teuthology.orchestra.run.smithi090.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-03T14:32:09.823 INFO:teuthology.orchestra.run.smithi090.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-03T14:32:09.823 INFO:teuthology.orchestra.run.smithi090.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-03T14:32:09.823 INFO:teuthology.orchestra.run.smithi090.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-03T14:32:09.823 INFO:teuthology.orchestra.run.smithi090.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-03T14:32:09.823 INFO:teuthology.orchestra.run.smithi090.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-03T14:32:09.823 INFO:teuthology.orchestra.run.smithi090.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-03T14:32:09.823 INFO:teuthology.orchestra.run.smithi090.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-03T14:32:09.823 INFO:teuthology.orchestra.run.smithi090.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-03T14:32:09.824 INFO:teuthology.orchestra.run.smithi090.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-03T14:32:09.824 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-03T14:32:09.824 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-03T14:32:09.824 INFO:teuthology.orchestra.run.smithi090.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-03T14:32:09.824 INFO:teuthology.orchestra.run.smithi090.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-03T14:32:09.824 INFO:teuthology.orchestra.run.smithi090.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-03T14:32:09.824 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-03T14:32:09.824 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-03T14:32:09.824 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-03T14:32:09.824 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-03T14:32:09.824 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-03T14:32:09.824 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-03T14:32:09.824 INFO:teuthology.orchestra.run.smithi090.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-03T14:32:09.824 INFO:teuthology.orchestra.run.smithi090.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-03T14:32:09.824 INFO:teuthology.orchestra.run.smithi090.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-03T14:32:09.825 INFO:teuthology.orchestra.run.smithi090.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-03T14:32:09.825 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-03T14:32:09.825 INFO:teuthology.orchestra.run.smithi090.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-03T14:32:09.825 INFO:teuthology.orchestra.run.smithi090.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-03T14:32:09.825 INFO:teuthology.orchestra.run.smithi090.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-03T14:32:09.825 INFO:teuthology.orchestra.run.smithi090.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-03T14:32:09.825 INFO:teuthology.orchestra.run.smithi090.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-03T14:32:09.825 INFO:teuthology.orchestra.run.smithi090.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-03T14:32:09.825 INFO:teuthology.orchestra.run.smithi090.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-03T14:32:09.825 INFO:teuthology.orchestra.run.smithi090.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-03T14:32:09.825 INFO:teuthology.orchestra.run.smithi090.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-03T14:32:09.825 INFO:teuthology.orchestra.run.smithi090.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-03T14:32:09.825 INFO:teuthology.orchestra.run.smithi090.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-03T14:32:09.825 INFO:teuthology.orchestra.run.smithi090.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-03T14:32:09.825 INFO:teuthology.orchestra.run.smithi090.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-03T14:32:09.825 INFO:teuthology.orchestra.run.smithi090.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-03T14:32:09.826 INFO:teuthology.orchestra.run.smithi090.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-03T14:32:09.826 INFO:teuthology.orchestra.run.smithi090.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-03T14:32:09.826 INFO:teuthology.orchestra.run.smithi090.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-03T14:32:09.826 INFO:teuthology.orchestra.run.smithi090.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-03T14:32:09.826 INFO:teuthology.orchestra.run.smithi090.stdout:Installing weak dependencies: 2024-05-03T14:32:09.826 INFO:teuthology.orchestra.run.smithi090.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-03T14:32:09.826 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-03T14:32:09.826 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-05-03T14:32:09.826 INFO:teuthology.orchestra.run.smithi090.stdout:======================================================================================= 2024-05-03T14:32:09.826 INFO:teuthology.orchestra.run.smithi090.stdout:Install 148 Packages 2024-05-03T14:32:09.826 INFO:teuthology.orchestra.run.smithi090.stdout:Upgrade 4 Packages 2024-05-03T14:32:09.826 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-03T14:32:09.827 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 222 M 2024-05-03T14:32:09.827 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-05-03T14:32:10.427 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - CRB 4.6 MB/s | 6.2 MB 00:01 2024-05-03T14:32:10.509 INFO:teuthology.orchestra.run.smithi090.stdout:(1/152): ceph-19.0.0-3450.g9a047fe7.el9.x86_64. 28 kB/s | 6.6 kB 00:00 2024-05-03T14:32:10.826 INFO:teuthology.orchestra.run.smithi090.stdout:(2/152): ceph-base-19.0.0-3450.g9a047fe7.el9.x8 9.8 MB/s | 5.4 MB 00:00 2024-05-03T14:32:10.876 INFO:teuthology.orchestra.run.smithi090.stdout:(3/152): ceph-immutable-object-cache-19.0.0-345 2.8 MB/s | 145 kB 00:00 2024-05-03T14:32:10.976 INFO:teuthology.orchestra.run.smithi090.stdout:(4/152): ceph-fuse-19.0.0-3450.g9a047fe7.el9.x8 1.9 MB/s | 891 kB 00:00 2024-05-03T14:32:11.085 INFO:teuthology.orchestra.run.smithi090.stdout:(5/152): ceph-mds-19.0.0-3450.g9a047fe7.el9.x86 11 MB/s | 2.4 MB 00:00 2024-05-03T14:32:11.402 INFO:teuthology.orchestra.run.smithi090.stdout:(6/152): ceph-mon-19.0.0-3450.g9a047fe7.el9.x86 15 MB/s | 4.7 MB 00:00 2024-05-03T14:32:11.444 INFO:teuthology.orchestra.run.smithi090.stdout:(7/152): ceph-mgr-19.0.0-3450.g9a047fe7.el9.x86 3.2 MB/s | 1.5 MB 00:00 2024-05-03T14:32:11.987 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 16 kB 00:00 2024-05-03T14:32:12.298 INFO:teuthology.orchestra.run.smithi060.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-05-03T14:32:12.561 INFO:teuthology.orchestra.run.smithi090.stdout:(8/152): ceph-osd-19.0.0-3450.g9a047fe7.el9.x86 15 MB/s | 17 MB 00:01 2024-05-03T14:32:12.613 INFO:teuthology.orchestra.run.smithi090.stdout:(9/152): ceph-selinux-19.0.0-3450.g9a047fe7.el9 505 kB/s | 25 kB 00:00 2024-05-03T14:32:13.004 INFO:teuthology.orchestra.run.smithi090.stdout:(10/152): ceph-radosgw-19.0.0-3450.g9a047fe7.el 6.8 MB/s | 11 MB 00:01 2024-05-03T14:32:13.112 INFO:teuthology.orchestra.run.smithi090.stdout:(11/152): cephfs-mirror-19.0.0-3450.g9a047fe7.e 2.0 MB/s | 225 kB 00:00 2024-05-03T14:32:13.196 INFO:teuthology.orchestra.run.smithi090.stdout:(12/152): cephfs-shell-19.0.0-3450.g9a047fe7.el 325 kB/s | 27 kB 00:00 2024-05-03T14:32:13.279 INFO:teuthology.orchestra.run.smithi090.stdout:(13/152): libcephfs-devel-19.0.0-3450.g9a047fe7 380 kB/s | 32 kB 00:00 2024-05-03T14:32:13.438 INFO:teuthology.orchestra.run.smithi090.stdout:(14/152): libcephfs2-19.0.0-3450.g9a047fe7.el9. 4.6 MB/s | 747 kB 00:00 2024-05-03T14:32:13.465 INFO:teuthology.orchestra.run.smithi149.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-05-03T14:32:13.531 INFO:teuthology.orchestra.run.smithi090.stdout:(15/152): libcephsqlite-19.0.0-3450.g9a047fe7.e 1.7 MB/s | 163 kB 00:00 2024-05-03T14:32:13.624 INFO:teuthology.orchestra.run.smithi090.stdout:(16/152): librados-devel-19.0.0-3450.g9a047fe7. 1.3 MB/s | 127 kB 00:00 2024-05-03T14:32:13.724 INFO:teuthology.orchestra.run.smithi090.stdout:(17/152): libradosstriper1-19.0.0-3450.g9a047fe 4.9 MB/s | 500 kB 00:00 2024-05-03T14:32:14.441 INFO:teuthology.orchestra.run.smithi090.stdout:(18/152): librgw2-19.0.0-3450.g9a047fe7.el9.x86 7.5 MB/s | 5.4 MB 00:00 2024-05-03T14:32:14.516 INFO:teuthology.orchestra.run.smithi090.stdout:(19/152): python3-ceph-argparse-19.0.0-3450.g9a 601 kB/s | 45 kB 00:00 2024-05-03T14:32:14.616 INFO:teuthology.orchestra.run.smithi090.stdout:(20/152): python3-ceph-common-19.0.0-3450.g9a04 1.3 MB/s | 135 kB 00:00 2024-05-03T14:32:14.717 INFO:teuthology.orchestra.run.smithi090.stdout:(21/152): python3-cephfs-19.0.0-3450.g9a047fe7. 1.6 MB/s | 161 kB 00:00 2024-05-03T14:32:14.818 INFO:teuthology.orchestra.run.smithi060.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-03T14:32:14.820 INFO:teuthology.orchestra.run.smithi060.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-03T14:32:14.834 INFO:teuthology.orchestra.run.smithi090.stdout:(22/152): python3-rados-19.0.0-3450.g9a047fe7.e 2.7 MB/s | 320 kB 00:00 2024-05-03T14:32:14.841 INFO:teuthology.orchestra.run.smithi060.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-03T14:32:14.843 INFO:teuthology.orchestra.run.smithi060.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-03T14:32:14.844 INFO:teuthology.orchestra.run.smithi060.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-03T14:32:14.846 INFO:teuthology.orchestra.run.smithi060.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-03T14:32:14.848 INFO:teuthology.orchestra.run.smithi060.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-03T14:32:14.852 INFO:teuthology.orchestra.run.smithi060.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-03T14:32:14.853 INFO:teuthology.orchestra.run.smithi060.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-03T14:32:14.930 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-05-03T14:32:14.934 INFO:teuthology.orchestra.run.smithi090.stdout:(23/152): python3-rbd-19.0.0-3450.g9a047fe7.el9 2.9 MB/s | 298 kB 00:00 2024-05-03T14:32:14.946 INFO:teuthology.orchestra.run.smithi060.stdout:======================================================================================= 2024-05-03T14:32:14.947 INFO:teuthology.orchestra.run.smithi060.stdout: Package Arch Version Repository Size 2024-05-03T14:32:14.947 INFO:teuthology.orchestra.run.smithi060.stdout:======================================================================================= 2024-05-03T14:32:14.947 INFO:teuthology.orchestra.run.smithi060.stdout:Installing: 2024-05-03T14:32:14.947 INFO:teuthology.orchestra.run.smithi060.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-03T14:32:14.947 INFO:teuthology.orchestra.run.smithi060.stdout: ceph x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 6.6 k 2024-05-03T14:32:14.947 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-base x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 5.4 M 2024-05-03T14:32:14.947 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-fuse x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 891 k 2024-05-03T14:32:14.947 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 145 k 2024-05-03T14:32:14.947 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 1.5 M 2024-05-03T14:32:14.947 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 142 k 2024-05-03T14:32:14.947 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 3.5 M 2024-05-03T14:32:14.947 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 7.4 M 2024-05-03T14:32:14.947 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-rook noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 49 k 2024-05-03T14:32:14.947 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-radosgw x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 11 M 2024-05-03T14:32:14.948 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-test x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 49 M 2024-05-03T14:32:14.948 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-volume noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 273 k 2024-05-03T14:32:14.948 INFO:teuthology.orchestra.run.smithi060.stdout: cephadm noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 763 k 2024-05-03T14:32:14.948 INFO:teuthology.orchestra.run.smithi060.stdout: cephfs-mirror x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 225 k 2024-05-03T14:32:14.948 INFO:teuthology.orchestra.run.smithi060.stdout: cephfs-shell x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 27 k 2024-05-03T14:32:14.948 INFO:teuthology.orchestra.run.smithi060.stdout: cephfs-top noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 18 k 2024-05-03T14:32:14.948 INFO:teuthology.orchestra.run.smithi060.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-03T14:32:14.948 INFO:teuthology.orchestra.run.smithi060.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-03T14:32:14.948 INFO:teuthology.orchestra.run.smithi060.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-03T14:32:14.948 INFO:teuthology.orchestra.run.smithi060.stdout: libcephfs-devel x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 32 k 2024-05-03T14:32:14.948 INFO:teuthology.orchestra.run.smithi060.stdout: libcephfs2 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 747 k 2024-05-03T14:32:14.948 INFO:teuthology.orchestra.run.smithi060.stdout: librados-devel x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 127 k 2024-05-03T14:32:14.948 INFO:teuthology.orchestra.run.smithi060.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-03T14:32:14.948 INFO:teuthology.orchestra.run.smithi060.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-03T14:32:14.948 INFO:teuthology.orchestra.run.smithi060.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-03T14:32:14.949 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-03T14:32:14.949 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-03T14:32:14.949 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-03T14:32:14.949 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cephfs x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 161 k 2024-05-03T14:32:14.949 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rados x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 320 k 2024-05-03T14:32:14.949 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rbd x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 298 k 2024-05-03T14:32:14.949 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rgw x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 100 k 2024-05-03T14:32:14.949 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-fuse x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 85 k 2024-05-03T14:32:14.949 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-mirror x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 3.1 M 2024-05-03T14:32:14.949 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-nbd x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 170 k 2024-05-03T14:32:14.949 INFO:teuthology.orchestra.run.smithi060.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-03T14:32:14.949 INFO:teuthology.orchestra.run.smithi060.stdout:Upgrading: 2024-05-03T14:32:14.949 INFO:teuthology.orchestra.run.smithi060.stdout: librados2 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 3.4 M 2024-05-03T14:32:14.950 INFO:teuthology.orchestra.run.smithi060.stdout: librbd1 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 3.1 M 2024-05-03T14:32:14.950 INFO:teuthology.orchestra.run.smithi060.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-03T14:32:14.950 INFO:teuthology.orchestra.run.smithi060.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-03T14:32:14.950 INFO:teuthology.orchestra.run.smithi060.stdout:Installing dependencies: 2024-05-03T14:32:14.950 INFO:teuthology.orchestra.run.smithi060.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-03T14:32:14.950 INFO:teuthology.orchestra.run.smithi060.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-03T14:32:14.950 INFO:teuthology.orchestra.run.smithi060.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-03T14:32:14.950 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-common x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 21 M 2024-05-03T14:32:14.950 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 34 k 2024-05-03T14:32:14.950 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mds x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 2.4 M 2024-05-03T14:32:14.950 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 274 k 2024-05-03T14:32:14.950 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mon x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 4.7 M 2024-05-03T14:32:14.950 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-osd x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 17 M 2024-05-03T14:32:14.950 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 16 k 2024-05-03T14:32:14.951 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-selinux x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 25 k 2024-05-03T14:32:14.951 INFO:teuthology.orchestra.run.smithi060.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-03T14:32:14.951 INFO:teuthology.orchestra.run.smithi060.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-03T14:32:14.951 INFO:teuthology.orchestra.run.smithi060.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-03T14:32:14.951 INFO:teuthology.orchestra.run.smithi060.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-03T14:32:14.951 INFO:teuthology.orchestra.run.smithi060.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-03T14:32:14.951 INFO:teuthology.orchestra.run.smithi060.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-03T14:32:14.951 INFO:teuthology.orchestra.run.smithi060.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-03T14:32:14.951 INFO:teuthology.orchestra.run.smithi060.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-03T14:32:14.951 INFO:teuthology.orchestra.run.smithi060.stdout: libcephsqlite x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 163 k 2024-05-03T14:32:14.951 INFO:teuthology.orchestra.run.smithi060.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-03T14:32:14.951 INFO:teuthology.orchestra.run.smithi060.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-03T14:32:14.951 INFO:teuthology.orchestra.run.smithi060.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-03T14:32:14.951 INFO:teuthology.orchestra.run.smithi060.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-03T14:32:14.951 INFO:teuthology.orchestra.run.smithi060.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-03T14:32:14.951 INFO:teuthology.orchestra.run.smithi060.stdout: libradosstriper1 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 500 k 2024-05-03T14:32:14.951 INFO:teuthology.orchestra.run.smithi060.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-03T14:32:14.952 INFO:teuthology.orchestra.run.smithi060.stdout: librgw2 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 5.4 M 2024-05-03T14:32:14.952 INFO:teuthology.orchestra.run.smithi060.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-03T14:32:14.952 INFO:teuthology.orchestra.run.smithi060.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-03T14:32:14.952 INFO:teuthology.orchestra.run.smithi060.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-03T14:32:14.952 INFO:teuthology.orchestra.run.smithi060.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-03T14:32:14.952 INFO:teuthology.orchestra.run.smithi060.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-03T14:32:14.952 INFO:teuthology.orchestra.run.smithi060.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-03T14:32:14.952 INFO:teuthology.orchestra.run.smithi060.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-03T14:32:14.952 INFO:teuthology.orchestra.run.smithi060.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-03T14:32:14.952 INFO:teuthology.orchestra.run.smithi060.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-03T14:32:14.952 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-03T14:32:14.952 INFO:teuthology.orchestra.run.smithi060.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-03T14:32:14.952 INFO:teuthology.orchestra.run.smithi060.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-03T14:32:14.952 INFO:teuthology.orchestra.run.smithi060.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-03T14:32:14.952 INFO:teuthology.orchestra.run.smithi060.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-03T14:32:14.953 INFO:teuthology.orchestra.run.smithi060.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-03T14:32:14.953 INFO:teuthology.orchestra.run.smithi060.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-03T14:32:14.953 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-03T14:32:14.953 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ceph-argparse x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 45 k 2024-05-03T14:32:14.953 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ceph-common x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 135 k 2024-05-03T14:32:14.953 INFO:teuthology.orchestra.run.smithi060.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-03T14:32:14.953 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-03T14:32:14.953 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-03T14:32:14.953 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-03T14:32:14.953 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-03T14:32:14.953 INFO:teuthology.orchestra.run.smithi060.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-03T14:32:14.953 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-03T14:32:14.953 INFO:teuthology.orchestra.run.smithi060.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-03T14:32:14.953 INFO:teuthology.orchestra.run.smithi060.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-03T14:32:14.954 INFO:teuthology.orchestra.run.smithi060.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-03T14:32:14.954 INFO:teuthology.orchestra.run.smithi060.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-03T14:32:14.954 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-03T14:32:14.954 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-03T14:32:14.954 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-03T14:32:14.954 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-03T14:32:14.954 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-03T14:32:14.954 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-03T14:32:14.954 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-03T14:32:14.954 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-03T14:32:14.954 INFO:teuthology.orchestra.run.smithi060.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-03T14:32:14.954 INFO:teuthology.orchestra.run.smithi060.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-03T14:32:14.954 INFO:teuthology.orchestra.run.smithi060.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-03T14:32:14.954 INFO:teuthology.orchestra.run.smithi060.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-03T14:32:14.955 INFO:teuthology.orchestra.run.smithi060.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-03T14:32:14.955 INFO:teuthology.orchestra.run.smithi060.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-03T14:32:14.955 INFO:teuthology.orchestra.run.smithi060.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-03T14:32:14.955 INFO:teuthology.orchestra.run.smithi060.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-03T14:32:14.955 INFO:teuthology.orchestra.run.smithi060.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-03T14:32:14.955 INFO:teuthology.orchestra.run.smithi060.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-03T14:32:14.955 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-03T14:32:14.955 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-03T14:32:14.955 INFO:teuthology.orchestra.run.smithi060.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-03T14:32:14.955 INFO:teuthology.orchestra.run.smithi060.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-03T14:32:14.955 INFO:teuthology.orchestra.run.smithi060.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-03T14:32:14.955 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-03T14:32:14.955 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-03T14:32:14.955 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-03T14:32:14.955 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-03T14:32:14.956 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-03T14:32:14.956 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-03T14:32:14.956 INFO:teuthology.orchestra.run.smithi060.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-03T14:32:14.956 INFO:teuthology.orchestra.run.smithi060.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-03T14:32:14.956 INFO:teuthology.orchestra.run.smithi060.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-03T14:32:14.956 INFO:teuthology.orchestra.run.smithi060.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-03T14:32:14.956 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-03T14:32:14.956 INFO:teuthology.orchestra.run.smithi060.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-03T14:32:14.956 INFO:teuthology.orchestra.run.smithi060.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-03T14:32:14.956 INFO:teuthology.orchestra.run.smithi060.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-03T14:32:14.956 INFO:teuthology.orchestra.run.smithi060.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-03T14:32:14.956 INFO:teuthology.orchestra.run.smithi060.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-03T14:32:14.956 INFO:teuthology.orchestra.run.smithi060.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-03T14:32:14.956 INFO:teuthology.orchestra.run.smithi060.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-03T14:32:14.957 INFO:teuthology.orchestra.run.smithi060.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-03T14:32:14.957 INFO:teuthology.orchestra.run.smithi060.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-03T14:32:14.957 INFO:teuthology.orchestra.run.smithi060.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-03T14:32:14.957 INFO:teuthology.orchestra.run.smithi060.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-03T14:32:14.957 INFO:teuthology.orchestra.run.smithi060.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-03T14:32:14.957 INFO:teuthology.orchestra.run.smithi060.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-03T14:32:14.957 INFO:teuthology.orchestra.run.smithi060.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-03T14:32:14.957 INFO:teuthology.orchestra.run.smithi060.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-03T14:32:14.957 INFO:teuthology.orchestra.run.smithi060.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-03T14:32:14.957 INFO:teuthology.orchestra.run.smithi060.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-03T14:32:14.957 INFO:teuthology.orchestra.run.smithi060.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-03T14:32:14.957 INFO:teuthology.orchestra.run.smithi060.stdout:Installing weak dependencies: 2024-05-03T14:32:14.957 INFO:teuthology.orchestra.run.smithi060.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-03T14:32:14.958 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:32:14.958 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2024-05-03T14:32:14.958 INFO:teuthology.orchestra.run.smithi060.stdout:======================================================================================= 2024-05-03T14:32:14.958 INFO:teuthology.orchestra.run.smithi060.stdout:Install 148 Packages 2024-05-03T14:32:14.958 INFO:teuthology.orchestra.run.smithi060.stdout:Upgrade 4 Packages 2024-05-03T14:32:14.958 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:32:14.958 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 222 M 2024-05-03T14:32:14.958 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2024-05-03T14:32:15.035 INFO:teuthology.orchestra.run.smithi090.stdout:(24/152): python3-rgw-19.0.0-3450.g9a047fe7.el9 999 kB/s | 100 kB 00:00 2024-05-03T14:32:15.527 INFO:teuthology.orchestra.run.smithi090.stdout:(25/152): ceph-test-19.0.0-3450.g9a047fe7.el9.x 17 MB/s | 49 MB 00:02 2024-05-03T14:32:15.552 INFO:teuthology.orchestra.run.smithi090.stdout:(26/152): rbd-fuse-19.0.0-3450.g9a047fe7.el9.x8 165 kB/s | 85 kB 00:00 2024-05-03T14:32:15.628 INFO:teuthology.orchestra.run.smithi090.stdout:(27/152): rbd-nbd-19.0.0-3450.g9a047fe7.el9.x86 2.2 MB/s | 170 kB 00:00 2024-05-03T14:32:15.736 INFO:teuthology.orchestra.run.smithi090.stdout:(28/152): rbd-mirror-19.0.0-3450.g9a047fe7.el9. 15 MB/s | 3.1 MB 00:00 2024-05-03T14:32:15.762 INFO:teuthology.orchestra.run.smithi090.stdout:(29/152): ceph-grafana-dashboards-19.0.0-3450.g 256 kB/s | 34 kB 00:00 2024-05-03T14:32:15.995 INFO:teuthology.orchestra.run.smithi090.stdout:(30/152): ceph-common-19.0.0-3450.g9a047fe7.el9 3.7 MB/s | 21 MB 00:05 2024-05-03T14:32:16.020 INFO:teuthology.orchestra.run.smithi090.stdout:(31/152): ceph-mgr-cephadm-19.0.0-3450.g9a047fe 501 kB/s | 142 kB 00:00 2024-05-03T14:32:16.034 INFO:teuthology.orchestra.run.smithi060.stdout:(1/152): ceph-19.0.0-3450.g9a047fe7.el9.x86_64. 21 kB/s | 6.6 kB 00:00 2024-05-03T14:32:16.171 INFO:teuthology.orchestra.run.smithi090.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3450.g9a 1.8 MB/s | 274 kB 00:00 2024-05-03T14:32:16.238 INFO:teuthology.orchestra.run.smithi090.stdout:(33/152): ceph-mgr-dashboard-19.0.0-3450.g9a047 7.4 MB/s | 3.5 MB 00:00 2024-05-03T14:32:16.263 INFO:teuthology.orchestra.run.smithi090.stdout:(34/152): ceph-mgr-rook-19.0.0-3450.g9a047fe7.e 537 kB/s | 49 kB 00:00 2024-05-03T14:32:16.330 INFO:teuthology.orchestra.run.smithi090.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3450.g9 179 kB/s | 16 kB 00:00 2024-05-03T14:32:16.464 INFO:teuthology.orchestra.run.smithi090.stdout:(36/152): ceph-mgr-diskprediction-local-19.0.0- 17 MB/s | 7.4 MB 00:00 2024-05-03T14:32:16.489 INFO:teuthology.orchestra.run.smithi090.stdout:(37/152): ceph-volume-19.0.0-3450.g9a047fe7.el9 1.2 MB/s | 273 kB 00:00 2024-05-03T14:32:16.514 INFO:teuthology.orchestra.run.smithi090.stdout:(38/152): cephfs-top-19.0.0-3450.g9a047fe7.el9. 364 kB/s | 18 kB 00:00 2024-05-03T14:32:16.558 INFO:teuthology.orchestra.run.smithi090.stdout:(39/152): cephadm-19.0.0-3450.g9a047fe7.el9.noa 3.3 MB/s | 763 kB 00:00 2024-05-03T14:32:16.592 INFO:teuthology.orchestra.run.smithi060.stdout:(2/152): ceph-fuse-19.0.0-3450.g9a047fe7.el9.x8 1.6 MB/s | 891 kB 00:00 2024-05-03T14:32:16.650 INFO:teuthology.orchestra.run.smithi090.stdout:(40/152): fuse-2.9.9-15.el9.x86_64.rpm 502 kB/s | 80 kB 00:00 2024-05-03T14:32:16.708 INFO:teuthology.orchestra.run.smithi090.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.3 MB/s | 195 kB 00:00 2024-05-03T14:32:16.726 INFO:teuthology.orchestra.run.smithi060.stdout:(3/152): ceph-immutable-object-cache-19.0.0-345 1.1 MB/s | 145 kB 00:00 2024-05-03T14:32:16.742 INFO:teuthology.orchestra.run.smithi090.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-05-03T14:32:16.776 INFO:teuthology.orchestra.run.smithi090.stdout:(43/152): libgfortran-11.4.1-3.el9.x86_64.rpm 3.0 MB/s | 803 kB 00:00 2024-05-03T14:32:16.809 INFO:teuthology.orchestra.run.smithi090.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-05-03T14:32:16.835 INFO:teuthology.orchestra.run.smithi090.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-05-03T14:32:16.860 INFO:teuthology.orchestra.run.smithi090.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-05-03T14:32:16.894 INFO:teuthology.orchestra.run.smithi090.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-05-03T14:32:16.936 INFO:teuthology.orchestra.run.smithi090.stdout:(48/152): python3-cryptography-36.0.1-4.el9.x86 5.4 MB/s | 1.2 MB 00:00 2024-05-03T14:32:17.094 INFO:teuthology.orchestra.run.smithi090.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 675 kB/s | 107 kB 00:00 2024-05-03T14:32:17.153 INFO:teuthology.orchestra.run.smithi090.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 369 kB/s | 21 kB 00:00 2024-05-03T14:32:17.187 INFO:teuthology.orchestra.run.smithi090.stdout:(51/152): automake-1.16.2-8.el9.noarch.rpm 2.1 MB/s | 689 kB 00:00 2024-05-03T14:32:17.245 INFO:teuthology.orchestra.run.smithi090.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 561 kB/s | 33 kB 00:00 2024-05-03T14:32:17.279 INFO:teuthology.orchestra.run.smithi090.stdout:(53/152): bison-3.7.4-5.el9.x86_64.rpm 2.4 MB/s | 943 kB 00:00 2024-05-03T14:32:17.313 INFO:teuthology.orchestra.run.smithi090.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 1.9 MB/s | 313 kB 00:00 2024-05-03T14:32:17.338 INFO:teuthology.orchestra.run.smithi090.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 311 kB/s | 18 kB 00:00 2024-05-03T14:32:17.372 INFO:teuthology.orchestra.run.smithi090.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 189 kB/s | 11 kB 00:00 2024-05-03T14:32:17.430 INFO:teuthology.orchestra.run.smithi090.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.7 MB/s | 160 kB 00:00 2024-05-03T14:32:17.456 INFO:teuthology.orchestra.run.smithi090.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 541 kB/s | 45 kB 00:00 2024-05-03T14:32:17.606 INFO:teuthology.orchestra.run.smithi090.stdout:(59/152): libtool-2.4.6-45.el9.x86_64.rpm 3.8 MB/s | 582 kB 00:00 2024-05-03T14:32:17.665 INFO:teuthology.orchestra.run.smithi090.stdout:(60/152): librdkafka-1.6.1-102.el9.x86_64.rpm 2.8 MB/s | 662 kB 00:00 2024-05-03T14:32:17.690 INFO:teuthology.orchestra.run.smithi090.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 304 kB/s | 25 kB 00:00 2024-05-03T14:32:17.732 INFO:teuthology.orchestra.run.smithi090.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 709 kB/s | 47 kB 00:00 2024-05-03T14:32:17.748 INFO:teuthology.orchestra.run.smithi060.stdout:(4/152): ceph-common-19.0.0-3450.g9a047fe7.el9. 10 MB/s | 21 MB 00:02 2024-05-03T14:32:17.826 INFO:teuthology.orchestra.run.smithi090.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.1 MB/s | 292 kB 00:00 2024-05-03T14:32:17.843 INFO:teuthology.orchestra.run.smithi060.stdout:(5/152): ceph-mds-19.0.0-3450.g9a047fe7.el9.x86 2.1 MB/s | 2.4 MB 00:01 2024-05-03T14:32:17.859 INFO:teuthology.orchestra.run.smithi090.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-05-03T14:32:17.893 INFO:teuthology.orchestra.run.smithi090.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 528 kB/s | 35 kB 00:00 2024-05-03T14:32:17.960 INFO:teuthology.orchestra.run.smithi060.stdout:(6/152): ceph-mgr-19.0.0-3450.g9a047fe7.el9.x86 7.2 MB/s | 1.5 MB 00:00 2024-05-03T14:32:18.027 INFO:teuthology.orchestra.run.smithi090.stdout:(66/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.8 MB/s | 3.0 MB 00:00 2024-05-03T14:32:18.394 INFO:teuthology.orchestra.run.smithi090.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 4.3 MB/s | 1.6 MB 00:00 2024-05-03T14:32:18.530 INFO:teuthology.orchestra.run.smithi149.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-05-03T14:32:18.602 INFO:teuthology.orchestra.run.smithi090.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 4.3 MB/s | 911 kB 00:00 2024-05-03T14:32:18.669 INFO:teuthology.orchestra.run.smithi090.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 133 kB 00:00 2024-05-03T14:32:18.793 INFO:teuthology.orchestra.run.smithi060.stdout:(7/152): ceph-base-19.0.0-3450.g9a047fe7.el9.x8 1.7 MB/s | 5.4 MB 00:03 2024-05-03T14:32:19.045 INFO:teuthology.orchestra.run.smithi090.stdout:(70/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 3.8 MB/s | 4.4 MB 00:01 2024-05-03T14:32:19.052 INFO:teuthology.orchestra.run.smithi060.stdout:(8/152): ceph-osd-19.0.0-3450.g9a047fe7.el9.x86 15 MB/s | 17 MB 00:01 2024-05-03T14:32:19.161 INFO:teuthology.orchestra.run.smithi060.stdout:(9/152): ceph-mon-19.0.0-3450.g9a047fe7.el9.x86 3.6 MB/s | 4.7 MB 00:01 2024-05-03T14:32:19.186 INFO:teuthology.orchestra.run.smithi060.stdout:(10/152): ceph-selinux-19.0.0-3450.g9a047fe7.el 190 kB/s | 25 kB 00:00 2024-05-03T14:32:19.237 INFO:teuthology.orchestra.run.smithi090.stdout:(71/152): openblas-openmp-0.3.21-2.el9.x86_64.r 3.4 MB/s | 4.7 MB 00:01 2024-05-03T14:32:19.261 INFO:teuthology.orchestra.run.smithi060.stdout:(11/152): cephfs-mirror-19.0.0-3450.g9a047fe7.e 3.0 MB/s | 225 kB 00:00 2024-05-03T14:32:19.295 INFO:teuthology.orchestra.run.smithi090.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-05-03T14:32:19.336 INFO:teuthology.orchestra.run.smithi060.stdout:(12/152): cephfs-shell-19.0.0-3450.g9a047fe7.el 364 kB/s | 27 kB 00:00 2024-05-03T14:32:19.370 INFO:teuthology.orchestra.run.smithi090.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 3.1 MB/s | 1.0 MB 00:00 2024-05-03T14:32:19.395 INFO:teuthology.orchestra.run.smithi060.stdout:(13/152): libcephfs-devel-19.0.0-3450.g9a047fe7 549 kB/s | 32 kB 00:00 2024-05-03T14:32:19.471 INFO:teuthology.orchestra.run.smithi090.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 2.4 MB/s | 244 kB 00:00 2024-05-03T14:32:19.512 INFO:teuthology.orchestra.run.smithi060.stdout:(14/152): libcephfs2-19.0.0-3450.g9a047fe7.el9. 6.3 MB/s | 747 kB 00:00 2024-05-03T14:32:19.596 INFO:teuthology.orchestra.run.smithi060.stdout:(15/152): libcephsqlite-19.0.0-3450.g9a047fe7.e 1.9 MB/s | 163 kB 00:00 2024-05-03T14:32:19.679 INFO:teuthology.orchestra.run.smithi060.stdout:(16/152): librados-devel-19.0.0-3450.g9a047fe7. 1.5 MB/s | 127 kB 00:00 2024-05-03T14:32:19.696 INFO:teuthology.orchestra.run.smithi090.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 1.1 MB/s | 248 kB 00:00 2024-05-03T14:32:19.780 INFO:teuthology.orchestra.run.smithi060.stdout:(17/152): libradosstriper1-19.0.0-3450.g9a047fe 5.0 MB/s | 500 kB 00:00 2024-05-03T14:32:19.830 INFO:teuthology.orchestra.run.smithi090.stdout:(76/152): postgresql-server-13.14-1.el9.x86_64. 5.0 MB/s | 5.8 MB 00:01 2024-05-03T14:32:19.855 INFO:teuthology.orchestra.run.smithi090.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 161 kB/s | 26 kB 00:00 2024-05-03T14:32:19.889 INFO:teuthology.orchestra.run.smithi090.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 327 kB/s | 19 kB 00:00 2024-05-03T14:32:19.948 INFO:teuthology.orchestra.run.smithi090.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 1.8 MB/s | 172 kB 00:00 2024-05-03T14:32:19.973 INFO:teuthology.orchestra.run.smithi090.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 414 kB/s | 35 kB 00:00 2024-05-03T14:32:20.073 INFO:teuthology.orchestra.run.smithi090.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.4 MB/s | 250 kB 00:00 2024-05-03T14:32:20.165 INFO:teuthology.orchestra.run.smithi090.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.4 MB/s | 222 kB 00:00 2024-05-03T14:32:20.172 INFO:teuthology.orchestra.run.smithi060.stdout:(18/152): librgw2-19.0.0-3450.g9a047fe7.el9.x86 14 MB/s | 5.4 MB 00:00 2024-05-03T14:32:20.216 INFO:teuthology.orchestra.run.smithi090.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-05-03T14:32:20.247 INFO:teuthology.orchestra.run.smithi060.stdout:(19/152): python3-ceph-argparse-19.0.0-3450.g9a 611 kB/s | 45 kB 00:00 2024-05-03T14:32:20.274 INFO:teuthology.orchestra.run.smithi090.stdout:(84/152): python3-prettytable-0.7.2-27.el9.noar 723 kB/s | 42 kB 00:00 2024-05-03T14:32:20.314 INFO:teuthology.orchestra.run.smithi060.stdout:(20/152): python3-ceph-common-19.0.0-3450.g9a04 2.0 MB/s | 135 kB 00:00 2024-05-03T14:32:20.367 INFO:teuthology.orchestra.run.smithi090.stdout:(85/152): python3-protobuf-3.14.0-13.el9.noarch 2.9 MB/s | 269 kB 00:00 2024-05-03T14:32:20.397 INFO:teuthology.orchestra.run.smithi060.stdout:(21/152): python3-cephfs-19.0.0-3450.g9a047fe7. 1.9 MB/s | 161 kB 00:00 2024-05-03T14:32:20.481 INFO:teuthology.orchestra.run.smithi060.stdout:(22/152): python3-rados-19.0.0-3450.g9a047fe7.e 3.8 MB/s | 320 kB 00:00 2024-05-03T14:32:20.573 INFO:teuthology.orchestra.run.smithi060.stdout:(23/152): python3-rbd-19.0.0-3450.g9a047fe7.el9 3.2 MB/s | 298 kB 00:00 2024-05-03T14:32:20.626 INFO:teuthology.orchestra.run.smithi090.stdout:(86/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 612 kB/s | 159 kB 00:00 2024-05-03T14:32:20.657 INFO:teuthology.orchestra.run.smithi060.stdout:(24/152): python3-rgw-19.0.0-3450.g9a047fe7.el9 1.2 MB/s | 100 kB 00:00 2024-05-03T14:32:20.673 INFO:teuthology.orchestra.run.smithi149.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-03T14:32:20.674 INFO:teuthology.orchestra.run.smithi149.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-03T14:32:20.684 INFO:teuthology.orchestra.run.smithi149.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-03T14:32:20.685 INFO:teuthology.orchestra.run.smithi149.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-03T14:32:20.686 INFO:teuthology.orchestra.run.smithi149.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-03T14:32:20.687 INFO:teuthology.orchestra.run.smithi149.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-03T14:32:20.689 INFO:teuthology.orchestra.run.smithi149.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-03T14:32:20.691 INFO:teuthology.orchestra.run.smithi149.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-03T14:32:20.692 INFO:teuthology.orchestra.run.smithi149.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-03T14:32:20.710 INFO:teuthology.orchestra.run.smithi090.stdout:(87/152): python3-babel-2.9.1-2.el9.noarch.rpm 4.2 MB/s | 6.0 MB 00:01 2024-05-03T14:32:20.732 INFO:teuthology.orchestra.run.smithi060.stdout:(25/152): rbd-fuse-19.0.0-3450.g9a047fe7.el9.x8 1.1 MB/s | 85 kB 00:00 2024-05-03T14:32:20.753 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-05-03T14:32:20.760 INFO:teuthology.orchestra.run.smithi090.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 2.0 MB/s | 279 kB 00:00 2024-05-03T14:32:20.776 INFO:teuthology.orchestra.run.smithi149.stdout:======================================================================================= 2024-05-03T14:32:20.776 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repository Size 2024-05-03T14:32:20.776 INFO:teuthology.orchestra.run.smithi149.stdout:======================================================================================= 2024-05-03T14:32:20.776 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-05-03T14:32:20.776 INFO:teuthology.orchestra.run.smithi149.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-03T14:32:20.776 INFO:teuthology.orchestra.run.smithi149.stdout: ceph x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 6.6 k 2024-05-03T14:32:20.776 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-base x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 5.4 M 2024-05-03T14:32:20.776 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-fuse x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 891 k 2024-05-03T14:32:20.776 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 145 k 2024-05-03T14:32:20.776 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 1.5 M 2024-05-03T14:32:20.776 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 142 k 2024-05-03T14:32:20.776 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 3.5 M 2024-05-03T14:32:20.776 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 7.4 M 2024-05-03T14:32:20.776 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-rook noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 49 k 2024-05-03T14:32:20.777 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-radosgw x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 11 M 2024-05-03T14:32:20.777 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-test x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 49 M 2024-05-03T14:32:20.777 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-volume noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 273 k 2024-05-03T14:32:20.777 INFO:teuthology.orchestra.run.smithi149.stdout: cephadm noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 763 k 2024-05-03T14:32:20.777 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-mirror x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 225 k 2024-05-03T14:32:20.777 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-shell x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 27 k 2024-05-03T14:32:20.777 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-top noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 18 k 2024-05-03T14:32:20.777 INFO:teuthology.orchestra.run.smithi149.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-03T14:32:20.777 INFO:teuthology.orchestra.run.smithi149.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-03T14:32:20.777 INFO:teuthology.orchestra.run.smithi149.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-03T14:32:20.777 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs-devel x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 32 k 2024-05-03T14:32:20.777 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs2 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 747 k 2024-05-03T14:32:20.777 INFO:teuthology.orchestra.run.smithi149.stdout: librados-devel x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 127 k 2024-05-03T14:32:20.778 INFO:teuthology.orchestra.run.smithi149.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-03T14:32:20.778 INFO:teuthology.orchestra.run.smithi149.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-03T14:32:20.778 INFO:teuthology.orchestra.run.smithi149.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-03T14:32:20.778 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-03T14:32:20.778 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-03T14:32:20.778 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-03T14:32:20.778 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cephfs x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 161 k 2024-05-03T14:32:20.778 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rados x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 320 k 2024-05-03T14:32:20.778 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rbd x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 298 k 2024-05-03T14:32:20.778 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rgw x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 100 k 2024-05-03T14:32:20.778 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-fuse x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 85 k 2024-05-03T14:32:20.778 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-mirror x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 3.1 M 2024-05-03T14:32:20.778 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-nbd x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 170 k 2024-05-03T14:32:20.778 INFO:teuthology.orchestra.run.smithi149.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-03T14:32:20.779 INFO:teuthology.orchestra.run.smithi149.stdout:Upgrading: 2024-05-03T14:32:20.779 INFO:teuthology.orchestra.run.smithi149.stdout: librados2 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 3.4 M 2024-05-03T14:32:20.779 INFO:teuthology.orchestra.run.smithi149.stdout: librbd1 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 3.1 M 2024-05-03T14:32:20.779 INFO:teuthology.orchestra.run.smithi149.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-03T14:32:20.779 INFO:teuthology.orchestra.run.smithi149.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-03T14:32:20.779 INFO:teuthology.orchestra.run.smithi149.stdout:Installing dependencies: 2024-05-03T14:32:20.779 INFO:teuthology.orchestra.run.smithi149.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-03T14:32:20.779 INFO:teuthology.orchestra.run.smithi149.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-03T14:32:20.779 INFO:teuthology.orchestra.run.smithi149.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-03T14:32:20.779 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-common x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 21 M 2024-05-03T14:32:20.779 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 34 k 2024-05-03T14:32:20.779 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mds x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 2.4 M 2024-05-03T14:32:20.779 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 274 k 2024-05-03T14:32:20.780 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mon x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 4.7 M 2024-05-03T14:32:20.780 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-osd x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 17 M 2024-05-03T14:32:20.780 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 16 k 2024-05-03T14:32:20.780 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-selinux x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 25 k 2024-05-03T14:32:20.780 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-03T14:32:20.780 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-03T14:32:20.780 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-03T14:32:20.780 INFO:teuthology.orchestra.run.smithi149.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-03T14:32:20.780 INFO:teuthology.orchestra.run.smithi149.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-03T14:32:20.780 INFO:teuthology.orchestra.run.smithi149.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-03T14:32:20.780 INFO:teuthology.orchestra.run.smithi149.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-03T14:32:20.780 INFO:teuthology.orchestra.run.smithi149.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-03T14:32:20.780 INFO:teuthology.orchestra.run.smithi149.stdout: libcephsqlite x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 163 k 2024-05-03T14:32:20.780 INFO:teuthology.orchestra.run.smithi149.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-03T14:32:20.780 INFO:teuthology.orchestra.run.smithi149.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-03T14:32:20.780 INFO:teuthology.orchestra.run.smithi149.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-03T14:32:20.781 INFO:teuthology.orchestra.run.smithi149.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-03T14:32:20.781 INFO:teuthology.orchestra.run.smithi149.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-03T14:32:20.781 INFO:teuthology.orchestra.run.smithi149.stdout: libradosstriper1 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 500 k 2024-05-03T14:32:20.781 INFO:teuthology.orchestra.run.smithi149.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-03T14:32:20.781 INFO:teuthology.orchestra.run.smithi149.stdout: librgw2 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 5.4 M 2024-05-03T14:32:20.781 INFO:teuthology.orchestra.run.smithi149.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-03T14:32:20.781 INFO:teuthology.orchestra.run.smithi149.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-03T14:32:20.781 INFO:teuthology.orchestra.run.smithi149.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-03T14:32:20.781 INFO:teuthology.orchestra.run.smithi149.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-03T14:32:20.781 INFO:teuthology.orchestra.run.smithi149.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-03T14:32:20.781 INFO:teuthology.orchestra.run.smithi149.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-03T14:32:20.781 INFO:teuthology.orchestra.run.smithi149.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-03T14:32:20.781 INFO:teuthology.orchestra.run.smithi149.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-03T14:32:20.781 INFO:teuthology.orchestra.run.smithi149.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-03T14:32:20.782 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-03T14:32:20.782 INFO:teuthology.orchestra.run.smithi149.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-03T14:32:20.782 INFO:teuthology.orchestra.run.smithi149.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-03T14:32:20.782 INFO:teuthology.orchestra.run.smithi149.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-03T14:32:20.782 INFO:teuthology.orchestra.run.smithi149.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-03T14:32:20.782 INFO:teuthology.orchestra.run.smithi149.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-03T14:32:20.782 INFO:teuthology.orchestra.run.smithi149.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-03T14:32:20.782 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-03T14:32:20.782 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-argparse x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 45 k 2024-05-03T14:32:20.782 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-common x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 135 k 2024-05-03T14:32:20.782 INFO:teuthology.orchestra.run.smithi149.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-03T14:32:20.782 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-03T14:32:20.782 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-03T14:32:20.783 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-03T14:32:20.783 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-03T14:32:20.783 INFO:teuthology.orchestra.run.smithi149.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-03T14:32:20.783 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-03T14:32:20.783 INFO:teuthology.orchestra.run.smithi149.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-03T14:32:20.783 INFO:teuthology.orchestra.run.smithi149.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-03T14:32:20.783 INFO:teuthology.orchestra.run.smithi149.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-03T14:32:20.783 INFO:teuthology.orchestra.run.smithi149.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-03T14:32:20.783 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-03T14:32:20.783 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-03T14:32:20.783 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-03T14:32:20.783 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-03T14:32:20.783 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-03T14:32:20.784 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-03T14:32:20.784 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-03T14:32:20.784 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-03T14:32:20.784 INFO:teuthology.orchestra.run.smithi149.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-03T14:32:20.784 INFO:teuthology.orchestra.run.smithi149.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-03T14:32:20.784 INFO:teuthology.orchestra.run.smithi149.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-03T14:32:20.784 INFO:teuthology.orchestra.run.smithi149.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-03T14:32:20.784 INFO:teuthology.orchestra.run.smithi149.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-03T14:32:20.784 INFO:teuthology.orchestra.run.smithi149.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-03T14:32:20.784 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-03T14:32:20.784 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-03T14:32:20.784 INFO:teuthology.orchestra.run.smithi149.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-03T14:32:20.784 INFO:teuthology.orchestra.run.smithi149.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-03T14:32:20.784 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-03T14:32:20.784 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-03T14:32:20.785 INFO:teuthology.orchestra.run.smithi149.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-03T14:32:20.785 INFO:teuthology.orchestra.run.smithi149.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-03T14:32:20.785 INFO:teuthology.orchestra.run.smithi149.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-03T14:32:20.785 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-03T14:32:20.785 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-03T14:32:20.785 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-03T14:32:20.785 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-03T14:32:20.785 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-03T14:32:20.785 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-03T14:32:20.785 INFO:teuthology.orchestra.run.smithi149.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-03T14:32:20.785 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-03T14:32:20.785 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-03T14:32:20.785 INFO:teuthology.orchestra.run.smithi149.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-03T14:32:20.785 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-03T14:32:20.786 INFO:teuthology.orchestra.run.smithi149.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-03T14:32:20.786 INFO:teuthology.orchestra.run.smithi149.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-03T14:32:20.786 INFO:teuthology.orchestra.run.smithi149.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-03T14:32:20.786 INFO:teuthology.orchestra.run.smithi149.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-03T14:32:20.786 INFO:teuthology.orchestra.run.smithi149.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-03T14:32:20.786 INFO:teuthology.orchestra.run.smithi149.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-03T14:32:20.786 INFO:teuthology.orchestra.run.smithi149.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-03T14:32:20.786 INFO:teuthology.orchestra.run.smithi149.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-03T14:32:20.786 INFO:teuthology.orchestra.run.smithi149.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-03T14:32:20.786 INFO:teuthology.orchestra.run.smithi149.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-03T14:32:20.786 INFO:teuthology.orchestra.run.smithi149.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-03T14:32:20.786 INFO:teuthology.orchestra.run.smithi149.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-03T14:32:20.786 INFO:teuthology.orchestra.run.smithi149.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-03T14:32:20.786 INFO:teuthology.orchestra.run.smithi149.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-03T14:32:20.787 INFO:teuthology.orchestra.run.smithi149.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-03T14:32:20.787 INFO:teuthology.orchestra.run.smithi149.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-03T14:32:20.787 INFO:teuthology.orchestra.run.smithi149.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-03T14:32:20.787 INFO:teuthology.orchestra.run.smithi149.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-03T14:32:20.787 INFO:teuthology.orchestra.run.smithi149.stdout:Installing weak dependencies: 2024-05-03T14:32:20.787 INFO:teuthology.orchestra.run.smithi149.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-03T14:32:20.787 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-03T14:32:20.787 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-05-03T14:32:20.787 INFO:teuthology.orchestra.run.smithi149.stdout:======================================================================================= 2024-05-03T14:32:20.787 INFO:teuthology.orchestra.run.smithi149.stdout:Install 148 Packages 2024-05-03T14:32:20.787 INFO:teuthology.orchestra.run.smithi149.stdout:Upgrade 4 Packages 2024-05-03T14:32:20.787 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-03T14:32:20.788 INFO:teuthology.orchestra.run.smithi090.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 679 kB/s | 51 kB 00:00 2024-05-03T14:32:20.788 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 222 M 2024-05-03T14:32:20.788 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-05-03T14:32:20.819 INFO:teuthology.orchestra.run.smithi090.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 913 kB/s | 54 kB 00:00 2024-05-03T14:32:20.878 INFO:teuthology.orchestra.run.smithi090.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 713 kB/s | 42 kB 00:00 2024-05-03T14:32:20.937 INFO:teuthology.orchestra.run.smithi090.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 759 kB/s | 44 kB 00:00 2024-05-03T14:32:20.991 INFO:teuthology.orchestra.run.smithi060.stdout:(26/152): ceph-radosgw-19.0.0-3450.g9a047fe7.el 4.8 MB/s | 11 MB 00:02 2024-05-03T14:32:20.995 INFO:teuthology.orchestra.run.smithi090.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-05-03T14:32:21.054 INFO:teuthology.orchestra.run.smithi090.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 615 kB/s | 36 kB 00:00 2024-05-03T14:32:21.100 INFO:teuthology.orchestra.run.smithi060.stdout:(27/152): rbd-nbd-19.0.0-3450.g9a047fe7.el9.x86 1.5 MB/s | 170 kB 00:00 2024-05-03T14:32:21.113 INFO:teuthology.orchestra.run.smithi090.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 811 kB/s | 47 kB 00:00 2024-05-03T14:32:21.222 INFO:teuthology.orchestra.run.smithi090.stdout:(96/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.0 MB/s | 5.1 MB 00:01 2024-05-03T14:32:21.233 INFO:teuthology.orchestra.run.smithi060.stdout:(28/152): rbd-mirror-19.0.0-3450.g9a047fe7.el9. 6.2 MB/s | 3.1 MB 00:00 2024-05-03T14:32:21.255 INFO:teuthology.orchestra.run.smithi090.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 2.1 MB/s | 305 kB 00:00 2024-05-03T14:32:21.258 INFO:teuthology.orchestra.run.smithi060.stdout:(29/152): ceph-grafana-dashboards-19.0.0-3450.g 217 kB/s | 34 kB 00:00 2024-05-03T14:32:21.281 INFO:teuthology.orchestra.run.smithi090.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 986 kB/s | 58 kB 00:00 2024-05-03T14:32:21.300 INFO:teuthology.orchestra.run.smithi060.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3450.g9a047fe 2.1 MB/s | 142 kB 00:00 2024-05-03T14:32:21.314 INFO:teuthology.orchestra.run.smithi090.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-05-03T14:32:21.348 INFO:teuthology.orchestra.run.smithi090.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 950 kB/s | 64 kB 00:00 2024-05-03T14:32:21.382 INFO:teuthology.orchestra.run.smithi090.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 668 kB/s | 45 kB 00:00 2024-05-03T14:32:21.531 INFO:teuthology.orchestra.run.smithi149.stdout:(1/152): ceph-19.0.0-3450.g9a047fe7.el9.x86_64. 20 kB/s | 6.6 kB 00:00 2024-05-03T14:32:21.551 INFO:teuthology.orchestra.run.smithi060.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3450.g9a047 12 MB/s | 3.5 MB 00:00 2024-05-03T14:32:21.682 INFO:teuthology.orchestra.run.smithi090.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 67 kB/s | 22 kB 00:00 2024-05-03T14:32:21.692 INFO:teuthology.orchestra.run.smithi060.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3450.g9a 1.9 MB/s | 274 kB 00:00 2024-05-03T14:32:21.851 INFO:teuthology.orchestra.run.smithi060.stdout:(33/152): ceph-mgr-rook-19.0.0-3450.g9a047fe7.e 313 kB/s | 49 kB 00:00 2024-05-03T14:32:21.957 INFO:teuthology.orchestra.run.smithi090.stdout:(103/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.9 MB/s | 548 kB 00:00 2024-05-03T14:32:21.985 INFO:teuthology.orchestra.run.smithi060.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3450.g9 124 kB/s | 16 kB 00:00 2024-05-03T14:32:21.999 INFO:teuthology.orchestra.run.smithi090.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 1.4 MB/s | 863 kB 00:00 2024-05-03T14:32:22.033 INFO:teuthology.orchestra.run.smithi090.stdout:(105/152): gperftools-libs-2.9.1-2.el9.x86_64.r 4.0 MB/s | 309 kB 00:00 2024-05-03T14:32:22.117 INFO:teuthology.orchestra.run.smithi090.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 166 kB/s | 19 kB 00:00 2024-05-03T14:32:22.119 INFO:teuthology.orchestra.run.smithi060.stdout:(35/152): ceph-volume-19.0.0-3450.g9a047fe7.el9 2.0 MB/s | 273 kB 00:00 2024-05-03T14:32:22.226 INFO:teuthology.orchestra.run.smithi090.stdout:(107/152): libarrow-9.0.0-11.el9.x86_64.rpm 23 MB/s | 4.4 MB 00:00 2024-05-03T14:32:22.251 INFO:teuthology.orchestra.run.smithi090.stdout:(108/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 194 kB/s | 26 kB 00:00 2024-05-03T14:32:22.284 INFO:teuthology.orchestra.run.smithi090.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 832 kB/s | 49 kB 00:00 2024-05-03T14:32:22.336 INFO:teuthology.orchestra.run.smithi090.stdout:(110/152): luarocks-3.9.2-1.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-05-03T14:32:22.361 INFO:teuthology.orchestra.run.smithi090.stdout:(111/152): libunwind-1.6.2-1.el9.x86_64.rpm 610 kB/s | 67 kB 00:00 2024-05-03T14:32:22.403 INFO:teuthology.orchestra.run.smithi090.stdout:(112/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 12 MB/s | 839 kB 00:00 2024-05-03T14:32:22.462 INFO:teuthology.orchestra.run.smithi090.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 745 kB/s | 43 kB 00:00 2024-05-03T14:32:22.521 INFO:teuthology.orchestra.run.smithi090.stdout:(114/152): python3-cachetools-4.2.4-1.el9.noarc 553 kB/s | 32 kB 00:00 2024-05-03T14:32:22.580 INFO:teuthology.orchestra.run.smithi090.stdout:(115/152): python3-certifi-2021.10.8-2.el9.noar 256 kB/s | 15 kB 00:00 2024-05-03T14:32:22.613 INFO:teuthology.orchestra.run.smithi090.stdout:(116/152): python3-asyncssh-2.13.2-5.el9.noarch 2.1 MB/s | 548 kB 00:00 2024-05-03T14:32:22.647 INFO:teuthology.orchestra.run.smithi090.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-05-03T14:32:22.689 INFO:teuthology.orchestra.run.smithi090.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 4.6 MB/s | 358 kB 00:00 2024-05-03T14:32:22.714 INFO:teuthology.orchestra.run.smithi090.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-05-03T14:32:22.723 INFO:teuthology.orchestra.run.smithi149.stdout:(2/152): ceph-common-19.0.0-3450.g9a047fe7.el9. 14 MB/s | 21 MB 00:01 2024-05-03T14:32:22.740 INFO:teuthology.orchestra.run.smithi090.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 703 kB/s | 35 kB 00:00 2024-05-03T14:32:22.756 INFO:teuthology.orchestra.run.smithi149.stdout:(3/152): ceph-fuse-19.0.0-3450.g9a047fe7.el9.x8 727 kB/s | 891 kB 00:01 2024-05-03T14:32:22.773 INFO:teuthology.orchestra.run.smithi090.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 3.4 MB/s | 206 kB 00:00 2024-05-03T14:32:22.782 INFO:teuthology.orchestra.run.smithi149.stdout:(4/152): ceph-immutable-object-cache-19.0.0-345 2.4 MB/s | 145 kB 00:00 2024-05-03T14:32:22.833 INFO:teuthology.orchestra.run.smithi090.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-05-03T14:32:22.844 INFO:teuthology.orchestra.run.smithi060.stdout:(36/152): cephadm-19.0.0-3450.g9a047fe7.el9.noa 1.0 MB/s | 763 kB 00:00 2024-05-03T14:32:22.890 INFO:teuthology.orchestra.run.smithi149.stdout:(5/152): ceph-mgr-19.0.0-3450.g9a047fe7.el9.x86 14 MB/s | 1.5 MB 00:00 2024-05-03T14:32:22.891 INFO:teuthology.orchestra.run.smithi090.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 14 MB/s | 2.0 MB 00:00 2024-05-03T14:32:22.916 INFO:teuthology.orchestra.run.smithi090.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 127 kB/s | 11 kB 00:00 2024-05-03T14:32:22.936 INFO:teuthology.orchestra.run.smithi060.stdout:(37/152): cephfs-top-19.0.0-3450.g9a047fe7.el9. 201 kB/s | 18 kB 00:00 2024-05-03T14:32:22.941 INFO:teuthology.orchestra.run.smithi090.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 351 kB/s | 18 kB 00:00 2024-05-03T14:32:22.967 INFO:teuthology.orchestra.run.smithi090.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 459 kB/s | 23 kB 00:00 2024-05-03T14:32:22.992 INFO:teuthology.orchestra.run.smithi090.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 384 kB/s | 19 kB 00:00 2024-05-03T14:32:23.018 INFO:teuthology.orchestra.run.smithi090.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 389 kB/s | 20 kB 00:00 2024-05-03T14:32:23.068 INFO:teuthology.orchestra.run.smithi090.stdout:(129/152): python3-kubernetes-26.1.0-2.el9.noar 14 MB/s | 1.0 MB 00:00 2024-05-03T14:32:23.093 INFO:teuthology.orchestra.run.smithi090.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 615 kB/s | 46 kB 00:00 2024-05-03T14:32:23.127 INFO:teuthology.orchestra.run.smithi090.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-05-03T14:32:23.152 INFO:teuthology.orchestra.run.smithi090.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 983 kB/s | 58 kB 00:00 2024-05-03T14:32:23.182 INFO:teuthology.orchestra.run.smithi149.stdout:(6/152): ceph-mon-19.0.0-3450.g9a047fe7.el9.x86 16 MB/s | 4.7 MB 00:00 2024-05-03T14:32:23.187 INFO:teuthology.orchestra.run.smithi090.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 4.4 MB/s | 272 kB 00:00 2024-05-03T14:32:23.213 INFO:teuthology.orchestra.run.smithi090.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 273 kB/s | 16 kB 00:00 2024-05-03T14:32:23.246 INFO:teuthology.orchestra.run.smithi090.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-05-03T14:32:23.272 INFO:teuthology.orchestra.run.smithi090.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 481 kB/s | 28 kB 00:00 2024-05-03T14:32:23.305 INFO:teuthology.orchestra.run.smithi090.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 523 kB/s | 31 kB 00:00 2024-05-03T14:32:23.340 INFO:teuthology.orchestra.run.smithi090.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 2.8 MB/s | 188 kB 00:00 2024-05-03T14:32:23.364 INFO:teuthology.orchestra.run.smithi090.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-05-03T14:32:23.398 INFO:teuthology.orchestra.run.smithi090.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 608 kB/s | 36 kB 00:00 2024-05-03T14:32:23.423 INFO:teuthology.orchestra.run.smithi090.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 871 kB/s | 51 kB 00:00 2024-05-03T14:32:23.428 INFO:teuthology.orchestra.run.smithi060.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 164 kB/s | 80 kB 00:00 2024-05-03T14:32:23.457 INFO:teuthology.orchestra.run.smithi090.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.8 MB/s | 230 kB 00:00 2024-05-03T14:32:23.482 INFO:teuthology.orchestra.run.smithi090.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-05-03T14:32:23.516 INFO:teuthology.orchestra.run.smithi090.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 7.1 MB/s | 427 kB 00:00 2024-05-03T14:32:23.541 INFO:teuthology.orchestra.run.smithi090.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 340 kB/s | 20 kB 00:00 2024-05-03T14:32:23.575 INFO:teuthology.orchestra.run.smithi090.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 3.2 MB/s | 192 kB 00:00 2024-05-03T14:32:23.678 INFO:teuthology.orchestra.run.smithi060.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 3.2 MB/s | 803 kB 00:00 2024-05-03T14:32:23.767 INFO:teuthology.orchestra.run.smithi090.stdout:(147/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 5.5 MB/s | 16 MB 00:02 2024-05-03T14:32:23.778 INFO:teuthology.orchestra.run.smithi060.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.9 MB/s | 195 kB 00:00 2024-05-03T14:32:23.851 INFO:teuthology.orchestra.run.smithi090.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 5.1 MB/s | 1.6 MB 00:00 2024-05-03T14:32:23.904 INFO:teuthology.orchestra.run.smithi060.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.0 MB/s | 253 kB 00:00 2024-05-03T14:32:24.024 INFO:teuthology.orchestra.run.smithi149.stdout:(7/152): ceph-mds-19.0.0-3450.g9a047fe7.el9.x86 1.9 MB/s | 2.4 MB 00:01 2024-05-03T14:32:24.043 INFO:teuthology.orchestra.run.smithi090.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 8.1 MB/s | 1.3 MB 00:00 2024-05-03T14:32:24.079 INFO:teuthology.orchestra.run.smithi060.stdout:(42/152): ceph-mgr-diskprediction-local-19.0.0- 2.7 MB/s | 7.4 MB 00:02 2024-05-03T14:32:24.135 INFO:teuthology.orchestra.run.smithi090.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 26 MB/s | 2.4 MB 00:00 2024-05-03T14:32:24.191 INFO:teuthology.orchestra.run.smithi149.stdout:(8/152): ceph-osd-19.0.0-3450.g9a047fe7.el9.x86 17 MB/s | 17 MB 00:01 2024-05-03T14:32:24.246 INFO:teuthology.orchestra.run.smithi060.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 3.6 MB/s | 1.2 MB 00:00 2024-05-03T14:32:24.275 INFO:teuthology.orchestra.run.smithi149.stdout:(9/152): ceph-selinux-19.0.0-3450.g9a047fe7.el9 303 kB/s | 25 kB 00:00 2024-05-03T14:32:24.318 INFO:teuthology.orchestra.run.smithi090.stdout:(151/152): librbd1-19.0.0-3450.g9a047fe7.el9.x8 6.0 MB/s | 3.1 MB 00:00 2024-05-03T14:32:24.329 INFO:teuthology.orchestra.run.smithi060.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-05-03T14:32:24.363 INFO:teuthology.orchestra.run.smithi060.stdout:(45/152): python3-ply-3.11-14.el9.noarch.rpm 376 kB/s | 106 kB 00:00 2024-05-03T14:32:24.414 INFO:teuthology.orchestra.run.smithi060.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-05-03T14:32:24.489 INFO:teuthology.orchestra.run.smithi060.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 1.7 MB/s | 215 kB 00:00 2024-05-03T14:32:24.731 INFO:teuthology.orchestra.run.smithi060.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 2.1 MB/s | 689 kB 00:00 2024-05-03T14:32:24.768 INFO:teuthology.orchestra.run.smithi090.stdout:(152/152): librados2-19.0.0-3450.g9a047fe7.el9. 2.8 MB/s | 3.4 MB 00:01 2024-05-03T14:32:24.774 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-05-03T14:32:24.775 INFO:teuthology.orchestra.run.smithi090.stdout:Total 15 MB/s | 222 MB 00:14 2024-05-03T14:32:24.790 INFO:teuthology.orchestra.run.smithi060.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-05-03T14:32:24.900 INFO:teuthology.orchestra.run.smithi060.stdout:(50/152): bison-3.7.4-5.el9.x86_64.rpm 2.2 MB/s | 943 kB 00:00 2024-05-03T14:32:24.924 INFO:teuthology.orchestra.run.smithi060.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 161 kB/s | 21 kB 00:00 2024-05-03T14:32:24.999 INFO:teuthology.orchestra.run.smithi060.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 442 kB/s | 33 kB 00:00 2024-05-03T14:32:25.033 INFO:teuthology.orchestra.run.smithi060.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 2.3 MB/s | 313 kB 00:00 2024-05-03T14:32:25.108 INFO:teuthology.orchestra.run.smithi060.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 244 kB/s | 18 kB 00:00 2024-05-03T14:32:25.175 INFO:teuthology.orchestra.run.smithi060.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 169 kB/s | 11 kB 00:00 2024-05-03T14:32:25.234 INFO:teuthology.orchestra.run.smithi060.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-05-03T14:32:25.384 INFO:teuthology.orchestra.run.smithi060.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 302 kB/s | 45 kB 00:00 2024-05-03T14:32:25.568 INFO:teuthology.orchestra.run.smithi060.stdout:(58/152): librdkafka-1.6.1-102.el9.x86_64.rpm 3.5 MB/s | 662 kB 00:00 2024-05-03T14:32:25.743 INFO:teuthology.orchestra.run.smithi060.stdout:(59/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.0 MB/s | 3.0 MB 00:00 2024-05-03T14:32:25.785 INFO:teuthology.orchestra.run.smithi060.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 2.6 MB/s | 582 kB 00:00 2024-05-03T14:32:25.802 INFO:teuthology.orchestra.run.smithi060.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 441 kB/s | 25 kB 00:00 2024-05-03T14:32:25.836 INFO:teuthology.orchestra.run.smithi060.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 962 kB/s | 47 kB 00:00 2024-05-03T14:32:25.858 INFO:teuthology.orchestra.run.smithi149.stdout:(10/152): ceph-radosgw-19.0.0-3450.g9a047fe7.el 5.8 MB/s | 11 MB 00:01 2024-05-03T14:32:25.870 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-05-03T14:32:25.894 INFO:teuthology.orchestra.run.smithi060.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.1 MB/s | 292 kB 00:00 2024-05-03T14:32:25.919 INFO:teuthology.orchestra.run.smithi060.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-05-03T14:32:25.953 INFO:teuthology.orchestra.run.smithi060.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 610 kB/s | 35 kB 00:00 2024-05-03T14:32:25.958 INFO:teuthology.orchestra.run.smithi149.stdout:(11/152): cephfs-mirror-19.0.0-3450.g9a047fe7.e 2.2 MB/s | 225 kB 00:00 2024-05-03T14:32:26.014 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-05-03T14:32:26.014 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-05-03T14:32:26.042 INFO:teuthology.orchestra.run.smithi149.stdout:(12/152): cephfs-shell-19.0.0-3450.g9a047fe7.el 324 kB/s | 27 kB 00:00 2024-05-03T14:32:26.134 INFO:teuthology.orchestra.run.smithi149.stdout:(13/152): libcephfs-devel-19.0.0-3450.g9a047fe7 347 kB/s | 32 kB 00:00 2024-05-03T14:32:26.301 INFO:teuthology.orchestra.run.smithi149.stdout:(14/152): libcephfs2-19.0.0-3450.g9a047fe7.el9. 4.4 MB/s | 747 kB 00:00 2024-05-03T14:32:26.435 INFO:teuthology.orchestra.run.smithi149.stdout:(15/152): libcephsqlite-19.0.0-3450.g9a047fe7.e 1.2 MB/s | 163 kB 00:00 2024-05-03T14:32:26.510 INFO:teuthology.orchestra.run.smithi149.stdout:(16/152): ceph-base-19.0.0-3450.g9a047fe7.el9.x 1.0 MB/s | 5.4 MB 00:05 2024-05-03T14:32:26.535 INFO:teuthology.orchestra.run.smithi149.stdout:(17/152): librados-devel-19.0.0-3450.g9a047fe7. 1.2 MB/s | 127 kB 00:00 2024-05-03T14:32:26.754 INFO:teuthology.orchestra.run.smithi060.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 5.7 MB/s | 4.7 MB 00:00 2024-05-03T14:32:26.919 INFO:teuthology.orchestra.run.smithi149.stdout:(18/152): libradosstriper1-19.0.0-3450.g9a047fe 1.2 MB/s | 500 kB 00:00 2024-05-03T14:32:26.950 INFO:teuthology.orchestra.run.smithi060.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 4.4 MB/s | 4.4 MB 00:00 2024-05-03T14:32:27.036 INFO:teuthology.orchestra.run.smithi149.stdout:(19/152): python3-ceph-argparse-19.0.0-3450.g9a 387 kB/s | 45 kB 00:00 2024-05-03T14:32:27.129 INFO:teuthology.orchestra.run.smithi060.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 4.2 MB/s | 1.6 MB 00:00 2024-05-03T14:32:27.161 INFO:teuthology.orchestra.run.smithi149.stdout:(20/152): librgw2-19.0.0-3450.g9a047fe7.el9.x86 8.6 MB/s | 5.4 MB 00:00 2024-05-03T14:32:27.196 INFO:teuthology.orchestra.run.smithi060.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 3.6 MB/s | 911 kB 00:00 2024-05-03T14:32:27.229 INFO:teuthology.orchestra.run.smithi060.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.3 MB/s | 133 kB 00:00 2024-05-03T14:32:27.405 INFO:teuthology.orchestra.run.smithi060.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 5.8 MB/s | 1.0 MB 00:00 2024-05-03T14:32:27.464 INFO:teuthology.orchestra.run.smithi060.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-05-03T14:32:27.595 INFO:teuthology.orchestra.run.smithi149.stdout:(21/152): ceph-test-19.0.0-3450.g9a047fe7.el9.x 15 MB/s | 49 MB 00:03 2024-05-03T14:32:27.620 INFO:teuthology.orchestra.run.smithi149.stdout:(22/152): python3-ceph-common-19.0.0-3450.g9a04 232 kB/s | 135 kB 00:00 2024-05-03T14:32:27.645 INFO:teuthology.orchestra.run.smithi149.stdout:(23/152): python3-cephfs-19.0.0-3450.g9a047fe7. 334 kB/s | 161 kB 00:00 2024-05-03T14:32:27.671 INFO:teuthology.orchestra.run.smithi149.stdout:(24/152): python3-rados-19.0.0-3450.g9a047fe7.e 4.2 MB/s | 320 kB 00:00 2024-05-03T14:32:27.704 INFO:teuthology.orchestra.run.smithi149.stdout:(25/152): python3-rgw-19.0.0-3450.g9a047fe7.el9 1.7 MB/s | 100 kB 00:00 2024-05-03T14:32:27.721 INFO:teuthology.orchestra.run.smithi149.stdout:(26/152): rbd-fuse-19.0.0-3450.g9a047fe7.el9.x8 1.7 MB/s | 85 kB 00:00 2024-05-03T14:32:27.772 INFO:teuthology.orchestra.run.smithi149.stdout:(27/152): rbd-nbd-19.0.0-3450.g9a047fe7.el9.x86 3.3 MB/s | 170 kB 00:00 2024-05-03T14:32:27.787 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-05-03T14:32:27.787 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-05-03T14:32:27.813 INFO:teuthology.orchestra.run.smithi149.stdout:(28/152): python3-rbd-19.0.0-3450.g9a047fe7.el9 1.5 MB/s | 298 kB 00:00 2024-05-03T14:32:27.839 INFO:teuthology.orchestra.run.smithi149.stdout:(29/152): ceph-grafana-dashboards-19.0.0-3450.g 512 kB/s | 34 kB 00:00 2024-05-03T14:32:27.972 INFO:teuthology.orchestra.run.smithi149.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3450.g9a047fe 898 kB/s | 142 kB 00:00 2024-05-03T14:32:28.039 INFO:teuthology.orchestra.run.smithi149.stdout:(31/152): rbd-mirror-19.0.0-3450.g9a047fe7.el9. 9.2 MB/s | 3.1 MB 00:00 2024-05-03T14:32:28.098 INFO:teuthology.orchestra.run.smithi149.stdout:(32/152): ceph-mgr-dashboard-19.0.0-3450.g9a047 14 MB/s | 3.5 MB 00:00 2024-05-03T14:32:28.123 INFO:teuthology.orchestra.run.smithi149.stdout:(33/152): ceph-mgr-modules-core-19.0.0-3450.g9a 3.2 MB/s | 274 kB 00:00 2024-05-03T14:32:28.148 INFO:teuthology.orchestra.run.smithi149.stdout:(34/152): ceph-mgr-rook-19.0.0-3450.g9a047fe7.e 986 kB/s | 49 kB 00:00 2024-05-03T14:32:28.182 INFO:teuthology.orchestra.run.smithi149.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3450.g9 282 kB/s | 16 kB 00:00 2024-05-03T14:32:28.189 INFO:teuthology.orchestra.run.smithi060.stdout:(73/152): python3-babel-2.9.1-2.el9.noarch.rpm 8.2 MB/s | 6.0 MB 00:00 2024-05-03T14:32:28.207 INFO:teuthology.orchestra.run.smithi149.stdout:(36/152): ceph-volume-19.0.0-3450.g9a047fe7.el9 4.6 MB/s | 273 kB 00:00 2024-05-03T14:32:28.256 INFO:teuthology.orchestra.run.smithi060.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 3.6 MB/s | 244 kB 00:00 2024-05-03T14:32:28.257 INFO:teuthology.orchestra.run.smithi149.stdout:(37/152): cephfs-top-19.0.0-3450.g9a047fe7.el9. 365 kB/s | 18 kB 00:00 2024-05-03T14:32:28.291 INFO:teuthology.orchestra.run.smithi149.stdout:(38/152): cephadm-19.0.0-3450.g9a047fe7.el9.noa 6.9 MB/s | 763 kB 00:00 2024-05-03T14:32:28.415 INFO:teuthology.orchestra.run.smithi060.stdout:(75/152): postgresql-server-13.14-1.el9.x86_64. 4.7 MB/s | 5.8 MB 00:01 2024-05-03T14:32:28.448 INFO:teuthology.orchestra.run.smithi060.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 1.3 MB/s | 248 kB 00:00 2024-05-03T14:32:28.465 INFO:teuthology.orchestra.run.smithi060.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 520 kB/s | 26 kB 00:00 2024-05-03T14:32:28.499 INFO:teuthology.orchestra.run.smithi060.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 388 kB/s | 19 kB 00:00 2024-05-03T14:32:28.532 INFO:teuthology.orchestra.run.smithi060.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.5 MB/s | 172 kB 00:00 2024-05-03T14:32:28.558 INFO:teuthology.orchestra.run.smithi060.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 602 kB/s | 35 kB 00:00 2024-05-03T14:32:28.625 INFO:teuthology.orchestra.run.smithi060.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.7 MB/s | 250 kB 00:00 2024-05-03T14:32:28.692 INFO:teuthology.orchestra.run.smithi060.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.3 MB/s | 222 kB 00:00 2024-05-03T14:32:28.750 INFO:teuthology.orchestra.run.smithi060.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-05-03T14:32:28.809 INFO:teuthology.orchestra.run.smithi060.stdout:(84/152): python3-prettytable-0.7.2-27.el9.noar 736 kB/s | 42 kB 00:00 2024-05-03T14:32:28.876 INFO:teuthology.orchestra.run.smithi060.stdout:(85/152): python3-protobuf-3.14.0-13.el9.noarch 4.0 MB/s | 269 kB 00:00 2024-05-03T14:32:28.951 INFO:teuthology.orchestra.run.smithi060.stdout:(86/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-05-03T14:32:28.983 INFO:teuthology.orchestra.run.smithi149.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 111 kB/s | 80 kB 00:00 2024-05-03T14:32:29.018 INFO:teuthology.orchestra.run.smithi060.stdout:(87/152): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 279 kB 00:00 2024-05-03T14:32:29.094 INFO:teuthology.orchestra.run.smithi060.stdout:(88/152): python3-pytz-2021.1-5.el9.noarch.rpm 691 kB/s | 51 kB 00:00 2024-05-03T14:32:29.152 INFO:teuthology.orchestra.run.smithi060.stdout:(89/152): python3-requests-oauthlib-1.3.0-12.el 934 kB/s | 54 kB 00:00 2024-05-03T14:32:29.342 INFO:teuthology.orchestra.run.smithi149.stdout:(40/152): libgfortran-11.4.1-3.el9.x86_64.rpm 765 kB/s | 803 kB 00:01 2024-05-03T14:32:29.544 INFO:teuthology.orchestra.run.smithi060.stdout:(90/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 5.0 MB/s | 5.1 MB 00:01 2024-05-03T14:32:29.611 INFO:teuthology.orchestra.run.smithi060.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 675 kB/s | 42 kB 00:00 2024-05-03T14:32:29.670 INFO:teuthology.orchestra.run.smithi060.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 777 kB/s | 44 kB 00:00 2024-05-03T14:32:29.737 INFO:teuthology.orchestra.run.smithi060.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.3 MB/s | 221 kB 00:00 2024-05-03T14:32:29.795 INFO:teuthology.orchestra.run.smithi060.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 627 kB/s | 36 kB 00:00 2024-05-03T14:32:29.817 INFO:teuthology.orchestra.run.smithi149.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 234 kB/s | 195 kB 00:00 2024-05-03T14:32:29.854 INFO:teuthology.orchestra.run.smithi060.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 826 kB/s | 47 kB 00:00 2024-05-03T14:32:29.955 INFO:teuthology.orchestra.run.smithi060.stdout:(96/152): socat-1.7.4.1-5.el9.x86_64.rpm 3.0 MB/s | 305 kB 00:00 2024-05-03T14:32:30.013 INFO:teuthology.orchestra.run.smithi060.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 1.0 MB/s | 58 kB 00:00 2024-05-03T14:32:30.103 INFO:teuthology.orchestra.run.smithi060.stdout:(98/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.0 MB/s | 94 kB 00:00 2024-05-03T14:32:30.173 INFO:teuthology.orchestra.run.smithi060.stdout:(99/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 942 kB/s | 64 kB 00:00 2024-05-03T14:32:30.183 INFO:teuthology.orchestra.run.smithi149.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 301 kB/s | 253 kB 00:00 2024-05-03T14:32:30.231 INFO:teuthology.orchestra.run.smithi060.stdout:(100/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 781 kB/s | 45 kB 00:00 2024-05-03T14:32:30.390 INFO:teuthology.orchestra.run.smithi060.stdout:(101/152): lua-devel-5.4.4-4.el9.x86_64.rpm 142 kB/s | 22 kB 00:00 2024-05-03T14:32:30.573 INFO:teuthology.orchestra.run.smithi060.stdout:(102/152): protobuf-compiler-3.14.0-13.el9.x86_ 4.6 MB/s | 863 kB 00:00 2024-05-03T14:32:30.609 INFO:teuthology.orchestra.run.smithi149.stdout:(43/152): ceph-mgr-diskprediction-local-19.0.0- 2.8 MB/s | 7.4 MB 00:02 2024-05-03T14:32:30.918 INFO:teuthology.orchestra.run.smithi149.stdout:(44/152): python3-cryptography-36.0.1-4.el9.x86 1.1 MB/s | 1.2 MB 00:01 2024-05-03T14:32:31.009 INFO:teuthology.orchestra.run.smithi149.stdout:(45/152): python3-ply-3.11-14.el9.noarch.rpm 129 kB/s | 106 kB 00:00 2024-05-03T14:32:31.082 INFO:teuthology.orchestra.run.smithi060.stdout:(103/152): ceph-test-19.0.0-3450.g9a047fe7.el9. 4.1 MB/s | 49 MB 00:11 2024-05-03T14:32:31.324 INFO:teuthology.orchestra.run.smithi060.stdout:(104/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 7.5 MB/s | 16 MB 00:02 2024-05-03T14:32:31.368 INFO:teuthology.orchestra.run.smithi149.stdout:(46/152): python3-pycparser-2.20-6.el9.noarch.r 178 kB/s | 135 kB 00:00 2024-05-03T14:32:31.416 INFO:teuthology.orchestra.run.smithi060.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 215 kB/s | 19 kB 00:00 2024-05-03T14:32:31.535 INFO:teuthology.orchestra.run.smithi149.stdout:(47/152): automake-1.16.2-8.el9.noarch.rpm 4.0 MB/s | 689 kB 00:00 2024-05-03T14:32:31.541 INFO:teuthology.orchestra.run.smithi060.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 675 kB/s | 309 kB 00:00 2024-05-03T14:32:31.600 INFO:teuthology.orchestra.run.smithi060.stdout:(107/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 452 kB/s | 26 kB 00:00 2024-05-03T14:32:31.602 INFO:teuthology.orchestra.run.smithi149.stdout:(48/152): bison-3.7.4-5.el9.x86_64.rpm 14 MB/s | 943 kB 00:00 2024-05-03T14:32:31.644 INFO:teuthology.orchestra.run.smithi149.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 107 kB 00:00 2024-05-03T14:32:31.675 INFO:teuthology.orchestra.run.smithi060.stdout:(108/152): abseil-cpp-20211102.0-3.el9.x86_64.r 498 kB/s | 548 kB 00:01 2024-05-03T14:32:31.686 INFO:teuthology.orchestra.run.smithi149.stdout:(50/152): python3-requests-2.25.1-8.el9.noarch. 163 kB/s | 125 kB 00:00 2024-05-03T14:32:31.700 INFO:teuthology.orchestra.run.smithi060.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 490 kB/s | 49 kB 00:00 2024-05-03T14:32:31.711 INFO:teuthology.orchestra.run.smithi149.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 321 kB/s | 21 kB 00:00 2024-05-03T14:32:31.745 INFO:teuthology.orchestra.run.smithi149.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 986 kB/s | 33 kB 00:00 2024-05-03T14:32:31.751 INFO:teuthology.orchestra.run.smithi060.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 905 kB/s | 67 kB 00:00 2024-05-03T14:32:31.837 INFO:teuthology.orchestra.run.smithi149.stdout:(53/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 33 MB/s | 3.0 MB 00:00 2024-05-03T14:32:31.859 INFO:teuthology.orchestra.run.smithi060.stdout:(111/152): libarrow-9.0.0-11.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-05-03T14:32:31.862 INFO:teuthology.orchestra.run.smithi149.stdout:(54/152): python3-urllib3-1.26.5-5.el9.noarch.r 252 kB/s | 215 kB 00:00 2024-05-03T14:32:31.879 INFO:teuthology.orchestra.run.smithi149.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 436 kB/s | 18 kB 00:00 2024-05-03T14:32:31.884 INFO:teuthology.orchestra.run.smithi060.stdout:(112/152): luarocks-3.9.2-1.el9.noarch.rpm 826 kB/s | 151 kB 00:00 2024-05-03T14:32:31.904 INFO:teuthology.orchestra.run.smithi149.stdout:(56/152): flex-2.6.4-9.el9.x86_64.rpm 1.4 MB/s | 313 kB 00:00 2024-05-03T14:32:31.921 INFO:teuthology.orchestra.run.smithi149.stdout:(57/152): libaio-devel-0.3.111-13.el9.x86_64.rp 190 kB/s | 11 kB 00:00 2024-05-03T14:32:31.938 INFO:teuthology.orchestra.run.smithi149.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 45 kB 00:00 2024-05-03T14:32:31.952 INFO:teuthology.orchestra.run.smithi060.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 5.9 MB/s | 548 kB 00:00 2024-05-03T14:32:31.963 INFO:teuthology.orchestra.run.smithi149.stdout:(59/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 160 kB 00:00 2024-05-03T14:32:31.977 INFO:teuthology.orchestra.run.smithi060.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 476 kB/s | 43 kB 00:00 2024-05-03T14:32:32.005 INFO:teuthology.orchestra.run.smithi149.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 613 kB/s | 25 kB 00:00 2024-05-03T14:32:32.039 INFO:teuthology.orchestra.run.smithi149.stdout:(61/152): libtool-2.4.6-45.el9.x86_64.rpm 5.7 MB/s | 582 kB 00:00 2024-05-03T14:32:32.072 INFO:teuthology.orchestra.run.smithi149.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 711 kB/s | 47 kB 00:00 2024-05-03T14:32:32.098 INFO:teuthology.orchestra.run.smithi149.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-05-03T14:32:32.131 INFO:teuthology.orchestra.run.smithi149.stdout:(64/152): librdkafka-1.6.1-102.el9.x86_64.rpm 3.1 MB/s | 662 kB 00:00 2024-05-03T14:32:32.156 INFO:teuthology.orchestra.run.smithi149.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 602 kB/s | 35 kB 00:00 2024-05-03T14:32:32.182 INFO:teuthology.orchestra.run.smithi149.stdout:(66/152): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-05-03T14:32:32.234 INFO:teuthology.orchestra.run.smithi060.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 114 kB/s | 32 kB 00:00 2024-05-03T14:32:32.241 INFO:teuthology.orchestra.run.smithi149.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 27 MB/s | 1.6 MB 00:00 2024-05-03T14:32:32.261 INFO:teuthology.orchestra.run.smithi060.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 55 kB/s | 15 kB 00:00 2024-05-03T14:32:32.303 INFO:teuthology.orchestra.run.smithi060.stdout:(117/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 1.5 MB/s | 839 kB 00:00 2024-05-03T14:32:32.316 INFO:teuthology.orchestra.run.smithi149.stdout:(68/152): openblas-openmp-0.3.21-2.el9.x86_64.r 26 MB/s | 4.7 MB 00:00 2024-05-03T14:32:32.361 INFO:teuthology.orchestra.run.smithi060.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-05-03T14:32:32.383 INFO:teuthology.orchestra.run.smithi149.stdout:(69/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-05-03T14:32:32.403 INFO:teuthology.orchestra.run.smithi060.stdout:(119/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.0 MB/s | 172 kB 00:00 2024-05-03T14:32:32.408 INFO:teuthology.orchestra.run.smithi149.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 133 kB 00:00 2024-05-03T14:32:32.429 INFO:teuthology.orchestra.run.smithi060.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 534 kB/s | 35 kB 00:00 2024-05-03T14:32:32.441 INFO:teuthology.orchestra.run.smithi149.stdout:(71/152): postgresql-contrib-13.14-1.el9.x86_64 4.4 MB/s | 911 kB 00:00 2024-05-03T14:32:32.462 INFO:teuthology.orchestra.run.smithi060.stdout:(121/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.9 MB/s | 303 kB 00:00 2024-05-03T14:32:32.476 INFO:teuthology.orchestra.run.smithi149.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-05-03T14:32:32.501 INFO:teuthology.orchestra.run.smithi149.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-05-03T14:32:32.571 INFO:teuthology.orchestra.run.smithi060.stdout:(122/152): python3-grpcio-1.46.7-10.el9.x86_64. 14 MB/s | 2.0 MB 00:00 2024-05-03T14:32:32.593 INFO:teuthology.orchestra.run.smithi149.stdout:(74/152): postgresql-server-13.14-1.el9.x86_64. 28 MB/s | 5.8 MB 00:00 2024-05-03T14:32:32.604 INFO:teuthology.orchestra.run.smithi060.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.0 MB/s | 144 kB 00:00 2024-05-03T14:32:32.618 INFO:teuthology.orchestra.run.smithi149.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 2.0 MB/s | 244 kB 00:00 2024-05-03T14:32:32.630 INFO:teuthology.orchestra.run.smithi060.stdout:(124/152): python3-google-auth-2.29.0-1.el9.noa 912 kB/s | 206 kB 00:00 2024-05-03T14:32:32.643 INFO:teuthology.orchestra.run.smithi149.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-05-03T14:32:32.655 INFO:teuthology.orchestra.run.smithi060.stdout:(125/152): python3-jaraco-8.2.1-3.el9.noarch.rp 129 kB/s | 11 kB 00:00 2024-05-03T14:32:32.668 INFO:teuthology.orchestra.run.smithi149.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 512 kB/s | 26 kB 00:00 2024-05-03T14:32:32.680 INFO:teuthology.orchestra.run.smithi060.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 236 kB/s | 18 kB 00:00 2024-05-03T14:32:32.694 INFO:teuthology.orchestra.run.smithi149.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-05-03T14:32:32.714 INFO:teuthology.orchestra.run.smithi060.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 277 kB/s | 23 kB 00:00 2024-05-03T14:32:32.719 INFO:teuthology.orchestra.run.smithi149.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-05-03T14:32:32.740 INFO:teuthology.orchestra.run.smithi060.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 233 kB/s | 19 kB 00:00 2024-05-03T14:32:32.744 INFO:teuthology.orchestra.run.smithi149.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 693 kB/s | 35 kB 00:00 2024-05-03T14:32:32.765 INFO:teuthology.orchestra.run.smithi060.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 236 kB/s | 20 kB 00:00 2024-05-03T14:32:32.845 INFO:teuthology.orchestra.run.smithi149.stdout:(81/152): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-05-03T14:32:32.870 INFO:teuthology.orchestra.run.smithi149.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.0 MB/s | 250 kB 00:00 2024-05-03T14:32:32.874 INFO:teuthology.orchestra.run.smithi060.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 348 kB/s | 46 kB 00:00 2024-05-03T14:32:32.895 INFO:teuthology.orchestra.run.smithi149.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-05-03T14:32:32.899 INFO:teuthology.orchestra.run.smithi060.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 594 kB/s | 79 kB 00:00 2024-05-03T14:32:32.920 INFO:teuthology.orchestra.run.smithi149.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-05-03T14:32:32.945 INFO:teuthology.orchestra.run.smithi149.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 840 kB/s | 42 kB 00:00 2024-05-03T14:32:32.949 INFO:teuthology.orchestra.run.smithi060.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 776 kB/s | 58 kB 00:00 2024-05-03T14:32:32.971 INFO:teuthology.orchestra.run.smithi149.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-05-03T14:32:32.983 INFO:teuthology.orchestra.run.smithi060.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-05-03T14:32:32.996 INFO:teuthology.orchestra.run.smithi149.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-05-03T14:32:33.022 INFO:teuthology.orchestra.run.smithi149.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-05-03T14:32:33.025 INFO:teuthology.orchestra.run.smithi060.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 221 kB/s | 16 kB 00:00 2024-05-03T14:32:33.047 INFO:teuthology.orchestra.run.smithi149.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-05-03T14:32:33.050 INFO:teuthology.orchestra.run.smithi060.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-05-03T14:32:33.072 INFO:teuthology.orchestra.run.smithi149.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-05-03T14:32:33.100 INFO:teuthology.orchestra.run.smithi060.stdout:(136/152): python3-repoze-lru-0.7-16.el9.noarch 625 kB/s | 31 kB 00:00 2024-05-03T14:32:33.142 INFO:teuthology.orchestra.run.smithi060.stdout:(137/152): python3-kubernetes-26.1.0-2.el9.noar 2.4 MB/s | 1.0 MB 00:00 2024-05-03T14:32:33.147 INFO:teuthology.orchestra.run.smithi149.stdout:(91/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-05-03T14:32:33.167 INFO:teuthology.orchestra.run.smithi060.stdout:(138/152): python3-pyperclip-1.8.2-2.el9.noarch 199 kB/s | 28 kB 00:00 2024-05-03T14:32:33.172 INFO:teuthology.orchestra.run.smithi149.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 415 kB/s | 42 kB 00:00 2024-05-03T14:32:33.190 INFO:teuthology.orchestra.run.smithi149.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-05-03T14:32:33.215 INFO:teuthology.orchestra.run.smithi149.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 5.1 MB/s | 221 kB 00:00 2024-05-03T14:32:33.257 INFO:teuthology.orchestra.run.smithi149.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 535 kB/s | 36 kB 00:00 2024-05-03T14:32:33.274 INFO:teuthology.orchestra.run.smithi149.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 812 kB/s | 47 kB 00:00 2024-05-03T14:32:33.284 INFO:teuthology.orchestra.run.smithi060.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.0 MB/s | 188 kB 00:00 2024-05-03T14:32:33.299 INFO:teuthology.orchestra.run.smithi149.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 7.1 MB/s | 305 kB 00:00 2024-05-03T14:32:33.310 INFO:teuthology.orchestra.run.smithi060.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 253 kB/s | 36 kB 00:00 2024-05-03T14:32:33.316 INFO:teuthology.orchestra.run.smithi149.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.3 MB/s | 58 kB 00:00 2024-05-03T14:32:33.335 INFO:teuthology.orchestra.run.smithi060.stdout:(141/152): python3-rsa-4.9-2.el9.noarch.rpm 307 kB/s | 59 kB 00:00 2024-05-03T14:32:33.341 INFO:teuthology.orchestra.run.smithi149.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 2.2 MB/s | 94 kB 00:00 2024-05-03T14:32:33.366 INFO:teuthology.orchestra.run.smithi149.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-05-03T14:32:33.383 INFO:teuthology.orchestra.run.smithi149.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.0 MB/s | 45 kB 00:00 2024-05-03T14:32:33.402 INFO:teuthology.orchestra.run.smithi060.stdout:(142/152): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-05-03T14:32:33.427 INFO:teuthology.orchestra.run.smithi060.stdout:(143/152): python3-typing-extensions-4.4.0-2.el 360 kB/s | 51 kB 00:00 2024-05-03T14:32:33.461 INFO:teuthology.orchestra.run.smithi060.stdout:(144/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.5 MB/s | 230 kB 00:00 2024-05-03T14:32:33.495 INFO:teuthology.orchestra.run.smithi060.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 301 kB/s | 20 kB 00:00 2024-05-03T14:32:33.545 INFO:teuthology.orchestra.run.smithi060.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 2.3 MB/s | 192 kB 00:00 2024-05-03T14:32:33.587 INFO:teuthology.orchestra.run.smithi060.stdout:(147/152): python3-werkzeug-2.0.3-3.el9.1.noarc 2.3 MB/s | 427 kB 00:00 2024-05-03T14:32:33.609 INFO:teuthology.orchestra.run.smithi149.stdout:(102/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 29 MB/s | 16 MB 00:00 2024-05-03T14:32:33.634 INFO:teuthology.orchestra.run.smithi149.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 83 kB/s | 22 kB 00:00 2024-05-03T14:32:33.654 INFO:teuthology.orchestra.run.smithi060.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2024-05-03T14:32:33.764 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-05-03T14:32:33.876 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-05-03T14:32:33.909 INFO:teuthology.orchestra.run.smithi149.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 1.6 MB/s | 863 kB 00:00 2024-05-03T14:32:33.922 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-05-03T14:32:33.985 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-05-03T14:32:34.027 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-05-03T14:32:34.053 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-05-03T14:32:34.087 INFO:teuthology.orchestra.run.smithi060.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 3.1 MB/s | 1.3 MB 00:00 2024-05-03T14:32:34.095 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-05-03T14:32:34.176 INFO:teuthology.orchestra.run.smithi149.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 73 kB/s | 19 kB 00:00 2024-05-03T14:32:34.354 INFO:teuthology.orchestra.run.smithi060.stdout:(150/152): librados2-19.0.0-3450.g9a047fe7.el9. 4.2 MB/s | 3.4 MB 00:00 2024-05-03T14:32:34.425 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-05-03T14:32:34.438 INFO:teuthology.orchestra.run.smithi060.stdout:(151/152): librbd1-19.0.0-3450.g9a047fe7.el9.x8 3.7 MB/s | 3.1 MB 00:00 2024-05-03T14:32:34.460 INFO:teuthology.orchestra.run.smithi149.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 374 kB/s | 309 kB 00:00 2024-05-03T14:32:34.504 INFO:teuthology.orchestra.run.smithi060.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 5.8 MB/s | 2.4 MB 00:00 2024-05-03T14:32:34.513 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2024-05-03T14:32:34.513 INFO:teuthology.orchestra.run.smithi060.stdout:Total 11 MB/s | 222 MB 00:19 2024-05-03T14:32:34.563 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : librados2-2:19.0.0-3450.g9a047fe7.el9.x86_64 8/156 2024-05-03T14:32:34.686 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librados2-2:19.0.0-3450.g9a047fe7.el9.x86_64 8/156 2024-05-03T14:32:34.715 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libcephfs2-2:19.0.0-3450.g9a047fe7.el9.x86_64 9/156 2024-05-03T14:32:34.735 INFO:teuthology.orchestra.run.smithi149.stdout:(107/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 94 kB/s | 26 kB 00:00 2024-05-03T14:32:34.807 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: libcephfs2-2:19.0.0-3450.g9a047fe7.el9.x86_64 9/156 2024-05-03T14:32:34.845 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-rados-2:19.0.0-3450.g9a047fe7.el9.x86_64 10/156 2024-05-03T14:32:34.919 INFO:teuthology.orchestra.run.smithi149.stdout:(108/152): abseil-cpp-20211102.0-3.el9.x86_64.r 419 kB/s | 548 kB 00:01 2024-05-03T14:32:35.002 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-05-03T14:32:35.059 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-05-03T14:32:35.077 INFO:teuthology.orchestra.run.smithi149.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 142 kB/s | 49 kB 00:00 2024-05-03T14:32:35.087 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libcephsqlite-2:19.0.0-3450.g9a047fe7.el9.x86_64 13/156 2024-05-03T14:32:35.191 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3450.g9a047fe7.el9.x86_64 13/156 2024-05-03T14:32:35.221 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libradosstriper1-2:19.0.0-3450.g9a047fe7.el9.x86 14/156 2024-05-03T14:32:35.244 INFO:teuthology.orchestra.run.smithi149.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 207 kB/s | 67 kB 00:00 2024-05-03T14:32:35.267 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3450.g9a047fe7.el9.x86 14/156 2024-05-03T14:32:35.299 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-05-03T14:32:35.323 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-05-03T14:32:35.516 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-05-03T14:32:35.549 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-05-03T14:32:35.595 INFO:teuthology.orchestra.run.smithi149.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 292 kB/s | 151 kB 00:00 2024-05-03T14:32:35.656 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-05-03T14:32:35.754 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-05-03T14:32:35.792 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-05-03T14:32:35.827 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-05-03T14:32:35.893 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-05-03T14:32:35.936 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-05-03T14:32:35.989 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-05-03T14:32:36.029 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-05-03T14:32:36.065 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-05-03T14:32:36.144 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-05-03T14:32:36.196 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-05-03T14:32:36.235 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-05-03T14:32:36.279 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-05-03T14:32:36.356 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-05-03T14:32:36.390 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-ceph-common-2:19.0.0-3450.g9a047fe7.el9. 31/156 2024-05-03T14:32:36.433 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-ceph-argparse-2:19.0.0-3450.g9a047fe7.el 32/156 2024-05-03T14:32:36.463 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cephfs-2:19.0.0-3450.g9a047fe7.el9.x86_6 33/156 2024-05-03T14:32:36.533 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-05-03T14:32:36.578 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-05-03T14:32:36.661 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-05-03T14:32:36.711 INFO:teuthology.orchestra.run.smithi149.stdout:(112/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 572 kB/s | 839 kB 00:01 2024-05-03T14:32:36.753 INFO:teuthology.orchestra.run.smithi149.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 473 kB/s | 548 kB 00:01 2024-05-03T14:32:36.805 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-05-03T14:32:36.856 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-05-03T14:32:37.012 INFO:teuthology.orchestra.run.smithi149.stdout:(114/152): python3-cachetools-4.2.4-1.el9.noarc 125 kB/s | 32 kB 00:00 2024-05-03T14:32:37.045 INFO:teuthology.orchestra.run.smithi149.stdout:(115/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 130 kB/s | 43 kB 00:00 2024-05-03T14:32:37.271 INFO:teuthology.orchestra.run.smithi149.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 58 kB/s | 15 kB 00:00 2024-05-03T14:32:37.377 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-05-03T14:32:37.485 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-05-03T14:32:37.492 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2024-05-03T14:32:37.524 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-05-03T14:32:37.558 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-05-03T14:32:37.597 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : librados-devel-2:19.0.0-3450.g9a047fe7.el9.x86_6 43/156 2024-05-03T14:32:37.646 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-05-03T14:32:37.721 INFO:teuthology.orchestra.run.smithi149.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 254 kB/s | 172 kB 00:00 2024-05-03T14:32:37.722 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2024-05-03T14:32:37.722 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2024-05-03T14:32:37.854 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-05-03T14:32:37.900 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-05-03T14:32:37.956 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-05-03T14:32:38.028 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-05-03T14:32:38.061 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-05-03T14:32:38.080 INFO:teuthology.orchestra.run.smithi149.stdout:(118/152): libarrow-9.0.0-11.el9.x86_64.rpm 1.1 MB/s | 4.4 MB 00:03 2024-05-03T14:32:38.097 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-05-03T14:32:38.144 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-05-03T14:32:38.189 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-05-03T14:32:38.232 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-05-03T14:32:38.271 INFO:teuthology.orchestra.run.smithi149.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 358 kB/s | 358 kB 00:01 2024-05-03T14:32:38.282 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-05-03T14:32:38.364 INFO:teuthology.orchestra.run.smithi149.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 124 kB/s | 35 kB 00:00 2024-05-03T14:32:38.417 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-05-03T14:32:38.464 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-05-03T14:32:38.494 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-05-03T14:32:38.562 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-05-03T14:32:38.646 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-05-03T14:32:38.687 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-05-03T14:32:38.697 INFO:teuthology.orchestra.run.smithi149.stdout:(121/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 311 kB/s | 303 kB 00:00 2024-05-03T14:32:38.727 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-05-03T14:32:38.761 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-05-03T14:32:38.786 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-05-03T14:32:38.861 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-05-03T14:32:38.897 INFO:teuthology.orchestra.run.smithi149.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 329 kB/s | 206 kB 00:00 2024-05-03T14:32:38.997 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-05-03T14:32:39.037 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-05-03T14:32:39.073 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-05-03T14:32:39.108 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-05-03T14:32:39.131 INFO:teuthology.orchestra.run.smithi149.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 46 kB/s | 11 kB 00:00 2024-05-03T14:32:39.147 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-05-03T14:32:39.187 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-05-03T14:32:39.217 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-05-03T14:32:39.252 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-05-03T14:32:39.285 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-05-03T14:32:39.326 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-05-03T14:32:39.339 INFO:teuthology.orchestra.run.smithi149.stdout:(124/152): python3-grpcio-tools-1.46.7-10.el9.x 225 kB/s | 144 kB 00:00 2024-05-03T14:32:39.362 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-05-03T14:32:39.423 INFO:teuthology.orchestra.run.smithi149.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 60 kB/s | 18 kB 00:00 2024-05-03T14:32:39.449 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-05-03T14:32:39.501 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-05-03T14:32:39.621 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-05-03T14:32:39.648 INFO:teuthology.orchestra.run.smithi149.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 75 kB/s | 23 kB 00:00 2024-05-03T14:32:39.732 INFO:teuthology.orchestra.run.smithi149.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 63 kB/s | 19 kB 00:00 2024-05-03T14:32:39.857 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-05-03T14:32:39.916 INFO:teuthology.orchestra.run.smithi149.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 74 kB/s | 20 kB 00:00 2024-05-03T14:32:39.964 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-05-03T14:32:40.018 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-05-03T14:32:40.249 INFO:teuthology.orchestra.run.smithi149.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 139 kB/s | 46 kB 00:00 2024-05-03T14:32:40.299 INFO:teuthology.orchestra.run.smithi149.stdout:(130/152): python3-grpcio-1.46.7-10.el9.x86_64. 1.1 MB/s | 2.0 MB 00:01 2024-05-03T14:32:40.324 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2024-05-03T14:32:40.325 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2024-05-03T14:32:40.592 INFO:teuthology.orchestra.run.smithi149.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 230 kB/s | 79 kB 00:00 2024-05-03T14:32:40.625 INFO:teuthology.orchestra.run.smithi149.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 177 kB/s | 58 kB 00:00 2024-05-03T14:32:40.734 INFO:teuthology.orchestra.run.smithi149.stdout:(133/152): python3-kubernetes-26.1.0-2.el9.noar 1.0 MB/s | 1.0 MB 00:01 2024-05-03T14:32:40.783 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-05-03T14:32:40.834 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-05-03T14:32:40.867 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-05-03T14:32:40.901 INFO:teuthology.orchestra.run.smithi149.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 60 kB/s | 16 kB 00:00 2024-05-03T14:32:40.967 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-05-03T14:32:41.026 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-05-03T14:32:41.051 INFO:teuthology.orchestra.run.smithi149.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 284 kB/s | 90 kB 00:00 2024-05-03T14:32:41.093 INFO:teuthology.orchestra.run.smithi149.stdout:(136/152): python3-pecan-1.4.2-3.el9.noarch.rpm 544 kB/s | 272 kB 00:00 2024-05-03T14:32:41.112 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-05-03T14:32:41.143 INFO:teuthology.orchestra.run.smithi149.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 116 kB/s | 28 kB 00:00 2024-05-03T14:32:41.145 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-05-03T14:32:41.313 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-05-03T14:32:41.327 INFO:teuthology.orchestra.run.smithi149.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 112 kB/s | 31 kB 00:00 2024-05-03T14:32:41.343 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-05-03T14:32:41.429 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-05-03T14:32:41.469 INFO:teuthology.orchestra.run.smithi149.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 182 kB/s | 59 kB 00:00 2024-05-03T14:32:41.511 INFO:teuthology.orchestra.run.smithi149.stdout:(140/152): python3-routes-2.5.1-5.el9.noarch.rp 451 kB/s | 188 kB 00:00 2024-05-03T14:32:41.578 INFO:teuthology.orchestra.run.smithi149.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 143 kB/s | 36 kB 00:00 2024-05-03T14:32:41.870 INFO:teuthology.orchestra.run.smithi149.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 127 kB/s | 51 kB 00:00 2024-05-03T14:32:41.978 INFO:teuthology.orchestra.run.smithi149.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 224 kB/s | 90 kB 00:00 2024-05-03T14:32:42.012 INFO:teuthology.orchestra.run.smithi149.stdout:(144/152): python3-webob-1.8.7-6.el9.noarch.rpm 458 kB/s | 230 kB 00:00 2024-05-03T14:32:42.069 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-05-03T14:32:42.183 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-05-03T14:32:42.237 INFO:teuthology.orchestra.run.smithi149.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 77 kB/s | 20 kB 00:00 2024-05-03T14:32:42.504 INFO:teuthology.orchestra.run.smithi149.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 389 kB/s | 192 kB 00:00 2024-05-03T14:32:42.529 INFO:teuthology.orchestra.run.smithi149.stdout:(147/152): python3-werkzeug-2.0.3-3.el9.1.noarc 648 kB/s | 427 kB 00:00 2024-05-03T14:32:42.988 INFO:teuthology.orchestra.run.smithi149.stdout:(148/152): librados2-19.0.0-3450.g9a047fe7.el9. 7.0 MB/s | 3.4 MB 00:00 2024-05-03T14:32:43.401 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-05-03T14:32:43.487 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-05-03T14:32:43.523 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-05-03T14:32:43.529 INFO:teuthology.orchestra.run.smithi149.stdout:(149/152): librbd1-19.0.0-3450.g9a047fe7.el9.x8 3.1 MB/s | 3.1 MB 00:00 2024-05-03T14:32:43.556 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-05-03T14:32:43.722 INFO:teuthology.orchestra.run.smithi149.stdout:(150/152): thrift-0.15.0-2.el9.x86_64.rpm 1.1 MB/s | 1.6 MB 00:01 2024-05-03T14:32:43.866 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-05-03T14:32:43.920 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : librbd1-2:19.0.0-3450.g9a047fe7.el9.x86_64 99/156 2024-05-03T14:32:44.011 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librbd1-2:19.0.0-3450.g9a047fe7.el9.x86_64 99/156 2024-05-03T14:32:44.043 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-rbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 100/156 2024-05-03T14:32:44.055 INFO:teuthology.orchestra.run.smithi149.stdout:(151/152): openssl-3.2.1-1.el9.x86_64.rpm 1.3 MB/s | 1.3 MB 00:01 2024-05-03T14:32:44.087 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-05-03T14:32:44.568 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-05-03T14:32:44.613 INFO:teuthology.orchestra.run.smithi149.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 2.2 MB/s | 2.4 MB 00:01 2024-05-03T14:32:44.616 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : librgw2-2:19.0.0-3450.g9a047fe7.el9.x86_64 103/156 2024-05-03T14:32:44.618 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-05-03T14:32:44.618 INFO:teuthology.orchestra.run.smithi149.stdout:Total 9.3 MB/s | 222 MB 00:23 2024-05-03T14:32:44.681 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librgw2-2:19.0.0-3450.g9a047fe7.el9.x86_64 103/156 2024-05-03T14:32:44.701 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-rgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 104/156 2024-05-03T14:32:45.701 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-05-03T14:32:45.846 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-05-03T14:32:45.847 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-05-03T14:32:46.982 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 105/156 2024-05-03T14:32:47.017 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 105/156 2024-05-03T14:32:47.039 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2024-05-03T14:32:47.147 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-05-03T14:32:47.240 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-05-03T14:32:47.361 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-05-03T14:32:47.429 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-05-03T14:32:47.465 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-05-03T14:32:47.561 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 105/156 2024-05-03T14:32:47.604 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-base-2:19.0.0-3450.g9a047fe7.el9.x86_64 106/156 2024-05-03T14:32:47.605 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-05-03T14:32:47.642 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-05-03T14:32:47.642 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-05-03T14:32:47.705 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-base-2:19.0.0-3450.g9a047fe7.el9.x86_64 106/156 2024-05-03T14:32:47.705 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-03T14:32:47.705 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-03T14:32:47.705 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-03T14:32:47.739 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-selinux-2:19.0.0-3450.g9a047fe7.el9.x86_64 107/156 2024-05-03T14:32:48.048 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-05-03T14:32:48.154 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : librados2-2:19.0.0-3450.g9a047fe7.el9.x86_64 8/156 2024-05-03T14:32:48.343 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librados2-2:19.0.0-3450.g9a047fe7.el9.x86_64 8/156 2024-05-03T14:32:48.372 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libcephfs2-2:19.0.0-3450.g9a047fe7.el9.x86_64 9/156 2024-05-03T14:32:48.502 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libcephfs2-2:19.0.0-3450.g9a047fe7.el9.x86_64 9/156 2024-05-03T14:32:48.547 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-rados-2:19.0.0-3450.g9a047fe7.el9.x86_64 10/156 2024-05-03T14:32:48.897 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-05-03T14:32:48.962 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-05-03T14:32:48.984 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libcephsqlite-2:19.0.0-3450.g9a047fe7.el9.x86_64 13/156 2024-05-03T14:32:49.141 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3450.g9a047fe7.el9.x86_64 13/156 2024-05-03T14:32:49.169 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libradosstriper1-2:19.0.0-3450.g9a047fe7.el9.x86 14/156 2024-05-03T14:32:49.265 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3450.g9a047fe7.el9.x86 14/156 2024-05-03T14:32:49.295 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-05-03T14:32:49.361 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-05-03T14:32:49.691 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-05-03T14:32:49.747 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-05-03T14:32:49.968 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-05-03T14:32:50.100 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-05-03T14:32:50.138 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-05-03T14:32:50.189 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-05-03T14:32:50.297 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-05-03T14:32:50.360 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-05-03T14:32:50.427 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-05-03T14:32:50.472 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-05-03T14:32:50.512 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-05-03T14:32:50.644 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-05-03T14:32:50.722 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-05-03T14:32:50.756 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-05-03T14:32:50.816 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-05-03T14:32:50.935 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-05-03T14:32:50.976 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-ceph-common-2:19.0.0-3450.g9a047fe7.el9. 31/156 2024-05-03T14:32:51.028 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-ceph-argparse-2:19.0.0-3450.g9a047fe7.el 32/156 2024-05-03T14:32:51.066 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cephfs-2:19.0.0-3450.g9a047fe7.el9.x86_6 33/156 2024-05-03T14:32:51.188 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-05-03T14:32:51.247 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-05-03T14:32:51.360 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-05-03T14:32:51.550 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-05-03T14:32:51.622 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-05-03T14:32:52.328 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-05-03T14:32:52.432 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-05-03T14:32:52.480 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-05-03T14:32:52.526 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-05-03T14:32:52.575 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : librados-devel-2:19.0.0-3450.g9a047fe7.el9.x86_6 43/156 2024-05-03T14:32:52.660 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-05-03T14:32:52.935 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-05-03T14:32:53.011 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-05-03T14:32:53.071 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-05-03T14:32:53.162 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-05-03T14:32:53.200 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-05-03T14:32:53.237 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-05-03T14:32:53.306 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-05-03T14:32:53.348 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-05-03T14:32:53.409 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-05-03T14:32:53.469 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-05-03T14:32:53.638 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-05-03T14:32:53.669 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-05-03T14:32:53.721 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-05-03T14:32:53.735 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-05-03T14:32:53.749 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-05-03T14:32:53.781 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-05-03T14:32:53.807 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-05-03T14:32:53.834 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-05-03T14:32:53.869 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-05-03T14:32:53.894 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-05-03T14:32:53.937 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-05-03T14:32:53.949 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-05-03T14:32:53.991 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-05-03T14:32:54.039 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-05-03T14:32:54.073 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-05-03T14:32:54.105 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-05-03T14:32:54.266 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-05-03T14:32:54.306 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-05-03T14:32:54.370 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : librados2-2:19.0.0-3450.g9a047fe7.el9.x86_64 8/156 2024-05-03T14:32:54.486 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-05-03T14:32:54.492 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librados2-2:19.0.0-3450.g9a047fe7.el9.x86_64 8/156 2024-05-03T14:32:54.513 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libcephfs2-2:19.0.0-3450.g9a047fe7.el9.x86_64 9/156 2024-05-03T14:32:54.529 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-05-03T14:32:54.560 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-05-03T14:32:54.605 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-05-03T14:32:54.629 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libcephfs2-2:19.0.0-3450.g9a047fe7.el9.x86_64 9/156 2024-05-03T14:32:54.665 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-05-03T14:32:54.667 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rados-2:19.0.0-3450.g9a047fe7.el9.x86_64 10/156 2024-05-03T14:32:54.716 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-05-03T14:32:54.758 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-05-03T14:32:54.791 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-05-03T14:32:54.819 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-05-03T14:32:54.835 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-05-03T14:32:54.873 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-05-03T14:32:54.901 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libcephsqlite-2:19.0.0-3450.g9a047fe7.el9.x86_64 13/156 2024-05-03T14:32:54.910 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-05-03T14:32:54.960 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-05-03T14:32:55.003 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3450.g9a047fe7.el9.x86_64 13/156 2024-05-03T14:32:55.026 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libradosstriper1-2:19.0.0-3450.g9a047fe7.el9.x86 14/156 2024-05-03T14:32:55.072 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3450.g9a047fe7.el9.x86 14/156 2024-05-03T14:32:55.094 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-05-03T14:32:55.107 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-05-03T14:32:55.117 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-05-03T14:32:55.165 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-05-03T14:32:55.209 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-05-03T14:32:55.280 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-05-03T14:32:55.311 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-05-03T14:32:55.344 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-05-03T14:32:55.430 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-05-03T14:32:55.452 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-05-03T14:32:55.498 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-05-03T14:32:55.546 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-05-03T14:32:55.577 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-05-03T14:32:55.611 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-05-03T14:32:55.676 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-05-03T14:32:55.719 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-05-03T14:32:55.762 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-05-03T14:32:55.795 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-05-03T14:32:55.823 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-05-03T14:32:55.892 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-05-03T14:32:55.944 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-05-03T14:32:55.975 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-05-03T14:32:56.010 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-05-03T14:32:56.079 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-05-03T14:32:56.113 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-ceph-common-2:19.0.0-3450.g9a047fe7.el9. 31/156 2024-05-03T14:32:56.157 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-ceph-argparse-2:19.0.0-3450.g9a047fe7.el 32/156 2024-05-03T14:32:56.185 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cephfs-2:19.0.0-3450.g9a047fe7.el9.x86_6 33/156 2024-05-03T14:32:56.255 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-05-03T14:32:56.300 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-05-03T14:32:56.374 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-05-03T14:32:56.491 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-05-03T14:32:56.507 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-05-03T14:32:56.555 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-05-03T14:32:56.560 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-05-03T14:32:56.589 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-05-03T14:32:56.721 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-05-03T14:32:56.792 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-05-03T14:32:56.903 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-05-03T14:32:56.973 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-05-03T14:32:57.088 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-05-03T14:32:57.164 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-05-03T14:32:57.193 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-05-03T14:32:57.225 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-05-03T14:32:57.236 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-05-03T14:32:57.253 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-05-03T14:32:57.283 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librados-devel-2:19.0.0-3450.g9a047fe7.el9.x86_6 43/156 2024-05-03T14:32:57.324 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-05-03T14:32:57.356 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-05-03T14:32:57.480 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-05-03T14:32:57.527 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-05-03T14:32:57.582 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-05-03T14:32:57.655 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-05-03T14:32:57.686 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-05-03T14:32:57.722 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-05-03T14:32:57.769 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-05-03T14:32:57.805 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-05-03T14:32:57.848 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-05-03T14:32:57.891 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-05-03T14:32:58.028 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-05-03T14:32:58.079 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-05-03T14:32:58.109 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-05-03T14:32:58.151 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-05-03T14:32:58.215 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-05-03T14:32:58.244 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-05-03T14:32:58.276 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-05-03T14:32:58.307 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-05-03T14:32:58.333 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-05-03T14:32:58.358 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-05-03T14:32:58.372 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-05-03T14:32:58.434 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-05-03T14:32:58.559 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-05-03T14:32:58.614 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-05-03T14:32:58.652 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-05-03T14:32:58.686 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-05-03T14:32:58.725 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-05-03T14:32:58.756 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-05-03T14:32:58.786 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-05-03T14:32:58.813 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-05-03T14:32:58.846 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-05-03T14:32:58.888 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-05-03T14:32:58.922 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-05-03T14:32:59.008 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-05-03T14:32:59.052 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-05-03T14:32:59.083 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-05-03T14:32:59.130 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-05-03T14:32:59.228 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-05-03T14:32:59.294 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-05-03T14:33:00.021 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-05-03T14:33:00.045 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-05-03T14:33:00.065 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-05-03T14:33:00.099 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-05-03T14:33:00.142 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-05-03T14:33:00.185 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-05-03T14:33:00.197 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-05-03T14:33:00.236 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-05-03T14:33:00.248 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-05-03T14:33:00.336 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-05-03T14:33:00.376 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-05-03T14:33:00.556 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-05-03T14:33:00.590 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-05-03T14:33:00.641 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-05-03T14:33:00.668 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-05-03T14:33:00.681 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : librbd1-2:19.0.0-3450.g9a047fe7.el9.x86_64 99/156 2024-05-03T14:33:00.741 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3450.g9a047fe7.el9.x86_64 107/156 2024-05-03T14:33:00.741 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /sys 2024-05-03T14:33:00.741 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /proc 2024-05-03T14:33:00.741 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /mnt 2024-05-03T14:33:00.741 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /var/tmp 2024-05-03T14:33:00.741 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /home 2024-05-03T14:33:00.741 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /root 2024-05-03T14:33:00.741 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /tmp 2024-05-03T14:33:00.741 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-03T14:33:00.804 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librbd1-2:19.0.0-3450.g9a047fe7.el9.x86_64 99/156 2024-05-03T14:33:00.837 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-rbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 100/156 2024-05-03T14:33:00.895 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-05-03T14:33:00.910 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-volume-2:19.0.0-3450.g9a047fe7.el9.noarch 108/156 2024-05-03T14:33:00.928 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-volume-2:19.0.0-3450.g9a047fe7.el9.noarch 108/156 2024-05-03T14:33:00.928 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T14:33:00.928 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-03T14:33:00.928 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-03T14:33:01.314 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-05-03T14:33:01.430 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-05-03T14:33:01.484 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-05-03T14:33:01.517 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : librgw2-2:19.0.0-3450.g9a047fe7.el9.x86_64 103/156 2024-05-03T14:33:01.610 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librgw2-2:19.0.0-3450.g9a047fe7.el9.x86_64 103/156 2024-05-03T14:33:01.636 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-rgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 104/156 2024-05-03T14:33:02.423 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-osd-2:19.0.0-3450.g9a047fe7.el9.x86_64 109/156 2024-05-03T14:33:02.449 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-osd-2:19.0.0-3450.g9a047fe7.el9.x86_64 109/156 2024-05-03T14:33:02.449 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T14:33:02.449 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-03T14:33:02.449 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-03T14:33:02.449 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-03T14:33:02.449 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-03T14:33:02.656 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-05-03T14:33:02.674 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mds-2:19.0.0-3450.g9a047fe7.el9.x86_64 110/156 2024-05-03T14:33:02.697 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mds-2:19.0.0-3450.g9a047fe7.el9.x86_64 110/156 2024-05-03T14:33:02.697 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T14:33:02.697 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-03T14:33:02.697 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-03T14:33:02.697 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-03T14:33:02.697 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-03T14:33:02.744 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-05-03T14:33:02.771 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-05-03T14:33:02.805 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-05-03T14:33:03.119 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-05-03T14:33:03.150 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mon-2:19.0.0-3450.g9a047fe7.el9.x86_64 111/156 2024-05-03T14:33:03.168 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : librbd1-2:19.0.0-3450.g9a047fe7.el9.x86_64 99/156 2024-05-03T14:33:03.172 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mon-2:19.0.0-3450.g9a047fe7.el9.x86_64 111/156 2024-05-03T14:33:03.172 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T14:33:03.172 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-03T14:33:03.172 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-03T14:33:03.173 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-03T14:33:03.173 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-03T14:33:03.258 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librbd1-2:19.0.0-3450.g9a047fe7.el9.x86_64 99/156 2024-05-03T14:33:03.280 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-05-03T14:33:03.289 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 100/156 2024-05-03T14:33:03.325 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-05-03T14:33:03.337 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-05-03T14:33:03.392 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-05-03T14:33:03.586 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-05-03T14:33:03.661 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-05-03T14:33:03.739 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-05-03T14:33:03.804 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-05-03T14:33:03.827 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-05-03T14:33:03.845 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librgw2-2:19.0.0-3450.g9a047fe7.el9.x86_64 103/156 2024-05-03T14:33:03.916 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librgw2-2:19.0.0-3450.g9a047fe7.el9.x86_64 103/156 2024-05-03T14:33:03.939 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 104/156 2024-05-03T14:33:03.966 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-05-03T14:33:04.011 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-05-03T14:33:04.044 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-05-03T14:33:04.080 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-05-03T14:33:04.474 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 105/156 2024-05-03T14:33:04.541 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 105/156 2024-05-03T14:33:04.550 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 123/156 2024-05-03T14:33:04.580 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 123/156 2024-05-03T14:33:04.620 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3450.g9a047fe7.e 124/156 2024-05-03T14:33:04.690 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3450.g9a047fe7. 125/156 2024-05-03T14:33:04.720 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3450.g9a047fe7.el9.noa 126/156 2024-05-03T14:33:05.234 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 105/156 2024-05-03T14:33:05.286 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-base-2:19.0.0-3450.g9a047fe7.el9.x86_64 106/156 2024-05-03T14:33:05.462 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-base-2:19.0.0-3450.g9a047fe7.el9.x86_64 106/156 2024-05-03T14:33:05.462 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-03T14:33:05.463 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-03T14:33:05.463 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:33:05.503 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-selinux-2:19.0.0-3450.g9a047fe7.el9.x86_64 107/156 2024-05-03T14:33:05.758 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3450.g9a047fe7.el9.noa 126/156 2024-05-03T14:33:05.797 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3450.g9a047fe7.el9.n 127/156 2024-05-03T14:33:06.237 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 105/156 2024-05-03T14:33:06.276 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 105/156 2024-05-03T14:33:06.766 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3450.g9a047fe7.el9.n 127/156 2024-05-03T14:33:06.817 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 105/156 2024-05-03T14:33:06.831 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3450.g9a0 128/156 2024-05-03T14:33:06.863 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-base-2:19.0.0-3450.g9a047fe7.el9.x86_64 106/156 2024-05-03T14:33:06.943 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3450.g9a0 128/156 2024-05-03T14:33:06.956 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-base-2:19.0.0-3450.g9a047fe7.el9.x86_64 106/156 2024-05-03T14:33:06.956 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-03T14:33:06.956 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-03T14:33:06.956 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-03T14:33:06.989 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-selinux-2:19.0.0-3450.g9a047fe7.el9.x86_64 107/156 2024-05-03T14:33:07.104 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3450.g9a047fe7.el 129/156 2024-05-03T14:33:07.135 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-2:19.0.0-3450.g9a047fe7.el9.x86_64 130/156 2024-05-03T14:33:07.158 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3450.g9a047fe7.el9.x86_64 130/156 2024-05-03T14:33:07.158 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T14:33:07.158 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-03T14:33:07.158 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-03T14:33:07.158 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-03T14:33:07.158 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-03T14:33:07.211 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-rook-2:19.0.0-3450.g9a047fe7.el9.noarch 131/156 2024-05-03T14:33:07.222 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3450.g9a047fe7.el9.noarch 131/156 2024-05-03T14:33:07.321 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-2:19.0.0-3450.g9a047fe7.el9.x86_64 132/156 2024-05-03T14:33:07.375 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 133/156 2024-05-03T14:33:08.302 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-05-03T14:33:08.372 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-radosgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 135/156 2024-05-03T14:33:08.395 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 135/156 2024-05-03T14:33:08.395 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T14:33:08.395 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-03T14:33:08.395 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-03T14:33:08.395 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-03T14:33:08.395 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-03T14:33:08.440 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3450.g9a047 136/156 2024-05-03T14:33:08.462 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3450.g9a047 136/156 2024-05-03T14:33:08.462 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T14:33:08.462 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-03T14:33:08.462 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-03T14:33:08.515 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : cephfs-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 137/156 2024-05-03T14:33:08.538 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 137/156 2024-05-03T14:33:08.538 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T14:33:08.538 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-03T14:33:08.538 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-03T14:33:08.538 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-03T14:33:08.538 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-03T14:33:08.841 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : rbd-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 138/156 2024-05-03T14:33:08.863 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 138/156 2024-05-03T14:33:08.863 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T14:33:08.863 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-03T14:33:08.863 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-03T14:33:08.863 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-03T14:33:08.864 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-03T14:33:13.464 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-test-2:19.0.0-3450.g9a047fe7.el9.x86_64 139/156 2024-05-03T14:33:13.505 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : rbd-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 140/156 2024-05-03T14:33:13.546 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : rbd-nbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 141/156 2024-05-03T14:33:13.578 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-05-03T14:33:13.614 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-05-03T14:33:13.799 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : cephfs-shell-2:19.0.0-3450.g9a047fe7.el9.x86_64 144/156 2024-05-03T14:33:13.862 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-05-03T14:33:13.892 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libcephfs-devel-2:19.0.0-3450.g9a047fe7.el9.x86_ 146/156 2024-05-03T14:33:14.518 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-03T14:33:14.559 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-03T14:33:14.579 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-03T14:33:15.413 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : cephfs-top-2:19.0.0-3450.g9a047fe7.el9.noarch 148/156 2024-05-03T14:33:15.518 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-05-03T14:33:15.558 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-05-03T14:33:15.647 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-05-03T14:33:15.681 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-05-03T14:33:15.681 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-05-03T14:33:15.765 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-05-03T14:33:15.789 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-05-03T14:33:15.790 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-05-03T14:33:15.892 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-05-03T14:33:15.965 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-03T14:33:18.769 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3450.g9a047fe7.el9.x86_64 107/156 2024-05-03T14:33:18.769 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /sys 2024-05-03T14:33:18.769 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /proc 2024-05-03T14:33:18.770 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /mnt 2024-05-03T14:33:18.770 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /var/tmp 2024-05-03T14:33:18.770 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /home 2024-05-03T14:33:18.770 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /root 2024-05-03T14:33:18.770 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /tmp 2024-05-03T14:33:18.770 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-03T14:33:18.770 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3450.g9a047fe7.el9.x86_64 107/156 2024-05-03T14:33:18.770 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /sys 2024-05-03T14:33:18.770 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /proc 2024-05-03T14:33:18.770 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /mnt 2024-05-03T14:33:18.770 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /var/tmp 2024-05-03T14:33:18.771 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /home 2024-05-03T14:33:18.771 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /root 2024-05-03T14:33:18.771 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /tmp 2024-05-03T14:33:18.771 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:33:18.946 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-volume-2:19.0.0-3450.g9a047fe7.el9.noarch 108/156 2024-05-03T14:33:18.963 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-volume-2:19.0.0-3450.g9a047fe7.el9.noarch 108/156 2024-05-03T14:33:18.963 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T14:33:18.963 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-03T14:33:18.963 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-03T14:33:18.996 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-volume-2:19.0.0-3450.g9a047fe7.el9.noarch 108/156 2024-05-03T14:33:19.043 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-volume-2:19.0.0-3450.g9a047fe7.el9.noarch 108/156 2024-05-03T14:33:19.043 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T14:33:19.043 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-03T14:33:19.043 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:33:20.441 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-osd-2:19.0.0-3450.g9a047fe7.el9.x86_64 109/156 2024-05-03T14:33:20.466 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-osd-2:19.0.0-3450.g9a047fe7.el9.x86_64 109/156 2024-05-03T14:33:20.466 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T14:33:20.466 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-03T14:33:20.466 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-03T14:33:20.467 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-03T14:33:20.467 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-03T14:33:20.691 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mds-2:19.0.0-3450.g9a047fe7.el9.x86_64 110/156 2024-05-03T14:33:20.700 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-osd-2:19.0.0-3450.g9a047fe7.el9.x86_64 109/156 2024-05-03T14:33:20.714 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mds-2:19.0.0-3450.g9a047fe7.el9.x86_64 110/156 2024-05-03T14:33:20.714 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T14:33:20.714 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-03T14:33:20.714 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-03T14:33:20.714 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-03T14:33:20.714 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-03T14:33:20.801 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-osd-2:19.0.0-3450.g9a047fe7.el9.x86_64 109/156 2024-05-03T14:33:20.801 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T14:33:20.801 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-03T14:33:20.801 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-03T14:33:20.801 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-03T14:33:20.801 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:33:21.084 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mds-2:19.0.0-3450.g9a047fe7.el9.x86_64 110/156 2024-05-03T14:33:21.150 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mon-2:19.0.0-3450.g9a047fe7.el9.x86_64 111/156 2024-05-03T14:33:21.173 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mon-2:19.0.0-3450.g9a047fe7.el9.x86_64 111/156 2024-05-03T14:33:21.173 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T14:33:21.173 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-03T14:33:21.173 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-03T14:33:21.173 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-03T14:33:21.173 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-03T14:33:21.182 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mds-2:19.0.0-3450.g9a047fe7.el9.x86_64 110/156 2024-05-03T14:33:21.182 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T14:33:21.182 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-03T14:33:21.182 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-03T14:33:21.183 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-03T14:33:21.183 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:33:21.281 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-05-03T14:33:21.338 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-05-03T14:33:21.393 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-05-03T14:33:21.582 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-05-03T14:33:21.635 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-05-03T14:33:21.701 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mon-2:19.0.0-3450.g9a047fe7.el9.x86_64 111/156 2024-05-03T14:33:21.713 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-05-03T14:33:21.774 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mon-2:19.0.0-3450.g9a047fe7.el9.x86_64 111/156 2024-05-03T14:33:21.774 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T14:33:21.774 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-03T14:33:21.774 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-03T14:33:21.774 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-03T14:33:21.774 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:33:21.802 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-05-03T14:33:21.919 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-05-03T14:33:21.942 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-05-03T14:33:21.985 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-05-03T14:33:21.993 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-05-03T14:33:22.025 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-05-03T14:33:22.052 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-05-03T14:33:22.059 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-05-03T14:33:22.307 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-05-03T14:33:22.382 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-05-03T14:33:22.478 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-05-03T14:33:22.505 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 123/156 2024-05-03T14:33:22.526 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 123/156 2024-05-03T14:33:22.549 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3450.g9a047fe7.e 124/156 2024-05-03T14:33:22.589 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-05-03T14:33:22.612 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3450.g9a047fe7. 125/156 2024-05-03T14:33:22.640 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3450.g9a047fe7.el9.noa 126/156 2024-05-03T14:33:22.775 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-05-03T14:33:22.823 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-05-03T14:33:22.863 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-05-03T14:33:22.904 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-05-03T14:33:23.539 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 123/156 2024-05-03T14:33:23.575 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 123/156 2024-05-03T14:33:23.613 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3450.g9a047fe7.e 124/156 2024-05-03T14:33:23.689 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3450.g9a047fe7.el9.noa 126/156 2024-05-03T14:33:23.720 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3450.g9a047fe7. 125/156 2024-05-03T14:33:23.726 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3450.g9a047fe7.el9.n 127/156 2024-05-03T14:33:23.759 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3450.g9a047fe7.el9.noa 126/156 2024-05-03T14:33:24.044 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-03T14:33:24.045 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-2:19.0.0-3450.g9a047fe7.el9.x86_64 1/156 2024-05-03T14:33:24.045 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-base-2:19.0.0-3450.g9a047fe7.el9.x86_64 2/156 2024-05-03T14:33:24.045 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 3/156 2024-05-03T14:33:24.045 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 4/156 2024-05-03T14:33:24.045 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3450.g9a047 5/156 2024-05-03T14:33:24.045 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mds-2:19.0.0-3450.g9a047fe7.el9.x86_64 6/156 2024-05-03T14:33:24.045 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-2:19.0.0-3450.g9a047fe7.el9.x86_64 7/156 2024-05-03T14:33:24.045 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mon-2:19.0.0-3450.g9a047fe7.el9.x86_64 8/156 2024-05-03T14:33:24.045 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-osd-2:19.0.0-3450.g9a047fe7.el9.x86_64 9/156 2024-05-03T14:33:24.045 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-radosgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 10/156 2024-05-03T14:33:24.045 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-selinux-2:19.0.0-3450.g9a047fe7.el9.x86_64 11/156 2024-05-03T14:33:24.046 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-test-2:19.0.0-3450.g9a047fe7.el9.x86_64 12/156 2024-05-03T14:33:24.046 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : cephfs-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 13/156 2024-05-03T14:33:24.046 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : cephfs-shell-2:19.0.0-3450.g9a047fe7.el9.x86_64 14/156 2024-05-03T14:33:24.046 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libcephfs-devel-2:19.0.0-3450.g9a047fe7.el9.x86_ 15/156 2024-05-03T14:33:24.046 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libcephfs2-2:19.0.0-3450.g9a047fe7.el9.x86_64 16/156 2024-05-03T14:33:24.046 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libcephsqlite-2:19.0.0-3450.g9a047fe7.el9.x86_64 17/156 2024-05-03T14:33:24.046 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librados-devel-2:19.0.0-3450.g9a047fe7.el9.x86_6 18/156 2024-05-03T14:33:24.046 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libradosstriper1-2:19.0.0-3450.g9a047fe7.el9.x86 19/156 2024-05-03T14:33:24.046 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librgw2-2:19.0.0-3450.g9a047fe7.el9.x86_64 20/156 2024-05-03T14:33:24.046 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3450.g9a047fe7.el 21/156 2024-05-03T14:33:24.046 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-ceph-common-2:19.0.0-3450.g9a047fe7.el9. 22/156 2024-05-03T14:33:24.046 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cephfs-2:19.0.0-3450.g9a047fe7.el9.x86_6 23/156 2024-05-03T14:33:24.046 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-rados-2:19.0.0-3450.g9a047fe7.el9.x86_64 24/156 2024-05-03T14:33:24.047 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-rbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 25/156 2024-05-03T14:33:24.047 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-rgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 26/156 2024-05-03T14:33:24.047 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : rbd-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 27/156 2024-05-03T14:33:24.047 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : rbd-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 28/156 2024-05-03T14:33:24.047 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : rbd-nbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 29/156 2024-05-03T14:33:24.047 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3450.g9a047fe7. 30/156 2024-05-03T14:33:24.047 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3450.g9a047fe7.el9.noa 31/156 2024-05-03T14:33:24.047 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3450.g9a047fe7.el9.n 32/156 2024-05-03T14:33:24.047 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3450.g9a0 33/156 2024-05-03T14:33:24.047 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3450.g9a047fe7.el 34/156 2024-05-03T14:33:24.047 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3450.g9a047fe7.el9.noarch 35/156 2024-05-03T14:33:24.047 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3450.g9a047fe7.e 36/156 2024-05-03T14:33:24.047 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-volume-2:19.0.0-3450.g9a047fe7.el9.noarch 37/156 2024-05-03T14:33:24.048 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 38/156 2024-05-03T14:33:24.048 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : cephfs-top-2:19.0.0-3450.g9a047fe7.el9.noarch 39/156 2024-05-03T14:33:24.048 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-05-03T14:33:24.048 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-05-03T14:33:24.048 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-05-03T14:33:24.048 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-05-03T14:33:24.048 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-05-03T14:33:24.048 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-05-03T14:33:24.048 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-05-03T14:33:24.048 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-05-03T14:33:24.048 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-05-03T14:33:24.048 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-05-03T14:33:24.049 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-05-03T14:33:24.049 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-05-03T14:33:24.049 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-05-03T14:33:24.049 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-05-03T14:33:24.049 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-05-03T14:33:24.049 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-05-03T14:33:24.049 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-05-03T14:33:24.049 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-05-03T14:33:24.049 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-05-03T14:33:24.049 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-05-03T14:33:24.049 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-05-03T14:33:24.049 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-05-03T14:33:24.049 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-05-03T14:33:24.049 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-05-03T14:33:24.050 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-05-03T14:33:24.050 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-05-03T14:33:24.050 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-05-03T14:33:24.050 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-05-03T14:33:24.050 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-05-03T14:33:24.050 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-05-03T14:33:24.050 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-05-03T14:33:24.050 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-05-03T14:33:24.050 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-05-03T14:33:24.050 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-05-03T14:33:24.050 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-05-03T14:33:24.050 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-05-03T14:33:24.051 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-05-03T14:33:24.051 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-05-03T14:33:24.051 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-05-03T14:33:24.051 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-05-03T14:33:24.051 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-05-03T14:33:24.051 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-05-03T14:33:24.051 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-05-03T14:33:24.051 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-05-03T14:33:24.051 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-05-03T14:33:24.051 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-05-03T14:33:24.051 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-05-03T14:33:24.051 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-05-03T14:33:24.051 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-05-03T14:33:24.052 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-05-03T14:33:24.052 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-05-03T14:33:24.052 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-05-03T14:33:24.052 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-05-03T14:33:24.052 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-05-03T14:33:24.052 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-05-03T14:33:24.052 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-05-03T14:33:24.052 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-05-03T14:33:24.052 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-05-03T14:33:24.052 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-05-03T14:33:24.052 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-05-03T14:33:24.052 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-05-03T14:33:24.052 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-05-03T14:33:24.052 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-05-03T14:33:24.053 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-05-03T14:33:24.053 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-05-03T14:33:24.053 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-05-03T14:33:24.053 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-05-03T14:33:24.053 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-05-03T14:33:24.053 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-05-03T14:33:24.053 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-05-03T14:33:24.053 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-05-03T14:33:24.053 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-05-03T14:33:24.053 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-05-03T14:33:24.053 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-05-03T14:33:24.053 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-05-03T14:33:24.053 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-05-03T14:33:24.054 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-05-03T14:33:24.054 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-05-03T14:33:24.054 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-05-03T14:33:24.054 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-05-03T14:33:24.054 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-05-03T14:33:24.054 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-05-03T14:33:24.054 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-05-03T14:33:24.054 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-05-03T14:33:24.054 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-05-03T14:33:24.054 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-05-03T14:33:24.054 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-05-03T14:33:24.054 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-05-03T14:33:24.055 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-05-03T14:33:24.055 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-05-03T14:33:24.055 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-05-03T14:33:24.055 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-05-03T14:33:24.055 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-05-03T14:33:24.055 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-05-03T14:33:24.055 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-05-03T14:33:24.055 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-05-03T14:33:24.055 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-05-03T14:33:24.055 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-05-03T14:33:24.055 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-05-03T14:33:24.055 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-05-03T14:33:24.055 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-05-03T14:33:24.056 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-05-03T14:33:24.056 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-05-03T14:33:24.056 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-05-03T14:33:24.056 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-05-03T14:33:24.056 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-05-03T14:33:24.056 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-05-03T14:33:24.056 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-05-03T14:33:24.056 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-05-03T14:33:24.056 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librados2-2:19.0.0-3450.g9a047fe7.el9.x86_64 149/156 2024-05-03T14:33:24.056 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-05-03T14:33:24.056 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librbd1-2:19.0.0-3450.g9a047fe7.el9.x86_64 151/156 2024-05-03T14:33:24.056 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-05-03T14:33:24.056 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-05-03T14:33:24.056 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-05-03T14:33:24.057 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-05-03T14:33:24.689 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3450.g9a047fe7.el9.n 127/156 2024-05-03T14:33:24.718 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3450.g9a0 128/156 2024-05-03T14:33:24.831 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3450.g9a0 128/156 2024-05-03T14:33:24.990 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3450.g9a047fe7.el 129/156 2024-05-03T14:33:25.020 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-2:19.0.0-3450.g9a047fe7.el9.x86_64 130/156 2024-05-03T14:33:25.043 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3450.g9a047fe7.el9.x86_64 130/156 2024-05-03T14:33:25.043 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T14:33:25.043 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-03T14:33:25.043 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-03T14:33:25.043 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-03T14:33:25.044 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-03T14:33:25.087 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-rook-2:19.0.0-3450.g9a047fe7.el9.noarch 131/156 2024-05-03T14:33:25.098 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3450.g9a047fe7.el9.noarch 131/156 2024-05-03T14:33:25.126 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3450.g9a047fe7.el9.noa 126/156 2024-05-03T14:33:25.169 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3450.g9a047fe7.el9.n 127/156 2024-05-03T14:33:25.197 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-2:19.0.0-3450.g9a047fe7.el9.x86_64 132/156 2024-05-03T14:33:25.243 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 133/156 2024-05-03T14:33:26.150 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-03T14:33:26.151 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-03T14:33:26.151 INFO:teuthology.orchestra.run.smithi090.stdout:Upgraded: 2024-05-03T14:33:26.151 INFO:teuthology.orchestra.run.smithi090.stdout: librados2-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:26.151 INFO:teuthology.orchestra.run.smithi090.stdout: librbd1-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:26.151 INFO:teuthology.orchestra.run.smithi090.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-03T14:33:26.151 INFO:teuthology.orchestra.run.smithi090.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-03T14:33:26.151 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-05-03T14:33:26.151 INFO:teuthology.orchestra.run.smithi090.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-03T14:33:26.151 INFO:teuthology.orchestra.run.smithi090.stdout: automake-1.16.2-8.el9.noarch 2024-05-03T14:33:26.151 INFO:teuthology.orchestra.run.smithi090.stdout: bison-3.7.4-5.el9.x86_64 2024-05-03T14:33:26.151 INFO:teuthology.orchestra.run.smithi090.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-03T14:33:26.152 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:26.152 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-base-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:26.152 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:26.152 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:26.152 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-grafana-dashboards-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T14:33:26.152 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-immutable-object-cache-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:26.152 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mds-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:26.152 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:26.152 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T14:33:26.152 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-dashboard-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T14:33:26.152 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T14:33:26.152 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-modules-core-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T14:33:26.152 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-rook-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T14:33:26.153 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mon-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:26.153 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-osd-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:26.153 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-prometheus-alerts-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T14:33:26.153 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-radosgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:26.153 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-selinux-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:26.153 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-test-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:26.153 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-volume-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T14:33:26.153 INFO:teuthology.orchestra.run.smithi090.stdout: cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T14:33:26.153 INFO:teuthology.orchestra.run.smithi090.stdout: cephfs-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:26.153 INFO:teuthology.orchestra.run.smithi090.stdout: cephfs-shell-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:26.153 INFO:teuthology.orchestra.run.smithi090.stdout: cephfs-top-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T14:33:26.153 INFO:teuthology.orchestra.run.smithi090.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-03T14:33:26.154 INFO:teuthology.orchestra.run.smithi090.stdout: flex-2.6.4-9.el9.x86_64 2024-05-03T14:33:26.154 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-03T14:33:26.154 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-03T14:33:26.154 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-03T14:33:26.154 INFO:teuthology.orchestra.run.smithi090.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-03T14:33:26.154 INFO:teuthology.orchestra.run.smithi090.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-03T14:33:26.154 INFO:teuthology.orchestra.run.smithi090.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-03T14:33:26.154 INFO:teuthology.orchestra.run.smithi090.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-03T14:33:26.154 INFO:teuthology.orchestra.run.smithi090.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-03T14:33:26.154 INFO:teuthology.orchestra.run.smithi090.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-03T14:33:26.154 INFO:teuthology.orchestra.run.smithi090.stdout: libcephfs-devel-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:26.154 INFO:teuthology.orchestra.run.smithi090.stdout: libcephfs2-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:26.154 INFO:teuthology.orchestra.run.smithi090.stdout: libcephsqlite-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:26.155 INFO:teuthology.orchestra.run.smithi090.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-03T14:33:26.155 INFO:teuthology.orchestra.run.smithi090.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-03T14:33:26.155 INFO:teuthology.orchestra.run.smithi090.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-03T14:33:26.155 INFO:teuthology.orchestra.run.smithi090.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-03T14:33:26.155 INFO:teuthology.orchestra.run.smithi090.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-03T14:33:26.155 INFO:teuthology.orchestra.run.smithi090.stdout: librados-devel-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:26.155 INFO:teuthology.orchestra.run.smithi090.stdout: libradosstriper1-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:26.155 INFO:teuthology.orchestra.run.smithi090.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-03T14:33:26.155 INFO:teuthology.orchestra.run.smithi090.stdout: librgw2-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:26.155 INFO:teuthology.orchestra.run.smithi090.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-03T14:33:26.155 INFO:teuthology.orchestra.run.smithi090.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-03T14:33:26.155 INFO:teuthology.orchestra.run.smithi090.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-03T14:33:26.155 INFO:teuthology.orchestra.run.smithi090.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-03T14:33:26.156 INFO:teuthology.orchestra.run.smithi090.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-03T14:33:26.156 INFO:teuthology.orchestra.run.smithi090.stdout: lua-5.4.4-4.el9.x86_64 2024-05-03T14:33:26.156 INFO:teuthology.orchestra.run.smithi090.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-03T14:33:26.156 INFO:teuthology.orchestra.run.smithi090.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-03T14:33:26.156 INFO:teuthology.orchestra.run.smithi090.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-03T14:33:26.156 INFO:teuthology.orchestra.run.smithi090.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-03T14:33:26.156 INFO:teuthology.orchestra.run.smithi090.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-03T14:33:26.156 INFO:teuthology.orchestra.run.smithi090.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-03T14:33:26.156 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-13.14-1.el9.x86_64 2024-05-03T14:33:26.156 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-03T14:33:26.156 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-03T14:33:26.156 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-03T14:33:26.156 INFO:teuthology.orchestra.run.smithi090.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-03T14:33:26.157 INFO:teuthology.orchestra.run.smithi090.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-03T14:33:26.157 INFO:teuthology.orchestra.run.smithi090.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-03T14:33:26.157 INFO:teuthology.orchestra.run.smithi090.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-03T14:33:26.157 INFO:teuthology.orchestra.run.smithi090.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-03T14:33:26.157 INFO:teuthology.orchestra.run.smithi090.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-03T14:33:26.157 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-03T14:33:26.157 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-argparse-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:26.157 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:26.157 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cephfs-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:26.157 INFO:teuthology.orchestra.run.smithi090.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-03T14:33:26.157 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-03T14:33:26.157 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-03T14:33:26.157 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-03T14:33:26.157 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-03T14:33:26.158 INFO:teuthology.orchestra.run.smithi090.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-03T14:33:26.158 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-03T14:33:26.158 INFO:teuthology.orchestra.run.smithi090.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-03T14:33:26.158 INFO:teuthology.orchestra.run.smithi090.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-03T14:33:26.158 INFO:teuthology.orchestra.run.smithi090.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-03T14:33:26.158 INFO:teuthology.orchestra.run.smithi090.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-03T14:33:26.158 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-03T14:33:26.158 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-03T14:33:26.158 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-03T14:33:26.158 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-03T14:33:26.158 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-03T14:33:26.158 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-03T14:33:26.158 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-03T14:33:26.158 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-03T14:33:26.158 INFO:teuthology.orchestra.run.smithi090.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-03T14:33:26.159 INFO:teuthology.orchestra.run.smithi090.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-03T14:33:26.159 INFO:teuthology.orchestra.run.smithi090.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-03T14:33:26.159 INFO:teuthology.orchestra.run.smithi090.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-03T14:33:26.159 INFO:teuthology.orchestra.run.smithi090.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-03T14:33:26.159 INFO:teuthology.orchestra.run.smithi090.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-03T14:33:26.159 INFO:teuthology.orchestra.run.smithi090.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-03T14:33:26.159 INFO:teuthology.orchestra.run.smithi090.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-03T14:33:26.159 INFO:teuthology.orchestra.run.smithi090.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-03T14:33:26.159 INFO:teuthology.orchestra.run.smithi090.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-03T14:33:26.159 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-03T14:33:26.159 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ply-3.11-14.el9.noarch 2024-05-03T14:33:26.159 INFO:teuthology.orchestra.run.smithi090.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-03T14:33:26.159 INFO:teuthology.orchestra.run.smithi090.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-03T14:33:26.159 INFO:teuthology.orchestra.run.smithi090.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-03T14:33:26.159 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-03T14:33:26.159 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-03T14:33:26.160 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-03T14:33:26.160 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-03T14:33:26.160 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-03T14:33:26.160 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-03T14:33:26.160 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rados-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:26.160 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:26.160 INFO:teuthology.orchestra.run.smithi090.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-03T14:33:26.160 INFO:teuthology.orchestra.run.smithi090.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-03T14:33:26.160 INFO:teuthology.orchestra.run.smithi090.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-03T14:33:26.160 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:26.160 INFO:teuthology.orchestra.run.smithi090.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-03T14:33:26.160 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-03T14:33:26.160 INFO:teuthology.orchestra.run.smithi090.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-03T14:33:26.161 INFO:teuthology.orchestra.run.smithi090.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-03T14:33:26.161 INFO:teuthology.orchestra.run.smithi090.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-03T14:33:26.161 INFO:teuthology.orchestra.run.smithi090.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-03T14:33:26.161 INFO:teuthology.orchestra.run.smithi090.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-03T14:33:26.161 INFO:teuthology.orchestra.run.smithi090.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-03T14:33:26.161 INFO:teuthology.orchestra.run.smithi090.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-03T14:33:26.161 INFO:teuthology.orchestra.run.smithi090.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-03T14:33:26.161 INFO:teuthology.orchestra.run.smithi090.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-03T14:33:26.161 INFO:teuthology.orchestra.run.smithi090.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-03T14:33:26.161 INFO:teuthology.orchestra.run.smithi090.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-03T14:33:26.161 INFO:teuthology.orchestra.run.smithi090.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-03T14:33:26.161 INFO:teuthology.orchestra.run.smithi090.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-03T14:33:26.161 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:26.161 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:26.162 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-nbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:26.162 INFO:teuthology.orchestra.run.smithi090.stdout: re2-1:20211101-3.el9.x86_64 2024-05-03T14:33:26.162 INFO:teuthology.orchestra.run.smithi090.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-03T14:33:26.162 INFO:teuthology.orchestra.run.smithi090.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-03T14:33:26.162 INFO:teuthology.orchestra.run.smithi090.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-03T14:33:26.162 INFO:teuthology.orchestra.run.smithi090.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-03T14:33:26.162 INFO:teuthology.orchestra.run.smithi090.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-03T14:33:26.162 INFO:teuthology.orchestra.run.smithi090.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-03T14:33:26.163 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-03T14:33:26.163 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-05-03T14:33:26.175 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-05-03T14:33:26.206 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-radosgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 135/156 2024-05-03T14:33:26.229 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 135/156 2024-05-03T14:33:26.229 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T14:33:26.229 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-03T14:33:26.229 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-03T14:33:26.229 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-03T14:33:26.230 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-03T14:33:26.273 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3450.g9a047 136/156 2024-05-03T14:33:26.295 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3450.g9a047 136/156 2024-05-03T14:33:26.296 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T14:33:26.296 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-03T14:33:26.296 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-03T14:33:26.335 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3450.g9a047fe7.el9.n 127/156 2024-05-03T14:33:26.349 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : cephfs-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 137/156 2024-05-03T14:33:26.361 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3450.g9a0 128/156 2024-05-03T14:33:26.371 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 137/156 2024-05-03T14:33:26.372 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T14:33:26.372 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-03T14:33:26.372 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-03T14:33:26.372 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-03T14:33:26.372 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-03T14:33:26.527 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3450.g9a0 128/156 2024-05-03T14:33:26.587 DEBUG:teuthology.parallel:result is None 2024-05-03T14:33:26.675 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : rbd-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 138/156 2024-05-03T14:33:26.697 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 138/156 2024-05-03T14:33:26.697 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T14:33:26.697 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-03T14:33:26.697 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-03T14:33:26.698 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-03T14:33:26.698 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-03T14:33:26.716 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3450.g9a047fe7.el 129/156 2024-05-03T14:33:26.746 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-2:19.0.0-3450.g9a047fe7.el9.x86_64 130/156 2024-05-03T14:33:26.836 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3450.g9a047fe7.el9.x86_64 130/156 2024-05-03T14:33:26.836 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T14:33:26.836 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-03T14:33:26.836 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-03T14:33:26.836 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-03T14:33:26.836 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:33:26.905 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-rook-2:19.0.0-3450.g9a047fe7.el9.noarch 131/156 2024-05-03T14:33:26.943 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3450.g9a047fe7.el9.noarch 131/156 2024-05-03T14:33:27.104 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-2:19.0.0-3450.g9a047fe7.el9.x86_64 132/156 2024-05-03T14:33:27.177 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 133/156 2024-05-03T14:33:28.269 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-05-03T14:33:28.310 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-radosgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 135/156 2024-05-03T14:33:28.408 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 135/156 2024-05-03T14:33:28.408 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T14:33:28.408 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-03T14:33:28.408 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-03T14:33:28.408 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-03T14:33:28.408 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:33:28.459 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3450.g9a047 136/156 2024-05-03T14:33:28.556 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3450.g9a047 136/156 2024-05-03T14:33:28.557 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T14:33:28.557 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-03T14:33:28.557 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:33:28.643 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : cephfs-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 137/156 2024-05-03T14:33:28.748 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 137/156 2024-05-03T14:33:28.749 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T14:33:28.749 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-03T14:33:28.749 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-03T14:33:28.749 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-03T14:33:28.749 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:33:29.152 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : rbd-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 138/156 2024-05-03T14:33:29.248 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 138/156 2024-05-03T14:33:29.248 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T14:33:29.248 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-03T14:33:29.248 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-03T14:33:29.248 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-03T14:33:29.248 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:33:31.296 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-test-2:19.0.0-3450.g9a047fe7.el9.x86_64 139/156 2024-05-03T14:33:31.336 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : rbd-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 140/156 2024-05-03T14:33:31.386 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : rbd-nbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 141/156 2024-05-03T14:33:31.426 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-05-03T14:33:31.453 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-05-03T14:33:31.639 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : cephfs-shell-2:19.0.0-3450.g9a047fe7.el9.x86_64 144/156 2024-05-03T14:33:31.691 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-05-03T14:33:31.721 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libcephfs-devel-2:19.0.0-3450.g9a047fe7.el9.x86_ 146/156 2024-05-03T14:33:32.345 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-03T14:33:32.385 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-03T14:33:32.406 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-03T14:33:33.262 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : cephfs-top-2:19.0.0-3450.g9a047fe7.el9.noarch 148/156 2024-05-03T14:33:33.348 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-05-03T14:33:33.390 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-05-03T14:33:33.480 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-05-03T14:33:33.511 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-05-03T14:33:33.511 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-05-03T14:33:33.595 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-05-03T14:33:33.636 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-05-03T14:33:33.636 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-05-03T14:33:33.733 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-05-03T14:33:33.801 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-03T14:33:34.592 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-test-2:19.0.0-3450.g9a047fe7.el9.x86_64 139/156 2024-05-03T14:33:34.654 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : rbd-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 140/156 2024-05-03T14:33:34.743 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : rbd-nbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 141/156 2024-05-03T14:33:34.804 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-05-03T14:33:34.854 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-05-03T14:33:35.218 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : cephfs-shell-2:19.0.0-3450.g9a047fe7.el9.x86_64 144/156 2024-05-03T14:33:35.374 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-05-03T14:33:35.400 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libcephfs-devel-2:19.0.0-3450.g9a047fe7.el9.x86_ 146/156 2024-05-03T14:33:36.357 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-03T14:33:36.405 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-03T14:33:36.463 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-03T14:33:37.936 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : cephfs-top-2:19.0.0-3450.g9a047fe7.el9.noarch 148/156 2024-05-03T14:33:38.046 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-05-03T14:33:38.136 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-05-03T14:33:38.322 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-05-03T14:33:38.368 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-05-03T14:33:38.368 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-05-03T14:33:38.495 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-05-03T14:33:38.542 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-05-03T14:33:38.543 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-05-03T14:33:38.682 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-05-03T14:33:38.828 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-03T14:33:39.121 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-03T14:33:39.121 INFO:teuthology.orchestra.run.smithi060.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-05-03T14:33:39.121 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:33:39.711 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-03T14:33:39.712 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-2:19.0.0-3450.g9a047fe7.el9.x86_64 1/156 2024-05-03T14:33:39.712 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-base-2:19.0.0-3450.g9a047fe7.el9.x86_64 2/156 2024-05-03T14:33:39.712 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 3/156 2024-05-03T14:33:39.712 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 4/156 2024-05-03T14:33:39.712 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3450.g9a047 5/156 2024-05-03T14:33:39.712 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mds-2:19.0.0-3450.g9a047fe7.el9.x86_64 6/156 2024-05-03T14:33:39.712 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-2:19.0.0-3450.g9a047fe7.el9.x86_64 7/156 2024-05-03T14:33:39.712 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mon-2:19.0.0-3450.g9a047fe7.el9.x86_64 8/156 2024-05-03T14:33:39.712 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-osd-2:19.0.0-3450.g9a047fe7.el9.x86_64 9/156 2024-05-03T14:33:39.712 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-radosgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 10/156 2024-05-03T14:33:39.712 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-selinux-2:19.0.0-3450.g9a047fe7.el9.x86_64 11/156 2024-05-03T14:33:39.713 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-test-2:19.0.0-3450.g9a047fe7.el9.x86_64 12/156 2024-05-03T14:33:39.713 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : cephfs-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 13/156 2024-05-03T14:33:39.713 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : cephfs-shell-2:19.0.0-3450.g9a047fe7.el9.x86_64 14/156 2024-05-03T14:33:39.713 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libcephfs-devel-2:19.0.0-3450.g9a047fe7.el9.x86_ 15/156 2024-05-03T14:33:39.713 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libcephfs2-2:19.0.0-3450.g9a047fe7.el9.x86_64 16/156 2024-05-03T14:33:39.713 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libcephsqlite-2:19.0.0-3450.g9a047fe7.el9.x86_64 17/156 2024-05-03T14:33:39.713 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librados-devel-2:19.0.0-3450.g9a047fe7.el9.x86_6 18/156 2024-05-03T14:33:39.713 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libradosstriper1-2:19.0.0-3450.g9a047fe7.el9.x86 19/156 2024-05-03T14:33:39.713 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librgw2-2:19.0.0-3450.g9a047fe7.el9.x86_64 20/156 2024-05-03T14:33:39.713 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3450.g9a047fe7.el 21/156 2024-05-03T14:33:39.713 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-ceph-common-2:19.0.0-3450.g9a047fe7.el9. 22/156 2024-05-03T14:33:39.714 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cephfs-2:19.0.0-3450.g9a047fe7.el9.x86_6 23/156 2024-05-03T14:33:39.714 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rados-2:19.0.0-3450.g9a047fe7.el9.x86_64 24/156 2024-05-03T14:33:39.714 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 25/156 2024-05-03T14:33:39.714 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 26/156 2024-05-03T14:33:39.714 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : rbd-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 27/156 2024-05-03T14:33:39.714 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : rbd-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 28/156 2024-05-03T14:33:39.715 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : rbd-nbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 29/156 2024-05-03T14:33:39.715 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3450.g9a047fe7. 30/156 2024-05-03T14:33:39.715 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3450.g9a047fe7.el9.noa 31/156 2024-05-03T14:33:39.715 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3450.g9a047fe7.el9.n 32/156 2024-05-03T14:33:39.715 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3450.g9a0 33/156 2024-05-03T14:33:39.715 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3450.g9a047fe7.el 34/156 2024-05-03T14:33:39.715 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3450.g9a047fe7.el9.noarch 35/156 2024-05-03T14:33:39.715 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3450.g9a047fe7.e 36/156 2024-05-03T14:33:39.715 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-volume-2:19.0.0-3450.g9a047fe7.el9.noarch 37/156 2024-05-03T14:33:39.716 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 38/156 2024-05-03T14:33:39.716 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : cephfs-top-2:19.0.0-3450.g9a047fe7.el9.noarch 39/156 2024-05-03T14:33:39.716 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-05-03T14:33:39.716 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-05-03T14:33:39.716 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-05-03T14:33:39.716 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-05-03T14:33:39.716 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-05-03T14:33:39.716 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-05-03T14:33:39.716 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-05-03T14:33:39.716 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-05-03T14:33:39.716 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-05-03T14:33:39.717 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-05-03T14:33:39.717 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-05-03T14:33:39.717 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-05-03T14:33:39.717 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-05-03T14:33:39.717 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-05-03T14:33:39.717 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-05-03T14:33:39.717 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-05-03T14:33:39.717 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-05-03T14:33:39.717 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-05-03T14:33:39.717 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-05-03T14:33:39.717 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-05-03T14:33:39.717 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-05-03T14:33:39.717 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-05-03T14:33:39.718 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-05-03T14:33:39.718 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-05-03T14:33:39.718 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-05-03T14:33:39.718 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-05-03T14:33:39.718 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-05-03T14:33:39.718 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-05-03T14:33:39.718 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-05-03T14:33:39.718 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-05-03T14:33:39.718 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-05-03T14:33:39.718 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-05-03T14:33:39.718 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-05-03T14:33:39.718 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-05-03T14:33:39.718 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-05-03T14:33:39.718 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-05-03T14:33:39.719 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-05-03T14:33:39.719 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-05-03T14:33:39.719 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-05-03T14:33:39.719 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-05-03T14:33:39.719 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-05-03T14:33:39.719 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-05-03T14:33:39.719 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-05-03T14:33:39.719 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-05-03T14:33:39.719 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-05-03T14:33:39.719 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-05-03T14:33:39.719 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-05-03T14:33:39.719 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-05-03T14:33:39.719 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-05-03T14:33:39.719 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-05-03T14:33:39.720 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-05-03T14:33:39.720 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-05-03T14:33:39.720 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-05-03T14:33:39.720 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-05-03T14:33:39.720 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-05-03T14:33:39.720 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-05-03T14:33:39.720 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-05-03T14:33:39.721 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-05-03T14:33:39.721 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-05-03T14:33:39.721 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-05-03T14:33:39.721 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-05-03T14:33:39.721 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-05-03T14:33:39.721 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-05-03T14:33:39.722 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-05-03T14:33:39.722 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-05-03T14:33:39.722 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-05-03T14:33:39.722 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-05-03T14:33:39.722 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-05-03T14:33:39.722 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-05-03T14:33:39.722 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-05-03T14:33:39.722 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-05-03T14:33:39.722 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-05-03T14:33:39.722 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-05-03T14:33:39.722 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-05-03T14:33:39.722 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-05-03T14:33:39.723 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-05-03T14:33:39.723 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-05-03T14:33:39.723 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-05-03T14:33:39.723 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-05-03T14:33:39.723 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-05-03T14:33:39.723 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-05-03T14:33:39.723 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-05-03T14:33:39.723 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-05-03T14:33:39.723 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-05-03T14:33:39.723 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-05-03T14:33:39.723 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-05-03T14:33:39.723 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-05-03T14:33:39.723 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-05-03T14:33:39.723 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-05-03T14:33:39.723 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-05-03T14:33:39.724 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-05-03T14:33:39.724 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-05-03T14:33:39.724 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-05-03T14:33:39.724 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-05-03T14:33:39.724 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-05-03T14:33:39.724 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-05-03T14:33:39.724 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-05-03T14:33:39.724 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-05-03T14:33:39.724 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-05-03T14:33:39.724 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-05-03T14:33:39.724 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-05-03T14:33:39.724 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-05-03T14:33:39.724 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-05-03T14:33:39.724 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-05-03T14:33:39.724 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-05-03T14:33:39.725 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-05-03T14:33:39.725 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-05-03T14:33:39.725 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-05-03T14:33:39.725 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-05-03T14:33:39.725 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librados2-2:19.0.0-3450.g9a047fe7.el9.x86_64 149/156 2024-05-03T14:33:39.725 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-05-03T14:33:39.725 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librbd1-2:19.0.0-3450.g9a047fe7.el9.x86_64 151/156 2024-05-03T14:33:39.725 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-05-03T14:33:39.725 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-05-03T14:33:39.725 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-05-03T14:33:39.725 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-05-03T14:33:41.825 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-03T14:33:41.825 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-03T14:33:41.825 INFO:teuthology.orchestra.run.smithi149.stdout:Upgraded: 2024-05-03T14:33:41.825 INFO:teuthology.orchestra.run.smithi149.stdout: librados2-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:41.825 INFO:teuthology.orchestra.run.smithi149.stdout: librbd1-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:41.826 INFO:teuthology.orchestra.run.smithi149.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-03T14:33:41.826 INFO:teuthology.orchestra.run.smithi149.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-03T14:33:41.826 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-05-03T14:33:41.826 INFO:teuthology.orchestra.run.smithi149.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-03T14:33:41.826 INFO:teuthology.orchestra.run.smithi149.stdout: automake-1.16.2-8.el9.noarch 2024-05-03T14:33:41.826 INFO:teuthology.orchestra.run.smithi149.stdout: bison-3.7.4-5.el9.x86_64 2024-05-03T14:33:41.826 INFO:teuthology.orchestra.run.smithi149.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-03T14:33:41.826 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:41.826 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-base-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:41.826 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:41.826 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:41.826 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-grafana-dashboards-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T14:33:41.826 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-immutable-object-cache-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:41.827 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mds-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:41.827 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:41.827 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T14:33:41.827 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-dashboard-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T14:33:41.827 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T14:33:41.827 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-modules-core-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T14:33:41.827 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-rook-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T14:33:41.827 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mon-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:41.827 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-osd-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:41.827 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-prometheus-alerts-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T14:33:41.827 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-radosgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:41.827 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-selinux-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:41.827 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-test-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:41.828 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-volume-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T14:33:41.828 INFO:teuthology.orchestra.run.smithi149.stdout: cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T14:33:41.828 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:41.828 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-shell-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:41.828 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-top-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T14:33:41.828 INFO:teuthology.orchestra.run.smithi149.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-03T14:33:41.828 INFO:teuthology.orchestra.run.smithi149.stdout: flex-2.6.4-9.el9.x86_64 2024-05-03T14:33:41.828 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-03T14:33:41.828 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-03T14:33:41.828 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-03T14:33:41.828 INFO:teuthology.orchestra.run.smithi149.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-03T14:33:41.828 INFO:teuthology.orchestra.run.smithi149.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-03T14:33:41.828 INFO:teuthology.orchestra.run.smithi149.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-03T14:33:41.828 INFO:teuthology.orchestra.run.smithi149.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-03T14:33:41.828 INFO:teuthology.orchestra.run.smithi149.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-03T14:33:41.829 INFO:teuthology.orchestra.run.smithi149.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-03T14:33:41.829 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs-devel-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:41.829 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs2-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:41.829 INFO:teuthology.orchestra.run.smithi149.stdout: libcephsqlite-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:41.829 INFO:teuthology.orchestra.run.smithi149.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-03T14:33:41.829 INFO:teuthology.orchestra.run.smithi149.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-03T14:33:41.829 INFO:teuthology.orchestra.run.smithi149.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-03T14:33:41.829 INFO:teuthology.orchestra.run.smithi149.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-03T14:33:41.829 INFO:teuthology.orchestra.run.smithi149.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-03T14:33:41.829 INFO:teuthology.orchestra.run.smithi149.stdout: librados-devel-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:41.829 INFO:teuthology.orchestra.run.smithi149.stdout: libradosstriper1-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:41.829 INFO:teuthology.orchestra.run.smithi149.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-03T14:33:41.829 INFO:teuthology.orchestra.run.smithi149.stdout: librgw2-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:41.829 INFO:teuthology.orchestra.run.smithi149.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-03T14:33:41.830 INFO:teuthology.orchestra.run.smithi149.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-03T14:33:41.830 INFO:teuthology.orchestra.run.smithi149.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-03T14:33:41.830 INFO:teuthology.orchestra.run.smithi149.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-03T14:33:41.830 INFO:teuthology.orchestra.run.smithi149.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-03T14:33:41.830 INFO:teuthology.orchestra.run.smithi149.stdout: lua-5.4.4-4.el9.x86_64 2024-05-03T14:33:41.830 INFO:teuthology.orchestra.run.smithi149.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-03T14:33:41.830 INFO:teuthology.orchestra.run.smithi149.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-03T14:33:41.830 INFO:teuthology.orchestra.run.smithi149.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-03T14:33:41.830 INFO:teuthology.orchestra.run.smithi149.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-03T14:33:41.830 INFO:teuthology.orchestra.run.smithi149.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-03T14:33:41.830 INFO:teuthology.orchestra.run.smithi149.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-03T14:33:41.830 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-13.14-1.el9.x86_64 2024-05-03T14:33:41.830 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-03T14:33:41.831 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-03T14:33:41.831 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-03T14:33:41.831 INFO:teuthology.orchestra.run.smithi149.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-03T14:33:41.831 INFO:teuthology.orchestra.run.smithi149.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-03T14:33:41.831 INFO:teuthology.orchestra.run.smithi149.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-03T14:33:41.831 INFO:teuthology.orchestra.run.smithi149.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-03T14:33:41.831 INFO:teuthology.orchestra.run.smithi149.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-03T14:33:41.831 INFO:teuthology.orchestra.run.smithi149.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-03T14:33:41.831 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-03T14:33:41.831 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-argparse-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:41.831 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:41.831 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cephfs-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:41.831 INFO:teuthology.orchestra.run.smithi149.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-03T14:33:41.831 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-03T14:33:41.832 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-03T14:33:41.832 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-03T14:33:41.832 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-03T14:33:41.832 INFO:teuthology.orchestra.run.smithi149.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-03T14:33:41.832 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-03T14:33:41.832 INFO:teuthology.orchestra.run.smithi149.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-03T14:33:41.832 INFO:teuthology.orchestra.run.smithi149.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-03T14:33:41.832 INFO:teuthology.orchestra.run.smithi149.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-03T14:33:41.832 INFO:teuthology.orchestra.run.smithi149.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-03T14:33:41.832 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-03T14:33:41.832 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-03T14:33:41.832 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-03T14:33:41.832 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-03T14:33:41.832 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-03T14:33:41.833 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-03T14:33:41.833 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-03T14:33:41.833 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-03T14:33:41.833 INFO:teuthology.orchestra.run.smithi149.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-03T14:33:41.833 INFO:teuthology.orchestra.run.smithi149.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-03T14:33:41.833 INFO:teuthology.orchestra.run.smithi149.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-03T14:33:41.833 INFO:teuthology.orchestra.run.smithi149.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-03T14:33:41.833 INFO:teuthology.orchestra.run.smithi149.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-03T14:33:41.833 INFO:teuthology.orchestra.run.smithi149.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-03T14:33:41.833 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-03T14:33:41.833 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-03T14:33:41.833 INFO:teuthology.orchestra.run.smithi149.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-03T14:33:41.833 INFO:teuthology.orchestra.run.smithi149.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-03T14:33:41.833 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-03T14:33:41.834 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ply-3.11-14.el9.noarch 2024-05-03T14:33:41.834 INFO:teuthology.orchestra.run.smithi149.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-03T14:33:41.834 INFO:teuthology.orchestra.run.smithi149.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-03T14:33:41.834 INFO:teuthology.orchestra.run.smithi149.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-03T14:33:41.834 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-03T14:33:41.834 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-03T14:33:41.834 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-03T14:33:41.835 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-03T14:33:41.835 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-03T14:33:41.835 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-03T14:33:41.835 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rados-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:41.835 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:41.835 INFO:teuthology.orchestra.run.smithi149.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-03T14:33:41.835 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-03T14:33:41.835 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-03T14:33:41.835 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:41.835 INFO:teuthology.orchestra.run.smithi149.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-03T14:33:41.835 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-03T14:33:41.835 INFO:teuthology.orchestra.run.smithi149.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-03T14:33:41.835 INFO:teuthology.orchestra.run.smithi149.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-03T14:33:41.835 INFO:teuthology.orchestra.run.smithi149.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-03T14:33:41.836 INFO:teuthology.orchestra.run.smithi149.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-03T14:33:41.836 INFO:teuthology.orchestra.run.smithi149.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-03T14:33:41.836 INFO:teuthology.orchestra.run.smithi149.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-03T14:33:41.836 INFO:teuthology.orchestra.run.smithi149.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-03T14:33:41.836 INFO:teuthology.orchestra.run.smithi149.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-03T14:33:41.836 INFO:teuthology.orchestra.run.smithi149.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-03T14:33:41.836 INFO:teuthology.orchestra.run.smithi149.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-03T14:33:41.836 INFO:teuthology.orchestra.run.smithi149.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-03T14:33:41.836 INFO:teuthology.orchestra.run.smithi149.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-03T14:33:41.836 INFO:teuthology.orchestra.run.smithi149.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-03T14:33:41.836 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:41.836 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:41.836 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-nbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:41.837 INFO:teuthology.orchestra.run.smithi149.stdout: re2-1:20211101-3.el9.x86_64 2024-05-03T14:33:41.837 INFO:teuthology.orchestra.run.smithi149.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-03T14:33:41.837 INFO:teuthology.orchestra.run.smithi149.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-03T14:33:41.837 INFO:teuthology.orchestra.run.smithi149.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-03T14:33:41.837 INFO:teuthology.orchestra.run.smithi149.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-03T14:33:41.837 INFO:teuthology.orchestra.run.smithi149.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-03T14:33:41.837 INFO:teuthology.orchestra.run.smithi149.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-03T14:33:41.837 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-03T14:33:41.837 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-05-03T14:33:42.242 DEBUG:teuthology.parallel:result is None 2024-05-03T14:33:44.979 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-2:19.0.0-3450.g9a047fe7.el9.x86_64 1/156 2024-05-03T14:33:44.979 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-base-2:19.0.0-3450.g9a047fe7.el9.x86_64 2/156 2024-05-03T14:33:44.979 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 3/156 2024-05-03T14:33:44.979 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 4/156 2024-05-03T14:33:44.979 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3450.g9a047 5/156 2024-05-03T14:33:44.979 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mds-2:19.0.0-3450.g9a047fe7.el9.x86_64 6/156 2024-05-03T14:33:44.979 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-2:19.0.0-3450.g9a047fe7.el9.x86_64 7/156 2024-05-03T14:33:44.979 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mon-2:19.0.0-3450.g9a047fe7.el9.x86_64 8/156 2024-05-03T14:33:44.980 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-osd-2:19.0.0-3450.g9a047fe7.el9.x86_64 9/156 2024-05-03T14:33:44.980 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-radosgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 10/156 2024-05-03T14:33:44.980 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-selinux-2:19.0.0-3450.g9a047fe7.el9.x86_64 11/156 2024-05-03T14:33:44.980 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-test-2:19.0.0-3450.g9a047fe7.el9.x86_64 12/156 2024-05-03T14:33:44.980 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : cephfs-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 13/156 2024-05-03T14:33:44.980 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : cephfs-shell-2:19.0.0-3450.g9a047fe7.el9.x86_64 14/156 2024-05-03T14:33:44.980 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libcephfs-devel-2:19.0.0-3450.g9a047fe7.el9.x86_ 15/156 2024-05-03T14:33:44.980 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libcephfs2-2:19.0.0-3450.g9a047fe7.el9.x86_64 16/156 2024-05-03T14:33:44.980 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libcephsqlite-2:19.0.0-3450.g9a047fe7.el9.x86_64 17/156 2024-05-03T14:33:44.980 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librados-devel-2:19.0.0-3450.g9a047fe7.el9.x86_6 18/156 2024-05-03T14:33:44.980 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libradosstriper1-2:19.0.0-3450.g9a047fe7.el9.x86 19/156 2024-05-03T14:33:44.980 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librgw2-2:19.0.0-3450.g9a047fe7.el9.x86_64 20/156 2024-05-03T14:33:44.980 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3450.g9a047fe7.el 21/156 2024-05-03T14:33:44.981 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-ceph-common-2:19.0.0-3450.g9a047fe7.el9. 22/156 2024-05-03T14:33:44.981 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cephfs-2:19.0.0-3450.g9a047fe7.el9.x86_6 23/156 2024-05-03T14:33:44.981 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-rados-2:19.0.0-3450.g9a047fe7.el9.x86_64 24/156 2024-05-03T14:33:44.981 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-rbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 25/156 2024-05-03T14:33:44.981 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-rgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 26/156 2024-05-03T14:33:44.981 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : rbd-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 27/156 2024-05-03T14:33:44.981 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : rbd-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 28/156 2024-05-03T14:33:44.981 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : rbd-nbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 29/156 2024-05-03T14:33:44.981 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3450.g9a047fe7. 30/156 2024-05-03T14:33:44.981 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3450.g9a047fe7.el9.noa 31/156 2024-05-03T14:33:44.981 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3450.g9a047fe7.el9.n 32/156 2024-05-03T14:33:44.992 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3450.g9a0 33/156 2024-05-03T14:33:44.992 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3450.g9a047fe7.el 34/156 2024-05-03T14:33:44.992 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3450.g9a047fe7.el9.noarch 35/156 2024-05-03T14:33:44.992 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3450.g9a047fe7.e 36/156 2024-05-03T14:33:44.993 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-volume-2:19.0.0-3450.g9a047fe7.el9.noarch 37/156 2024-05-03T14:33:44.993 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 38/156 2024-05-03T14:33:44.993 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : cephfs-top-2:19.0.0-3450.g9a047fe7.el9.noarch 39/156 2024-05-03T14:33:44.993 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-05-03T14:33:44.993 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-05-03T14:33:44.993 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-05-03T14:33:44.993 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-05-03T14:33:44.993 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-05-03T14:33:44.993 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-05-03T14:33:44.993 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-05-03T14:33:44.993 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-05-03T14:33:44.993 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-05-03T14:33:44.993 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-05-03T14:33:44.993 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-05-03T14:33:44.994 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-05-03T14:33:44.994 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-05-03T14:33:44.994 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-05-03T14:33:44.994 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-05-03T14:33:44.994 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-05-03T14:33:44.994 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-05-03T14:33:44.994 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-05-03T14:33:44.994 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-05-03T14:33:44.994 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-05-03T14:33:44.994 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-05-03T14:33:44.994 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-05-03T14:33:44.994 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-05-03T14:33:44.994 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-05-03T14:33:44.994 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-05-03T14:33:44.995 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-05-03T14:33:44.995 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-05-03T14:33:44.995 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-05-03T14:33:44.995 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-05-03T14:33:44.995 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-05-03T14:33:44.995 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-05-03T14:33:44.995 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-05-03T14:33:44.995 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-05-03T14:33:44.995 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-05-03T14:33:44.995 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-05-03T14:33:44.995 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-05-03T14:33:44.995 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-05-03T14:33:44.995 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-05-03T14:33:44.996 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-05-03T14:33:44.996 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-05-03T14:33:44.996 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-05-03T14:33:44.996 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-05-03T14:33:44.996 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-05-03T14:33:44.996 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-05-03T14:33:44.996 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-05-03T14:33:44.996 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-05-03T14:33:44.996 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-05-03T14:33:44.996 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-05-03T14:33:44.996 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-05-03T14:33:44.996 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-05-03T14:33:44.996 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-05-03T14:33:44.996 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-05-03T14:33:44.996 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-05-03T14:33:44.997 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-05-03T14:33:44.997 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-05-03T14:33:44.997 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-05-03T14:33:44.997 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-05-03T14:33:44.997 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-05-03T14:33:44.997 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-05-03T14:33:44.997 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-05-03T14:33:44.997 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-05-03T14:33:44.997 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-05-03T14:33:44.997 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-05-03T14:33:44.997 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-05-03T14:33:44.997 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-05-03T14:33:44.997 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-05-03T14:33:44.997 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-05-03T14:33:44.998 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-05-03T14:33:44.998 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-05-03T14:33:44.998 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-05-03T14:33:44.998 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-05-03T14:33:44.998 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-05-03T14:33:44.998 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-05-03T14:33:44.998 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-05-03T14:33:44.998 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-05-03T14:33:44.998 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-05-03T14:33:44.998 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-05-03T14:33:44.998 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-05-03T14:33:44.998 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-05-03T14:33:44.998 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-05-03T14:33:44.998 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-05-03T14:33:44.999 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-05-03T14:33:44.999 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-05-03T14:33:44.999 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-05-03T14:33:44.999 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-05-03T14:33:44.999 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-05-03T14:33:44.999 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-05-03T14:33:44.999 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-05-03T14:33:44.999 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-05-03T14:33:44.999 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-05-03T14:33:44.999 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-05-03T14:33:44.999 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-05-03T14:33:44.999 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-05-03T14:33:44.999 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-05-03T14:33:44.999 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-05-03T14:33:44.999 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-05-03T14:33:45.000 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-05-03T14:33:45.000 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-05-03T14:33:45.000 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-05-03T14:33:45.000 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-05-03T14:33:45.000 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-05-03T14:33:45.000 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-05-03T14:33:45.000 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-05-03T14:33:45.000 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-05-03T14:33:45.000 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-05-03T14:33:45.000 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-05-03T14:33:45.000 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-05-03T14:33:45.000 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-05-03T14:33:45.000 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-05-03T14:33:45.000 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librados2-2:19.0.0-3450.g9a047fe7.el9.x86_64 149/156 2024-05-03T14:33:45.000 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-05-03T14:33:45.001 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librbd1-2:19.0.0-3450.g9a047fe7.el9.x86_64 151/156 2024-05-03T14:33:45.001 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-05-03T14:33:45.001 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-05-03T14:33:45.001 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-05-03T14:33:45.001 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-05-03T14:33:47.544 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-03T14:33:47.545 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:33:47.545 INFO:teuthology.orchestra.run.smithi060.stdout:Upgraded: 2024-05-03T14:33:47.545 INFO:teuthology.orchestra.run.smithi060.stdout: librados2-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:47.545 INFO:teuthology.orchestra.run.smithi060.stdout: librbd1-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:47.545 INFO:teuthology.orchestra.run.smithi060.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-03T14:33:47.545 INFO:teuthology.orchestra.run.smithi060.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-03T14:33:47.545 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2024-05-03T14:33:47.545 INFO:teuthology.orchestra.run.smithi060.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-03T14:33:47.545 INFO:teuthology.orchestra.run.smithi060.stdout: automake-1.16.2-8.el9.noarch 2024-05-03T14:33:47.545 INFO:teuthology.orchestra.run.smithi060.stdout: bison-3.7.4-5.el9.x86_64 2024-05-03T14:33:47.546 INFO:teuthology.orchestra.run.smithi060.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-03T14:33:47.546 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:47.546 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-base-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:47.546 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:47.546 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:47.546 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-grafana-dashboards-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T14:33:47.546 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-immutable-object-cache-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:47.546 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mds-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:47.546 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:47.546 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T14:33:47.546 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-dashboard-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T14:33:47.546 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T14:33:47.546 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-modules-core-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T14:33:47.547 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-rook-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T14:33:47.547 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mon-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:47.547 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-osd-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:47.547 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-prometheus-alerts-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T14:33:47.547 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-radosgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:47.547 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-selinux-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:47.547 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-test-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:47.547 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-volume-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T14:33:47.547 INFO:teuthology.orchestra.run.smithi060.stdout: cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T14:33:47.547 INFO:teuthology.orchestra.run.smithi060.stdout: cephfs-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:47.547 INFO:teuthology.orchestra.run.smithi060.stdout: cephfs-shell-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:47.547 INFO:teuthology.orchestra.run.smithi060.stdout: cephfs-top-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T14:33:47.547 INFO:teuthology.orchestra.run.smithi060.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-03T14:33:47.547 INFO:teuthology.orchestra.run.smithi060.stdout: flex-2.6.4-9.el9.x86_64 2024-05-03T14:33:47.548 INFO:teuthology.orchestra.run.smithi060.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-03T14:33:47.548 INFO:teuthology.orchestra.run.smithi060.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-03T14:33:47.548 INFO:teuthology.orchestra.run.smithi060.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-03T14:33:47.548 INFO:teuthology.orchestra.run.smithi060.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-03T14:33:47.548 INFO:teuthology.orchestra.run.smithi060.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-03T14:33:47.548 INFO:teuthology.orchestra.run.smithi060.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-03T14:33:47.548 INFO:teuthology.orchestra.run.smithi060.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-03T14:33:47.548 INFO:teuthology.orchestra.run.smithi060.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-03T14:33:47.548 INFO:teuthology.orchestra.run.smithi060.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-03T14:33:47.548 INFO:teuthology.orchestra.run.smithi060.stdout: libcephfs-devel-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:47.548 INFO:teuthology.orchestra.run.smithi060.stdout: libcephfs2-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:47.548 INFO:teuthology.orchestra.run.smithi060.stdout: libcephsqlite-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:47.548 INFO:teuthology.orchestra.run.smithi060.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-03T14:33:47.548 INFO:teuthology.orchestra.run.smithi060.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-03T14:33:47.549 INFO:teuthology.orchestra.run.smithi060.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-03T14:33:47.549 INFO:teuthology.orchestra.run.smithi060.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-03T14:33:47.549 INFO:teuthology.orchestra.run.smithi060.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-03T14:33:47.549 INFO:teuthology.orchestra.run.smithi060.stdout: librados-devel-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:47.549 INFO:teuthology.orchestra.run.smithi060.stdout: libradosstriper1-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:47.549 INFO:teuthology.orchestra.run.smithi060.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-03T14:33:47.549 INFO:teuthology.orchestra.run.smithi060.stdout: librgw2-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:47.549 INFO:teuthology.orchestra.run.smithi060.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-03T14:33:47.549 INFO:teuthology.orchestra.run.smithi060.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-03T14:33:47.549 INFO:teuthology.orchestra.run.smithi060.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-03T14:33:47.549 INFO:teuthology.orchestra.run.smithi060.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-03T14:33:47.549 INFO:teuthology.orchestra.run.smithi060.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-03T14:33:47.549 INFO:teuthology.orchestra.run.smithi060.stdout: lua-5.4.4-4.el9.x86_64 2024-05-03T14:33:47.550 INFO:teuthology.orchestra.run.smithi060.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-03T14:33:47.550 INFO:teuthology.orchestra.run.smithi060.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-03T14:33:47.550 INFO:teuthology.orchestra.run.smithi060.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-03T14:33:47.550 INFO:teuthology.orchestra.run.smithi060.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-03T14:33:47.550 INFO:teuthology.orchestra.run.smithi060.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-03T14:33:47.550 INFO:teuthology.orchestra.run.smithi060.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-03T14:33:47.550 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-13.14-1.el9.x86_64 2024-05-03T14:33:47.550 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-03T14:33:47.550 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-03T14:33:47.550 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-03T14:33:47.550 INFO:teuthology.orchestra.run.smithi060.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-03T14:33:47.550 INFO:teuthology.orchestra.run.smithi060.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-03T14:33:47.550 INFO:teuthology.orchestra.run.smithi060.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-03T14:33:47.550 INFO:teuthology.orchestra.run.smithi060.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-03T14:33:47.551 INFO:teuthology.orchestra.run.smithi060.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-03T14:33:47.551 INFO:teuthology.orchestra.run.smithi060.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-03T14:33:47.551 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-03T14:33:47.551 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ceph-argparse-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:47.551 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:47.551 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cephfs-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:47.551 INFO:teuthology.orchestra.run.smithi060.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-03T14:33:47.551 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-03T14:33:47.551 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-03T14:33:47.551 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-03T14:33:47.551 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-03T14:33:47.551 INFO:teuthology.orchestra.run.smithi060.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-03T14:33:47.551 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-03T14:33:47.551 INFO:teuthology.orchestra.run.smithi060.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-03T14:33:47.551 INFO:teuthology.orchestra.run.smithi060.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-03T14:33:47.552 INFO:teuthology.orchestra.run.smithi060.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-03T14:33:47.552 INFO:teuthology.orchestra.run.smithi060.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-03T14:33:47.552 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-03T14:33:47.552 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-03T14:33:47.552 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-03T14:33:47.552 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-03T14:33:47.552 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-03T14:33:47.552 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-03T14:33:47.552 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-03T14:33:47.552 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-03T14:33:47.552 INFO:teuthology.orchestra.run.smithi060.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-03T14:33:47.552 INFO:teuthology.orchestra.run.smithi060.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-03T14:33:47.552 INFO:teuthology.orchestra.run.smithi060.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-03T14:33:47.552 INFO:teuthology.orchestra.run.smithi060.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-03T14:33:47.552 INFO:teuthology.orchestra.run.smithi060.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-03T14:33:47.553 INFO:teuthology.orchestra.run.smithi060.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-03T14:33:47.553 INFO:teuthology.orchestra.run.smithi060.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-03T14:33:47.553 INFO:teuthology.orchestra.run.smithi060.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-03T14:33:47.553 INFO:teuthology.orchestra.run.smithi060.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-03T14:33:47.553 INFO:teuthology.orchestra.run.smithi060.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-03T14:33:47.553 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-03T14:33:47.553 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ply-3.11-14.el9.noarch 2024-05-03T14:33:47.553 INFO:teuthology.orchestra.run.smithi060.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-03T14:33:47.553 INFO:teuthology.orchestra.run.smithi060.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-03T14:33:47.553 INFO:teuthology.orchestra.run.smithi060.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-03T14:33:47.553 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-03T14:33:47.553 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-03T14:33:47.553 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-03T14:33:47.554 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-03T14:33:47.554 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-03T14:33:47.554 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-03T14:33:47.554 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rados-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:47.554 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:47.554 INFO:teuthology.orchestra.run.smithi060.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-03T14:33:47.554 INFO:teuthology.orchestra.run.smithi060.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-03T14:33:47.554 INFO:teuthology.orchestra.run.smithi060.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-03T14:33:47.554 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:47.554 INFO:teuthology.orchestra.run.smithi060.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-03T14:33:47.554 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-03T14:33:47.554 INFO:teuthology.orchestra.run.smithi060.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-03T14:33:47.554 INFO:teuthology.orchestra.run.smithi060.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-03T14:33:47.554 INFO:teuthology.orchestra.run.smithi060.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-03T14:33:47.554 INFO:teuthology.orchestra.run.smithi060.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-03T14:33:47.555 INFO:teuthology.orchestra.run.smithi060.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-03T14:33:47.555 INFO:teuthology.orchestra.run.smithi060.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-03T14:33:47.555 INFO:teuthology.orchestra.run.smithi060.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-03T14:33:47.555 INFO:teuthology.orchestra.run.smithi060.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-03T14:33:47.555 INFO:teuthology.orchestra.run.smithi060.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-03T14:33:47.555 INFO:teuthology.orchestra.run.smithi060.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-03T14:33:47.555 INFO:teuthology.orchestra.run.smithi060.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-03T14:33:47.555 INFO:teuthology.orchestra.run.smithi060.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-03T14:33:47.555 INFO:teuthology.orchestra.run.smithi060.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-03T14:33:47.555 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:47.555 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:47.555 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-nbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T14:33:47.555 INFO:teuthology.orchestra.run.smithi060.stdout: re2-1:20211101-3.el9.x86_64 2024-05-03T14:33:47.555 INFO:teuthology.orchestra.run.smithi060.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-03T14:33:47.555 INFO:teuthology.orchestra.run.smithi060.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-03T14:33:47.556 INFO:teuthology.orchestra.run.smithi060.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-03T14:33:47.556 INFO:teuthology.orchestra.run.smithi060.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-03T14:33:47.556 INFO:teuthology.orchestra.run.smithi060.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-03T14:33:47.556 INFO:teuthology.orchestra.run.smithi060.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-03T14:33:47.556 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:33:47.556 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-05-03T14:33:48.089 DEBUG:teuthology.parallel:result is None 2024-05-03T14:33:48.089 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9a047fe7599a549042d06a3fb5b6813a203eabea 2024-05-03T14:33:48.345 DEBUG:teuthology.orchestra.run.smithi060:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-03T14:33:48.406 INFO:teuthology.orchestra.run.smithi060.stdout:19.0.0-3450.g9a047fe7.el9 2024-05-03T14:33:48.406 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3450.g9a047fe7.el9 2024-05-03T14:33:48.406 INFO:teuthology.task.install:The correct ceph version 19.0.0-3450.g9a047fe7 is installed. 2024-05-03T14:33:48.408 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9a047fe7599a549042d06a3fb5b6813a203eabea 2024-05-03T14:33:48.550 DEBUG:teuthology.orchestra.run.smithi090:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-03T14:33:48.576 INFO:teuthology.orchestra.run.smithi090.stdout:19.0.0-3450.g9a047fe7.el9 2024-05-03T14:33:48.576 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3450.g9a047fe7.el9 2024-05-03T14:33:48.576 INFO:teuthology.task.install:The correct ceph version 19.0.0-3450.g9a047fe7 is installed. 2024-05-03T14:33:48.578 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9a047fe7599a549042d06a3fb5b6813a203eabea 2024-05-03T14:33:48.717 DEBUG:teuthology.orchestra.run.smithi149:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-03T14:33:48.742 INFO:teuthology.orchestra.run.smithi149.stdout:19.0.0-3450.g9a047fe7.el9 2024-05-03T14:33:48.743 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3450.g9a047fe7.el9 2024-05-03T14:33:48.743 INFO:teuthology.task.install:The correct ceph version 19.0.0-3450.g9a047fe7 is installed. 2024-05-03T14:33:48.745 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-03T14:33:48.745 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-05-03T14:33:48.745 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-03T14:33:48.826 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-05-03T14:33:48.827 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-03T14:33:48.858 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-03T14:33:48.858 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-03T14:33:48.889 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-03T14:33:48.889 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-05-03T14:33:48.889 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/usr/bin/daemon-helper 2024-05-03T14:33:48.973 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-03T14:33:49.086 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-05-03T14:33:49.086 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/usr/bin/daemon-helper 2024-05-03T14:33:49.116 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-03T14:33:49.181 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-03T14:33:49.182 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/usr/bin/daemon-helper 2024-05-03T14:33:49.209 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-03T14:33:49.274 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-03T14:33:49.274 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-05-03T14:33:49.274 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-03T14:33:49.338 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-03T14:33:49.444 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-05-03T14:33:49.444 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-03T14:33:49.469 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-03T14:33:49.532 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-03T14:33:49.532 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-03T14:33:49.557 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-03T14:33:49.624 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-03T14:33:49.624 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-05-03T14:33:49.624 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/usr/bin/stdin-killer 2024-05-03T14:33:49.690 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-03T14:33:49.793 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-05-03T14:33:49.794 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/usr/bin/stdin-killer 2024-05-03T14:33:49.817 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-03T14:33:49.880 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-03T14:33:49.880 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/usr/bin/stdin-killer 2024-05-03T14:33:49.904 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-03T14:33:49.968 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-03T14:33:50.068 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'fuse default permissions': False, 'fuse set user groups': True, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '9a047fe7599a549042d06a3fb5b6813a203eabea', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-05-03T14:33:50.068 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea 2024-05-03T14:33:50.069 INFO:tasks.cephadm:Cluster fsid is 285d64b2-095a-11ef-bc97-c7b262605968 2024-05-03T14:33:50.069 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-03T14:33:50.069 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.60', 'mon.b': '172.21.15.90', 'mon.c': '172.21.15.149'} 2024-05-03T14:33:50.069 INFO:tasks.cephadm:First mon is mon.a on smithi060 2024-05-03T14:33:50.069 INFO:tasks.cephadm:First mgr is x 2024-05-03T14:33:50.069 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-03T14:33:50.069 DEBUG:teuthology.orchestra.run.smithi060:> sudo hostname $(hostname -s) 2024-05-03T14:33:50.134 DEBUG:teuthology.orchestra.run.smithi090:> sudo hostname $(hostname -s) 2024-05-03T14:33:50.157 DEBUG:teuthology.orchestra.run.smithi149:> sudo hostname $(hostname -s) 2024-05-03T14:33:50.181 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-05-03T14:33:50.181 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9a047fe7599a549042d06a3fb5b6813a203eabea 2024-05-03T14:33:50.335 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '9a047fe7599a549042d06a3fb5b6813a203eabea', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/79034/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3450-g9a047fe7', 'node_name': '172.21.5.34+adami04', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3450.g9a047fe7'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-02 00:36:37.664792', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-testing-20240501.193033', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-05-03T14:33:50.476 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-rishabh-testing-20240501.193033, sha1 9a047fe7599a549042d06a3fb5b6813a203eabea from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=9a047fe7599a549042d06a3fb5b6813a203eabea 2024-05-03T14:33:50.478 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/x86_64/flavors/default/cephadm 2024-05-03T14:33:50.478 DEBUG:teuthology.orchestra.run.smithi060:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-03T14:33:50.939 INFO:teuthology.orchestra.run.smithi060.stdout:-rw-r--r--. 1 ubuntu ubuntu 782467 May 3 14:33 /home/ubuntu/cephtest/cephadm 2024-05-03T14:33:50.939 DEBUG:teuthology.orchestra.run.smithi090:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-03T14:33:51.378 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-r--r--. 1 ubuntu ubuntu 782467 May 3 14:33 /home/ubuntu/cephtest/cephadm 2024-05-03T14:33:51.379 DEBUG:teuthology.orchestra.run.smithi149:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-03T14:33:51.804 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r--. 1 ubuntu ubuntu 782467 May 3 14:33 /home/ubuntu/cephtest/cephadm 2024-05-03T14:33:51.805 DEBUG:teuthology.orchestra.run.smithi060:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-03T14:33:51.852 DEBUG:teuthology.orchestra.run.smithi090:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-03T14:33:51.871 DEBUG:teuthology.orchestra.run.smithi149:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-03T14:33:51.906 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea on all hosts... 2024-05-03T14:33:51.906 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea pull 2024-05-03T14:33:51.910 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea pull 2024-05-03T14:33:51.913 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea pull 2024-05-03T14:33:53.656 INFO:teuthology.orchestra.run.smithi149.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea... 2024-05-03T14:33:54.060 INFO:teuthology.orchestra.run.smithi060.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea... 2024-05-03T14:33:54.357 INFO:teuthology.orchestra.run.smithi090.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea... 2024-05-03T14:34:26.945 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-03T14:34:26.946 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_version": "ceph version 19.0.0-3450-g9a047fe7 (9a047fe7599a549042d06a3fb5b6813a203eabea) squid (dev)", 2024-05-03T14:34:26.946 INFO:teuthology.orchestra.run.smithi149.stdout: "image_id": "ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012", 2024-05-03T14:34:26.946 INFO:teuthology.orchestra.run.smithi149.stdout: "repo_digests": [ 2024-05-03T14:34:26.946 INFO:teuthology.orchestra.run.smithi149.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd" 2024-05-03T14:34:26.946 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-03T14:34:26.946 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-03T14:34:30.975 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-05-03T14:34:30.975 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_version": "ceph version 19.0.0-3450-g9a047fe7 (9a047fe7599a549042d06a3fb5b6813a203eabea) squid (dev)", 2024-05-03T14:34:30.975 INFO:teuthology.orchestra.run.smithi090.stdout: "image_id": "ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012", 2024-05-03T14:34:30.975 INFO:teuthology.orchestra.run.smithi090.stdout: "repo_digests": [ 2024-05-03T14:34:30.975 INFO:teuthology.orchestra.run.smithi090.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd" 2024-05-03T14:34:30.976 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-05-03T14:34:30.976 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-05-03T14:34:31.978 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-05-03T14:34:31.978 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_version": "ceph version 19.0.0-3450-g9a047fe7 (9a047fe7599a549042d06a3fb5b6813a203eabea) squid (dev)", 2024-05-03T14:34:31.978 INFO:teuthology.orchestra.run.smithi060.stdout: "image_id": "ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012", 2024-05-03T14:34:31.978 INFO:teuthology.orchestra.run.smithi060.stdout: "repo_digests": [ 2024-05-03T14:34:31.978 INFO:teuthology.orchestra.run.smithi060.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd" 2024-05-03T14:34:31.978 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-05-03T14:34:31.978 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-05-03T14:34:32.021 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /etc/ceph 2024-05-03T14:34:32.096 DEBUG:teuthology.orchestra.run.smithi090:> sudo mkdir -p /etc/ceph 2024-05-03T14:34:32.127 DEBUG:teuthology.orchestra.run.smithi149:> sudo mkdir -p /etc/ceph 2024-05-03T14:34:32.157 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 777 /etc/ceph 2024-05-03T14:34:32.235 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 777 /etc/ceph 2024-05-03T14:34:32.263 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod 777 /etc/ceph 2024-05-03T14:34:32.290 INFO:tasks.cephadm:Writing seed config... 2024-05-03T14:34:32.291 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-03T14:34:32.291 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-03T14:34:32.291 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-03T14:34:32.292 INFO:tasks.cephadm: override: [client] fuse default permissions = False 2024-05-03T14:34:32.292 INFO:tasks.cephadm: override: [client] fuse set user groups = True 2024-05-03T14:34:32.292 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-03T14:34:32.292 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-03T14:34:32.292 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-05-03T14:34:32.292 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-03T14:34:32.292 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-03T14:34:32.292 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-03T14:34:32.292 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-05-03T14:34:32.292 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-05-03T14:34:32.292 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-05-03T14:34:32.292 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-05-03T14:34:32.292 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-03T14:34:32.293 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-03T14:34:32.293 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-03T14:34:32.293 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-03T14:34:32.293 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-05-03T14:34:32.293 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-03T14:34:32.293 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-03T14:34:32.293 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-03T14:34:32.293 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-05-03T14:34:32.293 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-05-03T14:34:32.293 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-03T14:34:32.293 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-03T14:34:32.293 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-03T14:34:32.293 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-03T14:34:32.293 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-03T14:34:32.294 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-05-03T14:34:32.294 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-03T14:34:32.294 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-03T14:34:32.294 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-05-03T14:34:32.294 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-03T14:34:32.294 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-05-03T14:34:32.294 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-05-03T14:34:32.294 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-03T14:34:32.294 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-03T14:34:32.294 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-05-03T14:34:32.294 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-03T14:34:32.294 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-03T14:34:32.294 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-03T14:34:32.294 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-03T14:34:32.294 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-03T14:34:32.294 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-03T14:34:32.295 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-05-03T14:34:32.295 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-05-03T14:34:32.295 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-05-03T14:34:32.295 DEBUG:teuthology.orchestra.run.smithi060:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-03T14:34:32.351 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 285d64b2-095a-11ef-bc97-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 fuse default permissions = False fuse set user groups = True rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-03T14:34:32.351 DEBUG:teuthology.orchestra.run.smithi060:mon.a> sudo journalctl -f -n 0 -u ceph-285d64b2-095a-11ef-bc97-c7b262605968@mon.a.service 2024-05-03T14:34:32.397 DEBUG:teuthology.orchestra.run.smithi060:mgr.x> sudo journalctl -f -n 0 -u ceph-285d64b2-095a-11ef-bc97-c7b262605968@mgr.x.service 2024-05-03T14:34:32.443 INFO:tasks.cephadm:Bootstrapping... 2024-05-03T14:34:32.443 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea -v bootstrap --fsid 285d64b2-095a-11ef-bc97-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.60 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-03T14:34:32.712 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2024-05-03T14:34:32.712 INFO:teuthology.orchestra.run.smithi060.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea', '-v', 'bootstrap', '--fsid', '285d64b2-095a-11ef-bc97-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.60', '--skip-admin-label'] 2024-05-03T14:34:32.713 INFO:teuthology.orchestra.run.smithi060.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-03T14:34:32.713 INFO:teuthology.orchestra.run.smithi060.stdout:Verifying podman|docker is present... 2024-05-03T14:34:32.771 INFO:teuthology.orchestra.run.smithi060.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-03T14:34:32.772 INFO:teuthology.orchestra.run.smithi060.stdout:Verifying lvm2 is present... 2024-05-03T14:34:32.772 INFO:teuthology.orchestra.run.smithi060.stdout:Verifying time synchronization is in place... 2024-05-03T14:34:32.787 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-03T14:34:32.787 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-03T14:34:32.802 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-03T14:34:32.802 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout inactive 2024-05-03T14:34:32.824 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout enabled 2024-05-03T14:34:32.846 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout active 2024-05-03T14:34:32.847 INFO:teuthology.orchestra.run.smithi060.stdout:Unit chronyd.service is enabled and running 2024-05-03T14:34:32.847 INFO:teuthology.orchestra.run.smithi060.stdout:Repeating the final host check... 2024-05-03T14:34:32.908 INFO:teuthology.orchestra.run.smithi060.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-03T14:34:32.908 INFO:teuthology.orchestra.run.smithi060.stdout:podman (/bin/podman) version 4.9.4 is present 2024-05-03T14:34:32.908 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl is present 2024-05-03T14:34:32.908 INFO:teuthology.orchestra.run.smithi060.stdout:lvcreate is present 2024-05-03T14:34:32.923 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-03T14:34:32.923 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-03T14:34:32.939 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-03T14:34:32.939 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout inactive 2024-05-03T14:34:32.958 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout enabled 2024-05-03T14:34:32.974 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout active 2024-05-03T14:34:32.974 INFO:teuthology.orchestra.run.smithi060.stdout:Unit chronyd.service is enabled and running 2024-05-03T14:34:32.974 INFO:teuthology.orchestra.run.smithi060.stdout:Host looks OK 2024-05-03T14:34:32.975 INFO:teuthology.orchestra.run.smithi060.stdout:Cluster fsid: 285d64b2-095a-11ef-bc97-c7b262605968 2024-05-03T14:34:32.975 INFO:teuthology.orchestra.run.smithi060.stdout:Acquiring lock 140225713450960 on /run/cephadm/285d64b2-095a-11ef-bc97-c7b262605968.lock 2024-05-03T14:34:32.975 INFO:teuthology.orchestra.run.smithi060.stdout:Lock 140225713450960 acquired on /run/cephadm/285d64b2-095a-11ef-bc97-c7b262605968.lock 2024-05-03T14:34:32.975 INFO:teuthology.orchestra.run.smithi060.stdout:Verifying IP 172.21.15.60 port 3300 ... 2024-05-03T14:34:32.976 INFO:teuthology.orchestra.run.smithi060.stdout:Verifying IP 172.21.15.60 port 6789 ... 2024-05-03T14:34:32.976 INFO:teuthology.orchestra.run.smithi060.stdout:Base mon IP(s) is [172.21.15.60:3300, 172.21.15.60:6789], mon addrv is [v2:172.21.15.60:3300,v1:172.21.15.60:6789] 2024-05-03T14:34:32.984 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.60 metric 100 2024-05-03T14:34:32.984 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.60 metric 100 2024-05-03T14:34:32.992 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-03T14:34:32.992 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-05-03T14:34:32.992 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-05-03T14:34:33.001 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-03T14:34:33.001 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-03T14:34:33.001 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-03T14:34:33.001 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-05-03T14:34:33.001 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1560/64 scope link noprefixroute 2024-05-03T14:34:33.001 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-03T14:34:33.002 INFO:teuthology.orchestra.run.smithi060.stdout:Mon IP `172.21.15.60` is in CIDR network `172.21.0.0/20` 2024-05-03T14:34:33.002 INFO:teuthology.orchestra.run.smithi060.stdout:Mon IP `172.21.15.60` is in CIDR network `172.21.0.0/20` 2024-05-03T14:34:33.002 INFO:teuthology.orchestra.run.smithi060.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-03T14:34:33.003 INFO:teuthology.orchestra.run.smithi060.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-03T14:34:33.004 INFO:teuthology.orchestra.run.smithi060.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea... 2024-05-03T14:34:33.709 INFO:teuthology.orchestra.run.smithi060.stdout:/bin/podman: stdout ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 2024-05-03T14:34:33.709 INFO:teuthology.orchestra.run.smithi060.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea... 2024-05-03T14:34:33.709 INFO:teuthology.orchestra.run.smithi060.stdout:/bin/podman: stderr Getting image source signatures 2024-05-03T14:34:33.709 INFO:teuthology.orchestra.run.smithi060.stdout:/bin/podman: stderr Copying blob sha256:8173fd91c96404ae0424bb08037c57749051dcbdfaeee9354a65e201dad7b0e2 2024-05-03T14:34:33.709 INFO:teuthology.orchestra.run.smithi060.stdout:/bin/podman: stderr Copying blob sha256:e2f2fd2adca92139da4b4b907fa90a6775d3eb4eddaa977c42ef37c66eea70bd 2024-05-03T14:34:33.709 INFO:teuthology.orchestra.run.smithi060.stdout:/bin/podman: stderr Copying config sha256:ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 2024-05-03T14:34:33.709 INFO:teuthology.orchestra.run.smithi060.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-03T14:34:34.954 INFO:teuthology.orchestra.run.smithi060.stdout:ceph: stdout ceph version 19.0.0-3450-g9a047fe7 (9a047fe7599a549042d06a3fb5b6813a203eabea) squid (dev) 2024-05-03T14:34:34.954 INFO:teuthology.orchestra.run.smithi060.stdout:Ceph version: ceph version 19.0.0-3450-g9a047fe7 (9a047fe7599a549042d06a3fb5b6813a203eabea) squid (dev) 2024-05-03T14:34:34.954 INFO:teuthology.orchestra.run.smithi060.stdout:Extracting ceph user uid/gid from container image... 2024-05-03T14:34:36.159 INFO:teuthology.orchestra.run.smithi060.stdout:stat: stdout 167 167 2024-05-03T14:34:36.159 INFO:teuthology.orchestra.run.smithi060.stdout:Creating initial keys... 2024-05-03T14:34:37.281 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph-authtool: stdout AQD89TRmaSU+JxAA6koxe2n4yoeAZ7jEm1dVqQ== 2024-05-03T14:34:38.492 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph-authtool: stdout AQD99TRmcTmYLRAA8Gu9Y64xxkCx9TGQuDrA/Q== 2024-05-03T14:34:39.595 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph-authtool: stdout AQD+9TRm/Un9OhAAAOlvMKZwGgC8oXAwXVlLdQ== 2024-05-03T14:34:39.597 INFO:teuthology.orchestra.run.smithi060.stdout:Creating initial monmap... 2024-05-03T14:34:40.801 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-03T14:34:40.802 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-05-03T14:34:40.802 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 285d64b2-095a-11ef-bc97-c7b262605968 2024-05-03T14:34:40.802 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-03T14:34:40.802 INFO:teuthology.orchestra.run.smithi060.stdout:monmaptool for a [v2:172.21.15.60:3300,v1:172.21.15.60:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-03T14:34:40.802 INFO:teuthology.orchestra.run.smithi060.stdout:setting min_mon_release = quincy 2024-05-03T14:34:40.802 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/monmaptool: set fsid to 285d64b2-095a-11ef-bc97-c7b262605968 2024-05-03T14:34:40.803 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-03T14:34:40.803 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:34:40.803 INFO:teuthology.orchestra.run.smithi060.stdout:Creating mon... 2024-05-03T14:34:51.133 INFO:teuthology.orchestra.run.smithi060.stdout:create mon.a on 2024-05-03T14:34:51.606 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-05-03T14:34:51.853 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-03T14:34:52.144 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-285d64b2-095a-11ef-bc97-c7b262605968.target → /etc/systemd/system/ceph-285d64b2-095a-11ef-bc97-c7b262605968.target. 2024-05-03T14:34:52.144 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-285d64b2-095a-11ef-bc97-c7b262605968.target → /etc/systemd/system/ceph-285d64b2-095a-11ef-bc97-c7b262605968.target. 2024-05-03T14:34:52.535 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-285d64b2-095a-11ef-bc97-c7b262605968@mon.a 2024-05-03T14:34:52.535 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stderr Failed to reset failed state of unit ceph-285d64b2-095a-11ef-bc97-c7b262605968@mon.a.service: Unit ceph-285d64b2-095a-11ef-bc97-c7b262605968@mon.a.service not loaded. 2024-05-03T14:34:52.833 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-285d64b2-095a-11ef-bc97-c7b262605968.target.wants/ceph-285d64b2-095a-11ef-bc97-c7b262605968@mon.a.service → /etc/systemd/system/ceph-285d64b2-095a-11ef-bc97-c7b262605968@.service. 2024-05-03T14:34:53.558 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:34:53 smithi060 podman[28490]: 2024-05-03 14:34:53.4786112 +0000 UTC m=+0.374383481 container start a712d2d80b4ebed8c63bad19f8a1390688e59c2faa713e1141312160275d3923 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mon-a, org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T14:34:53.558 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:34:53 smithi060 bash[28490]: a712d2d80b4ebed8c63bad19f8a1390688e59c2faa713e1141312160275d3923 2024-05-03T14:34:53.591 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-03T14:34:53.591 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout disabled 2024-05-03T14:34:53.606 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-03T14:34:53.607 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout inactive 2024-05-03T14:34:53.607 INFO:teuthology.orchestra.run.smithi060.stdout:firewalld.service is not enabled 2024-05-03T14:34:53.607 INFO:teuthology.orchestra.run.smithi060.stdout:Not possible to enable service . firewalld.service is not available 2024-05-03T14:34:53.609 INFO:teuthology.orchestra.run.smithi060.stdout:Waiting for mon to start... 2024-05-03T14:34:53.609 INFO:teuthology.orchestra.run.smithi060.stdout:Waiting for mon... 2024-05-03T14:34:53.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:34:53 smithi060 systemd[1]: Started Ceph mon.a for 285d64b2-095a-11ef-bc97-c7b262605968. 2024-05-03T14:34:53.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:34:53 smithi060 ceph-mon[28508]: mkfs 285d64b2-095a-11ef-bc97-c7b262605968 2024-05-03T14:34:53.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:34:53 smithi060 ceph-mon[28508]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-03T14:34:54.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:34:54 smithi060 ceph-mon[28508]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-03T14:34:54.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:34:54 smithi060 ceph-mon[28508]: monmap epoch 1 2024-05-03T14:34:54.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:34:54 smithi060 ceph-mon[28508]: fsid 285d64b2-095a-11ef-bc97-c7b262605968 2024-05-03T14:34:54.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:34:54 smithi060 ceph-mon[28508]: last_changed 2024-05-03T14:34:40.229121+0000 2024-05-03T14:34:54.839 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:34:54 smithi060 ceph-mon[28508]: created 2024-05-03T14:34:40.229121+0000 2024-05-03T14:34:54.839 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:34:54 smithi060 ceph-mon[28508]: min_mon_release 19 (squid) 2024-05-03T14:34:54.839 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:34:54 smithi060 ceph-mon[28508]: election_strategy: 1 2024-05-03T14:34:54.839 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:34:54 smithi060 ceph-mon[28508]: 0: [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] mon.a 2024-05-03T14:34:54.839 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:34:54 smithi060 ceph-mon[28508]: fsmap 2024-05-03T14:34:54.839 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:34:54 smithi060 ceph-mon[28508]: osdmap e1: 0 total, 0 up, 0 in 2024-05-03T14:34:54.839 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:34:54 smithi060 ceph-mon[28508]: mgrmap e1: no daemons active 2024-05-03T14:34:54.839 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:34:54 smithi060 ceph-mon[28508]: from='client.? 172.21.15.60:0/1305954460' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-03T14:34:54.962 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout cluster: 2024-05-03T14:34:54.963 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout id: 285d64b2-095a-11ef-bc97-c7b262605968 2024-05-03T14:34:54.964 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-03T14:34:54.964 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-05-03T14:34:54.964 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout services: 2024-05-03T14:34:54.964 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.716667s) 2024-05-03T14:34:54.964 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-03T14:34:54.964 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-03T14:34:54.967 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-05-03T14:34:54.967 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout data: 2024-05-03T14:34:54.967 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-03T14:34:54.967 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-03T14:34:54.967 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-03T14:34:54.967 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout pgs: 2024-05-03T14:34:54.967 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-05-03T14:34:54.967 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:54.181+0000 7fe20f600700 1 Processor -- start 2024-05-03T14:34:54.967 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:54.181+0000 7fe20f600700 1 -- start start 2024-05-03T14:34:54.968 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:54.181+0000 7fe20f600700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe20810b2d0 0x7fe20810b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:34:54.968 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:54.181+0000 7fe20f600700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe20810bcc0 con 0x7fe20810b2d0 2024-05-03T14:34:54.968 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:54.183+0000 7fe20cc00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe20810b2d0 0x7fe20810b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:34:54.968 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:54.183+0000 7fe20cc00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe20810b2d0 0x7fe20810b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41378/0 (socket says 172.21.15.60:41378) 2024-05-03T14:34:54.968 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:54.183+0000 7fe20cc00700 1 -- 172.21.15.60:0/708671238 learned_addr learned my addr 172.21.15.60:0/708671238 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:34:54.968 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:54.183+0000 7fe20cc00700 1 -- 172.21.15.60:0/708671238 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe20810c480 con 0x7fe20810b2d0 2024-05-03T14:34:54.968 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:54.183+0000 7fe20cc00700 1 --2- 172.21.15.60:0/708671238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe20810b2d0 0x7fe20810b6f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc0098a0 tx=0x7fe1fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9e9bc0e78cf274b server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:34:54.968 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:54.184+0000 7fe207400700 1 -- 172.21.15.60:0/708671238 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1fc004160 con 0x7fe20810b2d0 2024-05-03T14:34:54.968 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:54.184+0000 7fe207400700 1 -- 172.21.15.60:0/708671238 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1fc004300 con 0x7fe20810b2d0 2024-05-03T14:34:54.968 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:54.184+0000 7fe207400700 1 -- 172.21.15.60:0/708671238 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1fc0036a0 con 0x7fe20810b2d0 2024-05-03T14:34:54.968 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:54.184+0000 7fe20f600700 1 -- 172.21.15.60:0/708671238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe20810b2d0 msgr2=0x7fe20810b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:34:54.968 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:54.184+0000 7fe20f600700 1 --2- 172.21.15.60:0/708671238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe20810b2d0 0x7fe20810b6f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc0098a0 tx=0x7fe1fc009bb0 comp rx=0 tx=0).stop 2024-05-03T14:34:54.968 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:54.185+0000 7fe20f600700 1 -- 172.21.15.60:0/708671238 shutdown_connections 2024-05-03T14:34:54.972 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:54.185+0000 7fe20f600700 1 --2- 172.21.15.60:0/708671238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe20810b2d0 0x7fe20810b6f0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:34:54.972 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:54.185+0000 7fe20f600700 1 -- 172.21.15.60:0/708671238 >> 172.21.15.60:0/708671238 conn(0x7fe20807e900 msgr2=0x7fe2081094f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:34:54.972 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:54.185+0000 7fe20f600700 1 -- 172.21.15.60:0/708671238 shutdown_connections 2024-05-03T14:34:54.972 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:54.185+0000 7fe20f600700 1 -- 172.21.15.60:0/708671238 wait complete. 2024-05-03T14:34:54.973 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:54.186+0000 7fe20f600700 1 Processor -- start 2024-05-03T14:34:54.973 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:54.186+0000 7fe20f600700 1 -- start start 2024-05-03T14:34:54.973 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:54.187+0000 7fe20f600700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe20810b2d0 0x7fe20819e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:34:54.973 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:54.187+0000 7fe20f600700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe20810c9b0 con 0x7fe20810b2d0 2024-05-03T14:34:54.973 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:54.187+0000 7fe20cc00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe20810b2d0 0x7fe20819e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:34:54.973 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:54.187+0000 7fe20cc00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe20810b2d0 0x7fe20819e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41384/0 (socket says 172.21.15.60:41384) 2024-05-03T14:34:54.973 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:54.187+0000 7fe20cc00700 1 -- 172.21.15.60:0/1305954460 learned_addr learned my addr 172.21.15.60:0/1305954460 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:34:54.973 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:54.187+0000 7fe20cc00700 1 -- 172.21.15.60:0/1305954460 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe20819ef00 con 0x7fe20810b2d0 2024-05-03T14:34:54.973 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:54.188+0000 7fe20cc00700 1 --2- 172.21.15.60:0/1305954460 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe20810b2d0 0x7fe20819e9c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc000c00 tx=0x7fe1fc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:34:54.973 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:54.188+0000 7fe205600700 1 -- 172.21.15.60:0/1305954460 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1fc003db0 con 0x7fe20810b2d0 2024-05-03T14:34:54.973 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:54.188+0000 7fe205600700 1 -- 172.21.15.60:0/1305954460 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1fc011630 con 0x7fe20810b2d0 2024-05-03T14:34:54.973 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:54.188+0000 7fe205600700 1 -- 172.21.15.60:0/1305954460 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1fc011950 con 0x7fe20810b2d0 2024-05-03T14:34:54.973 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:54.188+0000 7fe20f600700 1 -- 172.21.15.60:0/1305954460 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe20819f190 con 0x7fe20810b2d0 2024-05-03T14:34:54.974 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:54.188+0000 7fe20f600700 1 -- 172.21.15.60:0/1305954460 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2081a1e80 con 0x7fe20810b2d0 2024-05-03T14:34:54.974 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:54.188+0000 7fe205600700 1 -- 172.21.15.60:0/1305954460 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe1fc011af0 con 0x7fe20810b2d0 2024-05-03T14:34:54.974 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:54.189+0000 7fe205600700 1 -- 172.21.15.60:0/1305954460 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe1fc01b790 con 0x7fe20810b2d0 2024-05-03T14:34:54.974 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:54.189+0000 7fe20f600700 1 -- 172.21.15.60:0/1305954460 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe208064b00 con 0x7fe20810b2d0 2024-05-03T14:34:54.974 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:54.193+0000 7fe205600700 1 -- 172.21.15.60:0/1305954460 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fe1fc033080 con 0x7fe20810b2d0 2024-05-03T14:34:54.974 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:54.274+0000 7fe20f600700 1 -- 172.21.15.60:0/1305954460 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fe20802e780 con 0x7fe20810b2d0 2024-05-03T14:34:54.974 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:54.275+0000 7fe205600700 1 -- 172.21.15.60:0/1305954460 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fe1fc011de0 con 0x7fe20810b2d0 2024-05-03T14:34:54.974 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:54.276+0000 7fe20f600700 1 -- 172.21.15.60:0/1305954460 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe20810b2d0 msgr2=0x7fe20819e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:34:54.974 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:54.276+0000 7fe20f600700 1 --2- 172.21.15.60:0/1305954460 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe20810b2d0 0x7fe20819e9c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc000c00 tx=0x7fe1fc004750 comp rx=0 tx=0).stop 2024-05-03T14:34:54.974 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:54.276+0000 7fe20f600700 1 -- 172.21.15.60:0/1305954460 shutdown_connections 2024-05-03T14:34:54.974 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:54.277+0000 7fe20f600700 1 --2- 172.21.15.60:0/1305954460 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe20810b2d0 0x7fe20819e9c0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:34:54.974 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:54.277+0000 7fe20f600700 1 -- 172.21.15.60:0/1305954460 >> 172.21.15.60:0/1305954460 conn(0x7fe20807e900 msgr2=0x7fe208195680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:34:54.974 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:54.277+0000 7fe20f600700 1 -- 172.21.15.60:0/1305954460 shutdown_connections 2024-05-03T14:34:54.974 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:54.277+0000 7fe20f600700 1 -- 172.21.15.60:0/1305954460 wait complete. 2024-05-03T14:34:54.975 INFO:teuthology.orchestra.run.smithi060.stdout:mon is available 2024-05-03T14:34:54.975 INFO:teuthology.orchestra.run.smithi060.stdout:Assimilating anything we can from ceph.conf... 2024-05-03T14:34:55.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:34:55 smithi060 ceph-mon[28508]: from='client.? 172.21.15.60:0/188891433' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-03T14:34:55.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:34:55 smithi060 ceph-mon[28508]: from='client.? 172.21.15.60:0/188891433' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-03T14:34:57.186 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-05-03T14:34:57.186 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout [global] 2024-05-03T14:34:57.186 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout fsid = 285d64b2-095a-11ef-bc97-c7b262605968 2024-05-03T14:34:57.186 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-03T14:34:57.187 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.60:3300,v1:172.21.15.60:6789] 2024-05-03T14:34:57.188 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-03T14:34:57.188 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-03T14:34:57.188 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-03T14:34:57.188 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-03T14:34:57.188 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-05-03T14:34:57.188 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-03T14:34:57.188 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-03T14:34:57.188 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-05-03T14:34:57.188 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout [osd] 2024-05-03T14:34:57.194 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-03T14:34:57.194 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-03T14:34:57.194 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-03T14:34:57.194 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-03T14:34:57.194 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-03T14:34:57.194 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:55.567+0000 7f94c7e00700 1 Processor -- start 2024-05-03T14:34:57.194 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:55.567+0000 7f94c7e00700 1 -- start start 2024-05-03T14:34:57.194 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:55.568+0000 7f94c7e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94c810b110 0x7f94c810b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:34:57.194 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:55.568+0000 7f94c7e00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f94c810bb00 con 0x7f94c810b110 2024-05-03T14:34:57.195 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:55.568+0000 7f94c6a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94c810b110 0x7f94c810b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:34:57.195 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:55.568+0000 7f94c6a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94c810b110 0x7f94c810b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41394/0 (socket says 172.21.15.60:41394) 2024-05-03T14:34:57.195 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:55.568+0000 7f94c6a00700 1 -- 172.21.15.60:0/4208193849 learned_addr learned my addr 172.21.15.60:0/4208193849 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:34:57.195 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:55.569+0000 7f94c6a00700 1 -- 172.21.15.60:0/4208193849 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94c810c2c0 con 0x7f94c810b110 2024-05-03T14:34:57.195 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:55.569+0000 7f94c6a00700 1 --2- 172.21.15.60:0/4208193849 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94c810b110 0x7f94c810b530 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f94b80098a0 tx=0x7f94b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c76132305edc625 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:34:57.195 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:55.570+0000 7f94c5600700 1 -- 172.21.15.60:0/4208193849 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f94b8004160 con 0x7f94c810b110 2024-05-03T14:34:57.195 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:55.570+0000 7f94c5600700 1 -- 172.21.15.60:0/4208193849 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94b8004300 con 0x7f94c810b110 2024-05-03T14:34:57.195 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:55.570+0000 7f94c5600700 1 -- 172.21.15.60:0/4208193849 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f94b80036a0 con 0x7f94c810b110 2024-05-03T14:34:57.195 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:55.570+0000 7f94c7e00700 1 -- 172.21.15.60:0/4208193849 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94c810b110 msgr2=0x7f94c810b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:34:57.195 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:55.570+0000 7f94c7e00700 1 --2- 172.21.15.60:0/4208193849 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94c810b110 0x7f94c810b530 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f94b80098a0 tx=0x7f94b8009bb0 comp rx=0 tx=0).stop 2024-05-03T14:34:57.195 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:55.571+0000 7f94c7e00700 1 -- 172.21.15.60:0/4208193849 shutdown_connections 2024-05-03T14:34:57.195 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:55.571+0000 7f94c7e00700 1 --2- 172.21.15.60:0/4208193849 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94c810b110 0x7f94c810b530 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:34:57.196 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:55.571+0000 7f94c7e00700 1 -- 172.21.15.60:0/4208193849 >> 172.21.15.60:0/4208193849 conn(0x7f94c807e540 msgr2=0x7f94c807e950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:34:57.196 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:55.571+0000 7f94c7e00700 1 -- 172.21.15.60:0/4208193849 shutdown_connections 2024-05-03T14:34:57.196 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:55.571+0000 7f94c7e00700 1 -- 172.21.15.60:0/4208193849 wait complete. 2024-05-03T14:34:57.196 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:55.572+0000 7f94c7e00700 1 Processor -- start 2024-05-03T14:34:57.196 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:55.573+0000 7f94c7e00700 1 -- start start 2024-05-03T14:34:57.196 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:55.573+0000 7f94c7e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94c810b110 0x7f94c819e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:34:57.196 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:55.573+0000 7f94c7e00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f94c810c7f0 con 0x7f94c810b110 2024-05-03T14:34:57.196 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:55.573+0000 7f94c6a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94c810b110 0x7f94c819e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:34:57.196 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:55.573+0000 7f94c6a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94c810b110 0x7f94c819e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41406/0 (socket says 172.21.15.60:41406) 2024-05-03T14:34:57.196 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:55.573+0000 7f94c6a00700 1 -- 172.21.15.60:0/188891433 learned_addr learned my addr 172.21.15.60:0/188891433 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:34:57.196 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:55.574+0000 7f94c6a00700 1 -- 172.21.15.60:0/188891433 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94c819ede0 con 0x7f94c810b110 2024-05-03T14:34:57.196 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:55.574+0000 7f94c6a00700 1 --2- 172.21.15.60:0/188891433 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94c810b110 0x7f94c819e8a0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f94b8000c00 tx=0x7f94b8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:34:57.196 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:55.574+0000 7f94bf400700 1 -- 172.21.15.60:0/188891433 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f94b8003db0 con 0x7f94c810b110 2024-05-03T14:34:57.196 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:55.574+0000 7f94bf400700 1 -- 172.21.15.60:0/188891433 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94b8011630 con 0x7f94c810b110 2024-05-03T14:34:57.197 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:55.574+0000 7f94bf400700 1 -- 172.21.15.60:0/188891433 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f94b8011950 con 0x7f94c810b110 2024-05-03T14:34:57.197 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:55.574+0000 7f94c7e00700 1 -- 172.21.15.60:0/188891433 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94c819f070 con 0x7f94c810b110 2024-05-03T14:34:57.197 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:55.575+0000 7f94c7e00700 1 -- 172.21.15.60:0/188891433 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94c81a1d60 con 0x7f94c810b110 2024-05-03T14:34:57.197 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:55.575+0000 7f94bf400700 1 -- 172.21.15.60:0/188891433 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f94b801a460 con 0x7f94c810b110 2024-05-03T14:34:57.197 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:55.575+0000 7f94bf400700 1 -- 172.21.15.60:0/188891433 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f94b801b780 con 0x7f94c810b110 2024-05-03T14:34:57.197 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:55.576+0000 7f94c7e00700 1 -- 172.21.15.60:0/188891433 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94c8064b00 con 0x7f94c810b110 2024-05-03T14:34:57.197 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:55.579+0000 7f94bf400700 1 -- 172.21.15.60:0/188891433 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f94b8031020 con 0x7f94c810b110 2024-05-03T14:34:57.197 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:55.657+0000 7f94c7e00700 1 -- 172.21.15.60:0/188891433 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f94c819f550 con 0x7f94c810b110 2024-05-03T14:34:57.197 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:55.665+0000 7f94bf400700 1 -- 172.21.15.60:0/188891433 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+465 (secure 0 0 0) 0x7f94c819f550 con 0x7f94c810b110 2024-05-03T14:34:57.197 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:55.667+0000 7f94bf400700 1 -- 172.21.15.60:0/188891433 <== mon.0 v2:172.21.15.60:3300/0 8 ==== config(28 keys) ==== 1139+0+0 (secure 0 0 0) 0x7f94b801a890 con 0x7f94c810b110 2024-05-03T14:34:57.197 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:55.667+0000 7f94c7e00700 1 -- 172.21.15.60:0/188891433 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94c810b110 msgr2=0x7f94c819e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:34:57.197 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:55.667+0000 7f94c7e00700 1 --2- 172.21.15.60:0/188891433 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94c810b110 0x7f94c819e8a0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f94b8000c00 tx=0x7f94b8004750 comp rx=0 tx=0).stop 2024-05-03T14:34:57.198 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:55.667+0000 7f94c7e00700 1 -- 172.21.15.60:0/188891433 shutdown_connections 2024-05-03T14:34:57.198 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:55.667+0000 7f94c7e00700 1 --2- 172.21.15.60:0/188891433 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94c810b110 0x7f94c819e8a0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:34:57.198 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:55.667+0000 7f94c7e00700 1 -- 172.21.15.60:0/188891433 >> 172.21.15.60:0/188891433 conn(0x7f94c807e540 msgr2=0x7f94c81954c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:34:57.198 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:55.667+0000 7f94c7e00700 1 -- 172.21.15.60:0/188891433 shutdown_connections 2024-05-03T14:34:57.198 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:55.668+0000 7f94c7e00700 1 -- 172.21.15.60:0/188891433 wait complete. 2024-05-03T14:34:57.198 INFO:teuthology.orchestra.run.smithi060.stdout:Generating new minimal ceph.conf... 2024-05-03T14:34:58.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:34:57 smithi060 ceph-mon[28508]: from='client.? 172.21.15.60:0/3720085980' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:34:58.464 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:57.799+0000 7f3abd400700 1 Processor -- start 2024-05-03T14:34:58.464 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:57.800+0000 7f3abd400700 1 -- start start 2024-05-03T14:34:58.464 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:57.800+0000 7f3abd400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ab810b350 0x7f3ab810b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:34:58.465 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:57.800+0000 7f3abd400700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab810bd40 con 0x7f3ab810b350 2024-05-03T14:34:58.465 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:57.801+0000 7f3ab6800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ab810b350 0x7f3ab810b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:34:58.465 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:57.801+0000 7f3ab6800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ab810b350 0x7f3ab810b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41414/0 (socket says 172.21.15.60:41414) 2024-05-03T14:34:58.466 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:57.801+0000 7f3ab6800700 1 -- 172.21.15.60:0/1873859014 learned_addr learned my addr 172.21.15.60:0/1873859014 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:34:58.467 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:57.801+0000 7f3ab6800700 1 -- 172.21.15.60:0/1873859014 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ab810c4b0 con 0x7f3ab810b350 2024-05-03T14:34:58.467 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:57.802+0000 7f3ab6800700 1 --2- 172.21.15.60:0/1873859014 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ab810b350 0x7f3ab810b770 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f3aa80098a0 tx=0x7f3aa8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53b24877ec9fb723 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:34:58.467 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:57.802+0000 7f3ab5e00700 1 -- 172.21.15.60:0/1873859014 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3aa8004160 con 0x7f3ab810b350 2024-05-03T14:34:58.467 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:57.802+0000 7f3ab5e00700 1 -- 172.21.15.60:0/1873859014 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1139+0+0 (secure 0 0 0) 0x7f3aa8004300 con 0x7f3ab810b350 2024-05-03T14:34:58.467 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:57.802+0000 7f3ab5e00700 1 -- 172.21.15.60:0/1873859014 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3aa8011c00 con 0x7f3ab810b350 2024-05-03T14:34:58.467 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:57.803+0000 7f3abd400700 1 -- 172.21.15.60:0/1873859014 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ab810b350 msgr2=0x7f3ab810b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:34:58.467 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:57.803+0000 7f3abd400700 1 --2- 172.21.15.60:0/1873859014 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ab810b350 0x7f3ab810b770 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f3aa80098a0 tx=0x7f3aa8009bb0 comp rx=0 tx=0).stop 2024-05-03T14:34:58.467 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:57.803+0000 7f3abd400700 1 -- 172.21.15.60:0/1873859014 shutdown_connections 2024-05-03T14:34:58.467 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:57.803+0000 7f3abd400700 1 --2- 172.21.15.60:0/1873859014 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ab810b350 0x7f3ab810b770 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:34:58.467 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:57.803+0000 7f3abd400700 1 -- 172.21.15.60:0/1873859014 >> 172.21.15.60:0/1873859014 conn(0x7f3ab807e860 msgr2=0x7f3ab81094f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:34:58.467 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:57.803+0000 7f3abd400700 1 -- 172.21.15.60:0/1873859014 shutdown_connections 2024-05-03T14:34:58.467 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:57.803+0000 7f3abd400700 1 -- 172.21.15.60:0/1873859014 wait complete. 2024-05-03T14:34:58.467 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:57.804+0000 7f3abd400700 1 Processor -- start 2024-05-03T14:34:58.467 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:57.804+0000 7f3abd400700 1 -- start start 2024-05-03T14:34:58.470 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:57.805+0000 7f3abd400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ab810b350 0x7f3ab819f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:34:58.470 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:57.805+0000 7f3abd400700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab810f860 con 0x7f3ab810b350 2024-05-03T14:34:58.470 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:57.805+0000 7f3ab6800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ab810b350 0x7f3ab819f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:34:58.470 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:57.805+0000 7f3ab6800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ab810b350 0x7f3ab819f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41426/0 (socket says 172.21.15.60:41426) 2024-05-03T14:34:58.470 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:57.805+0000 7f3ab6800700 1 -- 172.21.15.60:0/3720085980 learned_addr learned my addr 172.21.15.60:0/3720085980 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:34:58.471 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:57.805+0000 7f3ab6800700 1 -- 172.21.15.60:0/3720085980 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ab819f8a0 con 0x7f3ab810b350 2024-05-03T14:34:58.471 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:57.806+0000 7f3ab6800700 1 --2- 172.21.15.60:0/3720085980 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ab810b350 0x7f3ab819f360 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3aa80118c0 tx=0x7f3aa8011c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:34:58.471 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:57.806+0000 7f3aafe00700 1 -- 172.21.15.60:0/3720085980 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3aa8003af0 con 0x7f3ab810b350 2024-05-03T14:34:58.471 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:57.806+0000 7f3aafe00700 1 -- 172.21.15.60:0/3720085980 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1139+0+0 (secure 0 0 0) 0x7f3aa8020070 con 0x7f3ab810b350 2024-05-03T14:34:58.471 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:57.806+0000 7f3abd400700 1 -- 172.21.15.60:0/3720085980 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ab819fb30 con 0x7f3ab810b350 2024-05-03T14:34:58.471 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:57.806+0000 7f3aafe00700 1 -- 172.21.15.60:0/3720085980 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3aa80195e0 con 0x7f3ab810b350 2024-05-03T14:34:58.471 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:57.806+0000 7f3abd400700 1 -- 172.21.15.60:0/3720085980 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ab81a2820 con 0x7f3ab810b350 2024-05-03T14:34:58.471 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:57.806+0000 7f3aafe00700 1 -- 172.21.15.60:0/3720085980 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3aa8019be0 con 0x7f3ab810b350 2024-05-03T14:34:58.471 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:57.806+0000 7f3aafe00700 1 -- 172.21.15.60:0/3720085980 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3aa800b720 con 0x7f3ab810b350 2024-05-03T14:34:58.471 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:57.807+0000 7f3abd400700 1 -- 172.21.15.60:0/3720085980 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ab810f9e0 con 0x7f3ab810b350 2024-05-03T14:34:58.471 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:57.810+0000 7f3aafe00700 1 -- 172.21.15.60:0/3720085980 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f3aa8003c90 con 0x7f3ab810b350 2024-05-03T14:34:58.471 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:57.890+0000 7f3abd400700 1 -- 172.21.15.60:0/3720085980 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f3ab81a29f0 con 0x7f3ab810b350 2024-05-03T14:34:58.471 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:57.890+0000 7f3aafe00700 1 -- 172.21.15.60:0/3720085980 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f3aa801a9a0 con 0x7f3ab810b350 2024-05-03T14:34:58.471 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:57.892+0000 7f3abd400700 1 -- 172.21.15.60:0/3720085980 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ab810b350 msgr2=0x7f3ab819f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:34:58.472 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:57.892+0000 7f3abd400700 1 --2- 172.21.15.60:0/3720085980 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ab810b350 0x7f3ab819f360 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3aa80118c0 tx=0x7f3aa8011c80 comp rx=0 tx=0).stop 2024-05-03T14:34:58.472 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:57.892+0000 7f3abd400700 1 -- 172.21.15.60:0/3720085980 shutdown_connections 2024-05-03T14:34:58.472 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:57.892+0000 7f3abd400700 1 --2- 172.21.15.60:0/3720085980 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ab810b350 0x7f3ab819f360 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:34:58.472 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:57.892+0000 7f3abd400700 1 -- 172.21.15.60:0/3720085980 >> 172.21.15.60:0/3720085980 conn(0x7f3ab807e860 msgr2=0x7f3ab8108d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:34:58.472 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:57.892+0000 7f3abd400700 1 -- 172.21.15.60:0/3720085980 shutdown_connections 2024-05-03T14:34:58.472 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:34:57.892+0000 7f3abd400700 1 -- 172.21.15.60:0/3720085980 wait complete. 2024-05-03T14:34:58.472 INFO:teuthology.orchestra.run.smithi060.stdout:Restarting the monitor... 2024-05-03T14:34:58.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:34:58 smithi060 systemd[1]: Stopping Ceph mon.a for 285d64b2-095a-11ef-bc97-c7b262605968... 2024-05-03T14:34:59.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:34:58 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mon-a[28504]: 2024-05-03T14:34:58.952+0000 7f24c1400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T14:34:59.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:34:58 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mon-a[28504]: 2024-05-03T14:34:58.952+0000 7f24c1400700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-05-03T14:34:59.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:34:59 smithi060 podman[28749]: 2024-05-03 14:34:59.127819323 +0000 UTC m=+0.320983358 container died a712d2d80b4ebed8c63bad19f8a1390688e59c2faa713e1141312160275d3923 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-05-03T14:34:59.655 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:34:59 smithi060 podman[28749]: 2024-05-03 14:34:59.485733108 +0000 UTC m=+0.678897144 container cleanup a712d2d80b4ebed8c63bad19f8a1390688e59c2faa713e1141312160275d3923 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mon-a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-05-03T14:34:59.655 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:34:59 smithi060 bash[28749]: ceph-285d64b2-095a-11ef-bc97-c7b262605968-mon-a 2024-05-03T14:35:00.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:34:59 smithi060 podman[28763]: 2024-05-03 14:34:59.652913391 +0000 UTC m=+0.507646383 container remove a712d2d80b4ebed8c63bad19f8a1390688e59c2faa713e1141312160275d3923 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430) 2024-05-03T14:35:00.375 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 systemd[1]: ceph-285d64b2-095a-11ef-bc97-c7b262605968@mon.a.service: Deactivated successfully. 2024-05-03T14:35:00.375 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 systemd[1]: Stopped Ceph mon.a for 285d64b2-095a-11ef-bc97-c7b262605968. 2024-05-03T14:35:00.376 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 systemd[1]: ceph-285d64b2-095a-11ef-bc97-c7b262605968@mon.a.service: Consumed 1.947s CPU time. 2024-05-03T14:35:00.376 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 systemd[1]: Starting Ceph mon.a for 285d64b2-095a-11ef-bc97-c7b262605968... 2024-05-03T14:35:00.698 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 podman[28854]: 2024-05-03 14:35:00.456572929 +0000 UTC m=+0.035951827 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea 2024-05-03T14:35:00.698 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 podman[28854]: 2024-05-03 14:35:00.553908641 +0000 UTC m=+0.133287564 container create 1149168d47be4367a47e8c3614bd2067ca5a7b86039f95aa814d0f49b8a0581d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mon-a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS) 2024-05-03T14:35:00.892 INFO:teuthology.orchestra.run.smithi060.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-03T14:35:01.044 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 podman[28854]: 2024-05-03 14:35:00.774198445 +0000 UTC m=+0.353577411 container init 1149168d47be4367a47e8c3614bd2067ca5a7b86039f95aa814d0f49b8a0581d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-03T14:35:01.045 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 podman[28854]: 2024-05-03 14:35:00.785560953 +0000 UTC m=+0.364939882 container start 1149168d47be4367a47e8c3614bd2067ca5a7b86039f95aa814d0f49b8a0581d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-05-03T14:35:01.045 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: set uid:gid to 167:167 (ceph:ceph) 2024-05-03T14:35:01.045 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: ceph version 19.0.0-3450-g9a047fe7 (9a047fe7599a549042d06a3fb5b6813a203eabea) squid (dev), process ceph-mon, pid 2 2024-05-03T14:35:01.045 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: pidfile_write: ignore empty --pid-file 2024-05-03T14:35:01.045 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: load: jerasure load: lrc 2024-05-03T14:35:01.045 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: RocksDB version: 7.9.2 2024-05-03T14:35:01.045 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Git sha 0 2024-05-03T14:35:01.045 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Compile date 2024-05-01 21:12:03 2024-05-03T14:35:01.045 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: DB SUMMARY 2024-05-03T14:35:01.045 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: DB Session ID: C5PV6RCQEQAB5B39986U 2024-05-03T14:35:01.045 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: CURRENT file: CURRENT 2024-05-03T14:35:01.045 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: IDENTITY file: IDENTITY 2024-05-03T14:35:01.045 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-03T14:35:01.046 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-05-03T14:35:01.046 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105704 ; 2024-05-03T14:35:01.046 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.error_if_exists: 0 2024-05-03T14:35:01.046 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.create_if_missing: 0 2024-05-03T14:35:01.046 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.paranoid_checks: 1 2024-05-03T14:35:01.046 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-03T14:35:01.046 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-03T14:35:01.046 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-03T14:35:01.046 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.env: 0x563065955cc0 2024-05-03T14:35:01.046 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.fs: PosixFileSystem 2024-05-03T14:35:01.046 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.info_log: 0x56307e467de0 2024-05-03T14:35:01.046 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.max_file_opening_threads: 16 2024-05-03T14:35:01.047 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.statistics: (nil) 2024-05-03T14:35:01.047 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.use_fsync: 0 2024-05-03T14:35:01.047 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.max_log_file_size: 0 2024-05-03T14:35:01.047 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-03T14:35:01.047 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-03T14:35:01.047 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.keep_log_file_num: 1000 2024-05-03T14:35:01.048 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.recycle_log_file_num: 0 2024-05-03T14:35:01.048 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.allow_fallocate: 1 2024-05-03T14:35:01.048 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.allow_mmap_reads: 0 2024-05-03T14:35:01.048 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.allow_mmap_writes: 0 2024-05-03T14:35:01.048 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.use_direct_reads: 0 2024-05-03T14:35:01.048 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-03T14:35:01.048 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.create_missing_column_families: 0 2024-05-03T14:35:01.048 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.db_log_dir: 2024-05-03T14:35:01.048 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.wal_dir: 2024-05-03T14:35:01.048 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-03T14:35:01.048 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-03T14:35:01.048 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-03T14:35:01.048 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-03T14:35:01.048 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-03T14:35:01.048 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-03T14:35:01.049 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.advise_random_on_open: 1 2024-05-03T14:35:01.049 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.db_write_buffer_size: 0 2024-05-03T14:35:01.049 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.write_buffer_manager: 0x56307e4390e0 2024-05-03T14:35:01.049 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-03T14:35:01.049 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-03T14:35:01.049 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-03T14:35:01.049 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.rate_limiter: (nil) 2024-05-03T14:35:01.049 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-03T14:35:01.049 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.wal_recovery_mode: 2 2024-05-03T14:35:01.049 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.enable_thread_tracking: 0 2024-05-03T14:35:01.049 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.enable_pipelined_write: 0 2024-05-03T14:35:01.049 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.unordered_write: 0 2024-05-03T14:35:01.049 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-03T14:35:01.050 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-03T14:35:01.050 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-03T14:35:01.050 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-03T14:35:01.050 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.row_cache: None 2024-05-03T14:35:01.050 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.wal_filter: None 2024-05-03T14:35:01.050 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-03T14:35:01.050 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.allow_ingest_behind: 0 2024-05-03T14:35:01.050 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.two_write_queues: 0 2024-05-03T14:35:01.050 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.manual_wal_flush: 0 2024-05-03T14:35:01.050 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.wal_compression: 0 2024-05-03T14:35:01.050 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.atomic_flush: 0 2024-05-03T14:35:01.050 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-03T14:35:01.051 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-03T14:35:01.051 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-03T14:35:01.051 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.log_readahead_size: 0 2024-05-03T14:35:01.051 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-03T14:35:01.051 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.best_efforts_recovery: 0 2024-05-03T14:35:01.051 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-03T14:35:01.051 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-03T14:35:01.051 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.allow_data_in_errors: 0 2024-05-03T14:35:01.051 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.db_host_id: __hostname__ 2024-05-03T14:35:01.051 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-03T14:35:01.051 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.max_background_jobs: 2 2024-05-03T14:35:01.051 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.max_background_compactions: -1 2024-05-03T14:35:01.052 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.max_subcompactions: 1 2024-05-03T14:35:01.052 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-03T14:35:01.052 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-03T14:35:01.052 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-03T14:35:01.052 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.max_total_wal_size: 0 2024-05-03T14:35:01.052 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-03T14:35:01.052 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-03T14:35:01.052 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-03T14:35:01.052 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-03T14:35:01.052 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.max_open_files: -1 2024-05-03T14:35:01.052 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.bytes_per_sync: 0 2024-05-03T14:35:01.052 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-03T14:35:01.052 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-03T14:35:01.052 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.compaction_readahead_size: 0 2024-05-03T14:35:01.053 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.max_background_flushes: -1 2024-05-03T14:35:01.053 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Compression algorithms supported: 2024-05-03T14:35:01.053 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-03T14:35:01.053 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: kZSTD supported: 0 2024-05-03T14:35:01.053 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: kXpressCompression supported: 0 2024-05-03T14:35:01.053 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: kLZ4HCCompression supported: 1 2024-05-03T14:35:01.053 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: kZlibCompression supported: 1 2024-05-03T14:35:01.053 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: kSnappyCompression supported: 1 2024-05-03T14:35:01.053 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: kLZ4Compression supported: 1 2024-05-03T14:35:01.053 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: kBZip2Compression supported: 0 2024-05-03T14:35:01.053 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-03T14:35:01.053 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-03T14:35:01.053 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-05-03T14:35:01.053 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-03T14:35:01.054 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-03T14:35:01.054 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.merge_operator: 2024-05-03T14:35:01.054 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.compaction_filter: None 2024-05-03T14:35:01.054 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.compaction_filter_factory: None 2024-05-03T14:35:01.054 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.sst_partitioner_factory: None 2024-05-03T14:35:01.054 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-03T14:35:01.054 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-03T14:35:01.054 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56307e467fa0) 2024-05-03T14:35:01.054 INFO:journalctl@ceph.mon.a.smithi060.stdout: cache_index_and_filter_blocks: 1 2024-05-03T14:35:01.054 INFO:journalctl@ceph.mon.a.smithi060.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-03T14:35:01.054 INFO:journalctl@ceph.mon.a.smithi060.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-03T14:35:01.054 INFO:journalctl@ceph.mon.a.smithi060.stdout: pin_top_level_index_and_filter: 1 2024-05-03T14:35:01.054 INFO:journalctl@ceph.mon.a.smithi060.stdout: index_type: 0 2024-05-03T14:35:01.055 INFO:journalctl@ceph.mon.a.smithi060.stdout: data_block_index_type: 0 2024-05-03T14:35:01.055 INFO:journalctl@ceph.mon.a.smithi060.stdout: index_shortening: 1 2024-05-03T14:35:01.055 INFO:journalctl@ceph.mon.a.smithi060.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-03T14:35:01.055 INFO:journalctl@ceph.mon.a.smithi060.stdout: checksum: 4 2024-05-03T14:35:01.055 INFO:journalctl@ceph.mon.a.smithi060.stdout: no_block_cache: 0 2024-05-03T14:35:01.055 INFO:journalctl@ceph.mon.a.smithi060.stdout: block_cache: 0x56307e46b1f0 2024-05-03T14:35:01.055 INFO:journalctl@ceph.mon.a.smithi060.stdout: block_cache_name: BinnedLRUCache 2024-05-03T14:35:01.055 INFO:journalctl@ceph.mon.a.smithi060.stdout: block_cache_options: 2024-05-03T14:35:01.055 INFO:journalctl@ceph.mon.a.smithi060.stdout: capacity : 536870912 2024-05-03T14:35:01.055 INFO:journalctl@ceph.mon.a.smithi060.stdout: num_shard_bits : 4 2024-05-03T14:35:01.055 INFO:journalctl@ceph.mon.a.smithi060.stdout: strict_capacity_limit : 0 2024-05-03T14:35:01.055 INFO:journalctl@ceph.mon.a.smithi060.stdout: high_pri_pool_ratio: 0.000 2024-05-03T14:35:01.055 INFO:journalctl@ceph.mon.a.smithi060.stdout: block_cache_compressed: (nil) 2024-05-03T14:35:01.055 INFO:journalctl@ceph.mon.a.smithi060.stdout: persistent_cache: (nil) 2024-05-03T14:35:01.056 INFO:journalctl@ceph.mon.a.smithi060.stdout: block_size: 4096 2024-05-03T14:35:01.056 INFO:journalctl@ceph.mon.a.smithi060.stdout: block_size_deviation: 10 2024-05-03T14:35:01.056 INFO:journalctl@ceph.mon.a.smithi060.stdout: block_restart_interval: 16 2024-05-03T14:35:01.056 INFO:journalctl@ceph.mon.a.smithi060.stdout: index_block_restart_interval: 1 2024-05-03T14:35:01.056 INFO:journalctl@ceph.mon.a.smithi060.stdout: metadata_block_size: 4096 2024-05-03T14:35:01.056 INFO:journalctl@ceph.mon.a.smithi060.stdout: partition_filters: 0 2024-05-03T14:35:01.056 INFO:journalctl@ceph.mon.a.smithi060.stdout: use_delta_encoding: 1 2024-05-03T14:35:01.056 INFO:journalctl@ceph.mon.a.smithi060.stdout: filter_policy: bloomfilter 2024-05-03T14:35:01.056 INFO:journalctl@ceph.mon.a.smithi060.stdout: whole_key_filtering: 1 2024-05-03T14:35:01.056 INFO:journalctl@ceph.mon.a.smithi060.stdout: verify_compression: 0 2024-05-03T14:35:01.056 INFO:journalctl@ceph.mon.a.smithi060.stdout: read_amp_bytes_per_bit: 0 2024-05-03T14:35:01.056 INFO:journalctl@ceph.mon.a.smithi060.stdout: format_version: 5 2024-05-03T14:35:01.056 INFO:journalctl@ceph.mon.a.smithi060.stdout: enable_index_compression: 1 2024-05-03T14:35:01.056 INFO:journalctl@ceph.mon.a.smithi060.stdout: block_align: 0 2024-05-03T14:35:01.056 INFO:journalctl@ceph.mon.a.smithi060.stdout: max_auto_readahead_size: 262144 2024-05-03T14:35:01.056 INFO:journalctl@ceph.mon.a.smithi060.stdout: prepopulate_block_cache: 0 2024-05-03T14:35:01.056 INFO:journalctl@ceph.mon.a.smithi060.stdout: initial_auto_readahead_size: 8192 2024-05-03T14:35:01.057 INFO:journalctl@ceph.mon.a.smithi060.stdout: num_file_reads_for_auto_readahead: 2 2024-05-03T14:35:01.057 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.write_buffer_size: 33554432 2024-05-03T14:35:01.058 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.max_write_buffer_number: 2 2024-05-03T14:35:01.058 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.compression: NoCompression 2024-05-03T14:35:01.058 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.bottommost_compression: Disabled 2024-05-03T14:35:01.058 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.prefix_extractor: nullptr 2024-05-03T14:35:01.058 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-03T14:35:01.058 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.num_levels: 7 2024-05-03T14:35:01.058 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-03T14:35:01.058 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-03T14:35:01.058 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-03T14:35:01.058 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-03T14:35:01.058 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-03T14:35:01.058 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-03T14:35:01.059 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-03T14:35:01.059 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-03T14:35:01.059 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-03T14:35:01.059 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-03T14:35:01.059 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-03T14:35:01.059 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-03T14:35:01.059 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-03T14:35:01.059 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.compression_opts.level: 32767 2024-05-03T14:35:01.059 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.compression_opts.strategy: 0 2024-05-03T14:35:01.059 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-03T14:35:01.059 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-03T14:35:01.059 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-03T14:35:01.059 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-03T14:35:01.059 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.compression_opts.enabled: false 2024-05-03T14:35:01.059 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-03T14:35:01.060 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-03T14:35:01.060 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-03T14:35:01.060 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-03T14:35:01.060 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.target_file_size_base: 67108864 2024-05-03T14:35:01.060 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-03T14:35:01.060 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-03T14:35:01.060 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-03T14:35:01.060 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-03T14:35:01.060 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-03T14:35:01.060 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-03T14:35:01.060 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-03T14:35:01.060 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-03T14:35:01.060 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-03T14:35:01.060 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-03T14:35:01.060 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-03T14:35:01.060 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-03T14:35:01.061 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-03T14:35:01.061 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-03T14:35:01.061 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.arena_block_size: 1048576 2024-05-03T14:35:01.061 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-03T14:35:01.061 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-03T14:35:01.061 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.disable_auto_compactions: 0 2024-05-03T14:35:01.061 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-03T14:35:01.061 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-03T14:35:01.061 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-03T14:35:01.061 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-03T14:35:01.061 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-03T14:35:01.061 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-03T14:35:01.061 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-03T14:35:01.061 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-03T14:35:01.061 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-03T14:35:01.062 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-03T14:35:01.062 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-03T14:35:01.062 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.inplace_update_support: 0 2024-05-03T14:35:01.062 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-03T14:35:01.062 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-03T14:35:01.062 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-03T14:35:01.062 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-03T14:35:01.062 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.bloom_locality: 0 2024-05-03T14:35:01.062 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.max_successive_merges: 0 2024-05-03T14:35:01.062 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-03T14:35:01.062 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.paranoid_file_checks: 0 2024-05-03T14:35:01.062 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.force_consistency_checks: 1 2024-05-03T14:35:01.062 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.report_bg_io_stats: 0 2024-05-03T14:35:01.062 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.ttl: 2592000 2024-05-03T14:35:01.063 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-03T14:35:01.063 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-03T14:35:01.063 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-03T14:35:01.063 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.enable_blob_files: false 2024-05-03T14:35:01.063 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.min_blob_size: 0 2024-05-03T14:35:01.063 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.blob_file_size: 268435456 2024-05-03T14:35:01.063 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-03T14:35:01.063 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-03T14:35:01.063 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-03T14:35:01.063 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-03T14:35:01.063 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-03T14:35:01.063 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.blob_file_starting_level: 0 2024-05-03T14:35:01.063 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-03T14:35:01.063 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-03T14:35:01.063 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-03T14:35:01.064 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 2dc7c42a-68ed-4c15-b3a4-a815d1bf2a93 2024-05-03T14:35:01.064 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714746900845840, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-03T14:35:01.064 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-03T14:35:01.064 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714746900847725, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100804, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 328, "table_properties": {"data_size": 98693, "index_size": 303, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14316, "raw_average_key_size": 49, "raw_value_size": 90903, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 287, "num_filter_entries": 287, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714746900, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "2dc7c42a-68ed-4c15-b3a4-a815d1bf2a93", "db_session_id": "C5PV6RCQEQAB5B39986U", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-03T14:35:01.064 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714746900847988, "job": 1, "event": "recovery_finished"} 2024-05-03T14:35:01.064 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-03T14:35:01.064 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-03T14:35:01.064 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56307e588000 2024-05-03T14:35:01.064 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: DB pointer 0x56307e574000 2024-05-03T14:35:01.064 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-03T14:35:01.064 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-03T14:35:01.064 INFO:journalctl@ceph.mon.a.smithi060.stdout: ** DB Stats ** 2024-05-03T14:35:01.064 INFO:journalctl@ceph.mon.a.smithi060.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-03T14:35:01.064 INFO:journalctl@ceph.mon.a.smithi060.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-03T14:35:01.064 INFO:journalctl@ceph.mon.a.smithi060.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-03T14:35:01.065 INFO:journalctl@ceph.mon.a.smithi060.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-03T14:35:01.065 INFO:journalctl@ceph.mon.a.smithi060.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-03T14:35:01.065 INFO:journalctl@ceph.mon.a.smithi060.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-03T14:35:01.065 INFO:journalctl@ceph.mon.a.smithi060.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-03T14:35:01.065 INFO:journalctl@ceph.mon.a.smithi060.stdout: 2024-05-03T14:35:01.065 INFO:journalctl@ceph.mon.a.smithi060.stdout: ** Compaction Stats [default] ** 2024-05-03T14:35:01.065 INFO:journalctl@ceph.mon.a.smithi060.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-03T14:35:01.065 INFO:journalctl@ceph.mon.a.smithi060.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-03T14:35:01.065 INFO:journalctl@ceph.mon.a.smithi060.stdout: L0 2/0 100.30 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 73.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-03T14:35:01.065 INFO:journalctl@ceph.mon.a.smithi060.stdout: Sum 2/0 100.30 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 73.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-03T14:35:01.065 INFO:journalctl@ceph.mon.a.smithi060.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 73.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-03T14:35:01.065 INFO:journalctl@ceph.mon.a.smithi060.stdout: 2024-05-03T14:35:01.065 INFO:journalctl@ceph.mon.a.smithi060.stdout: ** Compaction Stats [default] ** 2024-05-03T14:35:01.065 INFO:journalctl@ceph.mon.a.smithi060.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-03T14:35:01.066 INFO:journalctl@ceph.mon.a.smithi060.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-03T14:35:01.066 INFO:journalctl@ceph.mon.a.smithi060.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 73.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-03T14:35:01.066 INFO:journalctl@ceph.mon.a.smithi060.stdout: 2024-05-03T14:35:01.066 INFO:journalctl@ceph.mon.a.smithi060.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-03T14:35:01.066 INFO:journalctl@ceph.mon.a.smithi060.stdout: 2024-05-03T14:35:01.066 INFO:journalctl@ceph.mon.a.smithi060.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-03T14:35:01.066 INFO:journalctl@ceph.mon.a.smithi060.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-03T14:35:01.066 INFO:journalctl@ceph.mon.a.smithi060.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-03T14:35:01.066 INFO:journalctl@ceph.mon.a.smithi060.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-03T14:35:01.066 INFO:journalctl@ceph.mon.a.smithi060.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-03T14:35:01.066 INFO:journalctl@ceph.mon.a.smithi060.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-03T14:35:01.066 INFO:journalctl@ceph.mon.a.smithi060.stdout: Cumulative compaction: 0.00 GB write, 12.88 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-03T14:35:01.066 INFO:journalctl@ceph.mon.a.smithi060.stdout: Interval compaction: 0.00 GB write, 12.88 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-03T14:35:01.067 INFO:journalctl@ceph.mon.a.smithi060.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-03T14:35:01.067 INFO:journalctl@ceph.mon.a.smithi060.stdout: Block cache BinnedLRUCache@0x56307e46b1f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-05-03T14:35:01.067 INFO:journalctl@ceph.mon.a.smithi060.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-05-03T14:35:01.067 INFO:journalctl@ceph.mon.a.smithi060.stdout: 2024-05-03T14:35:01.067 INFO:journalctl@ceph.mon.a.smithi060.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-03T14:35:01.067 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: starting mon.a rank 0 at public addrs [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] at bind addrs [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 285d64b2-095a-11ef-bc97-c7b262605968 2024-05-03T14:35:01.067 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: mon.a@-1(???) e1 preinit fsid 285d64b2-095a-11ef-bc97-c7b262605968 2024-05-03T14:35:01.067 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: mon.a@-1(???).mds e1 new map 2024-05-03T14:35:01.067 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: mon.a@-1(???).mds e1 print_map 2024-05-03T14:35:01.067 INFO:journalctl@ceph.mon.a.smithi060.stdout: e1 2024-05-03T14:35:01.067 INFO:journalctl@ceph.mon.a.smithi060.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-03T14:35:01.068 INFO:journalctl@ceph.mon.a.smithi060.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-03T14:35:01.068 INFO:journalctl@ceph.mon.a.smithi060.stdout: legacy client fscid: -1 2024-05-03T14:35:01.068 INFO:journalctl@ceph.mon.a.smithi060.stdout: 2024-05-03T14:35:01.068 INFO:journalctl@ceph.mon.a.smithi060.stdout: No filesystems configured 2024-05-03T14:35:01.068 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-03T14:35:01.068 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-03T14:35:01.069 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-03T14:35:01.069 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-03T14:35:01.069 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-03T14:35:01.069 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 bash[28854]: 1149168d47be4367a47e8c3614bd2067ca5a7b86039f95aa814d0f49b8a0581d 2024-05-03T14:35:01.069 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 systemd[1]: Started Ceph mon.a for 285d64b2-095a-11ef-bc97-c7b262605968. 2024-05-03T14:35:01.069 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-03T14:35:01.069 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: monmap epoch 1 2024-05-03T14:35:01.069 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: fsid 285d64b2-095a-11ef-bc97-c7b262605968 2024-05-03T14:35:01.069 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: last_changed 2024-05-03T14:34:40.229121+0000 2024-05-03T14:35:01.070 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: created 2024-05-03T14:34:40.229121+0000 2024-05-03T14:35:01.070 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: min_mon_release 19 (squid) 2024-05-03T14:35:01.070 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: election_strategy: 1 2024-05-03T14:35:01.070 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: 0: [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] mon.a 2024-05-03T14:35:01.070 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: fsmap 2024-05-03T14:35:01.070 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: osdmap e1: 0 total, 0 up, 0 in 2024-05-03T14:35:01.070 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:00 smithi060 ceph-mon[28873]: mgrmap e1: no daemons active 2024-05-03T14:35:02.389 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:01.577+0000 7fb735000700 1 Processor -- start 2024-05-03T14:35:02.390 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:01.578+0000 7fb735000700 1 -- start start 2024-05-03T14:35:02.390 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:01.578+0000 7fb735000700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb73007edb0 0x7fb73007d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:35:02.390 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:01.578+0000 7fb735000700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fb73007d860 con 0x7fb73007edb0 2024-05-03T14:35:02.390 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:01.579+0000 7fb72de00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb73007edb0 0x7fb73007d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:35:02.390 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:01.579+0000 7fb72de00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb73007edb0 0x7fb73007d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41428/0 (socket says 172.21.15.60:41428) 2024-05-03T14:35:02.392 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:01.579+0000 7fb72de00700 1 -- 172.21.15.60:0/1312291808 learned_addr learned my addr 172.21.15.60:0/1312291808 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:35:02.392 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:01.580+0000 7fb72de00700 1 -- 172.21.15.60:0/1312291808 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb73007d9e0 con 0x7fb73007edb0 2024-05-03T14:35:02.392 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:01.580+0000 7fb72de00700 1 --2- 172.21.15.60:0/1312291808 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb73007edb0 0x7fb73007d320 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb7200098a0 tx=0x7fb720009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70a203a2f6b331e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:35:02.392 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:01.580+0000 7fb72ca00700 1 -- 172.21.15.60:0/1312291808 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb720004160 con 0x7fb73007edb0 2024-05-03T14:35:02.392 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:01.580+0000 7fb72ca00700 1 -- 172.21.15.60:0/1312291808 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1139+0+0 (secure 0 0 0) 0x7fb720004300 con 0x7fb73007edb0 2024-05-03T14:35:02.392 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:01.580+0000 7fb72ca00700 1 -- 172.21.15.60:0/1312291808 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb720011c00 con 0x7fb73007edb0 2024-05-03T14:35:02.392 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:01.581+0000 7fb735000700 1 -- 172.21.15.60:0/1312291808 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb73007edb0 msgr2=0x7fb73007d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:35:02.392 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:01.581+0000 7fb735000700 1 --2- 172.21.15.60:0/1312291808 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb73007edb0 0x7fb73007d320 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb7200098a0 tx=0x7fb720009bb0 comp rx=0 tx=0).stop 2024-05-03T14:35:02.392 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:01.581+0000 7fb735000700 1 -- 172.21.15.60:0/1312291808 shutdown_connections 2024-05-03T14:35:02.392 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:01.581+0000 7fb735000700 1 --2- 172.21.15.60:0/1312291808 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb73007edb0 0x7fb73007d320 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:35:02.392 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:01.581+0000 7fb735000700 1 -- 172.21.15.60:0/1312291808 >> 172.21.15.60:0/1312291808 conn(0x7fb730104750 msgr2=0x7fb730106b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:35:02.392 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:01.582+0000 7fb735000700 1 -- 172.21.15.60:0/1312291808 shutdown_connections 2024-05-03T14:35:02.396 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:01.582+0000 7fb735000700 1 -- 172.21.15.60:0/1312291808 wait complete. 2024-05-03T14:35:02.396 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:01.583+0000 7fb735000700 1 Processor -- start 2024-05-03T14:35:02.396 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:01.583+0000 7fb735000700 1 -- start start 2024-05-03T14:35:02.396 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:01.583+0000 7fb735000700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb73007edb0 0x7fb73019f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:35:02.396 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:01.583+0000 7fb735000700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fb73010f600 con 0x7fb73007edb0 2024-05-03T14:35:02.396 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:01.584+0000 7fb72de00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb73007edb0 0x7fb73019f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:35:02.396 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:01.584+0000 7fb72de00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb73007edb0 0x7fb73019f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41432/0 (socket says 172.21.15.60:41432) 2024-05-03T14:35:02.396 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:01.584+0000 7fb72de00700 1 -- 172.21.15.60:0/1759365003 learned_addr learned my addr 172.21.15.60:0/1759365003 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:35:02.396 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:01.584+0000 7fb72de00700 1 -- 172.21.15.60:0/1759365003 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb73019f7f0 con 0x7fb73007edb0 2024-05-03T14:35:02.396 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:01.584+0000 7fb72de00700 1 --2- 172.21.15.60:0/1759365003 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb73007edb0 0x7fb73019f2b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb720011bd0 tx=0x7fb720011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:35:02.396 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:01.585+0000 7fb726a00700 1 -- 172.21.15.60:0/1759365003 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb720003b20 con 0x7fb73007edb0 2024-05-03T14:35:02.396 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:01.585+0000 7fb726a00700 1 -- 172.21.15.60:0/1759365003 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1139+0+0 (secure 0 0 0) 0x7fb720003cc0 con 0x7fb73007edb0 2024-05-03T14:35:02.397 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:01.585+0000 7fb726a00700 1 -- 172.21.15.60:0/1759365003 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb720024930 con 0x7fb73007edb0 2024-05-03T14:35:02.397 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:01.585+0000 7fb735000700 1 -- 172.21.15.60:0/1759365003 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb73019fa80 con 0x7fb73007edb0 2024-05-03T14:35:02.397 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:01.585+0000 7fb735000700 1 -- 172.21.15.60:0/1759365003 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7301a2770 con 0x7fb73007edb0 2024-05-03T14:35:02.397 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:01.585+0000 7fb726a00700 1 -- 172.21.15.60:0/1759365003 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb720022020 con 0x7fb73007edb0 2024-05-03T14:35:02.397 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:01.585+0000 7fb726a00700 1 -- 172.21.15.60:0/1759365003 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb7200193c0 con 0x7fb73007edb0 2024-05-03T14:35:02.397 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:01.586+0000 7fb735000700 1 -- 172.21.15.60:0/1759365003 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb730064b00 con 0x7fb73007edb0 2024-05-03T14:35:02.397 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:01.589+0000 7fb726a00700 1 -- 172.21.15.60:0/1759365003 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fb72001aca0 con 0x7fb73007edb0 2024-05-03T14:35:02.397 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:01.669+0000 7fb735000700 1 -- 172.21.15.60:0/1759365003 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fb7301a2940 con 0x7fb73007edb0 2024-05-03T14:35:02.397 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:01.672+0000 7fb726a00700 1 -- 172.21.15.60:0/1759365003 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fb72002d3f0 con 0x7fb73007edb0 2024-05-03T14:35:02.397 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:01.673+0000 7fb735000700 1 -- 172.21.15.60:0/1759365003 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb73007edb0 msgr2=0x7fb73019f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:35:02.397 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:01.673+0000 7fb735000700 1 --2- 172.21.15.60:0/1759365003 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb73007edb0 0x7fb73019f2b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb720011bd0 tx=0x7fb720011cb0 comp rx=0 tx=0).stop 2024-05-03T14:35:02.397 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:01.673+0000 7fb735000700 1 -- 172.21.15.60:0/1759365003 shutdown_connections 2024-05-03T14:35:02.397 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:01.673+0000 7fb735000700 1 --2- 172.21.15.60:0/1759365003 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb73007edb0 0x7fb73019f2b0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:35:02.397 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:01.673+0000 7fb735000700 1 -- 172.21.15.60:0/1759365003 >> 172.21.15.60:0/1759365003 conn(0x7fb730104750 msgr2=0x7fb7301052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:35:02.397 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:01.674+0000 7fb735000700 1 -- 172.21.15.60:0/1759365003 shutdown_connections 2024-05-03T14:35:02.398 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:01.674+0000 7fb735000700 1 -- 172.21.15.60:0/1759365003 wait complete. 2024-05-03T14:35:02.398 INFO:teuthology.orchestra.run.smithi060.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-03T14:35:02.423 INFO:teuthology.orchestra.run.smithi060.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-03T14:35:02.423 INFO:teuthology.orchestra.run.smithi060.stdout:Creating mgr... 2024-05-03T14:35:02.424 INFO:teuthology.orchestra.run.smithi060.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-03T14:35:02.425 INFO:teuthology.orchestra.run.smithi060.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-03T14:35:02.860 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-285d64b2-095a-11ef-bc97-c7b262605968@mgr.x 2024-05-03T14:35:02.861 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stderr Failed to reset failed state of unit ceph-285d64b2-095a-11ef-bc97-c7b262605968@mgr.x.service: Unit ceph-285d64b2-095a-11ef-bc97-c7b262605968@mgr.x.service not loaded. 2024-05-03T14:35:03.029 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:02 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/1759365003' entity='client.admin' 2024-05-03T14:35:03.129 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-285d64b2-095a-11ef-bc97-c7b262605968.target.wants/ceph-285d64b2-095a-11ef-bc97-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-285d64b2-095a-11ef-bc97-c7b262605968@.service. 2024-05-03T14:35:03.338 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:03 smithi060 systemd[1]: Starting Ceph mgr.x for 285d64b2-095a-11ef-bc97-c7b262605968... 2024-05-03T14:35:03.839 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:03 smithi060 podman[29096]: 2024-05-03 14:35:03.45143027 +0000 UTC m=+0.040349569 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea 2024-05-03T14:35:03.839 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:03 smithi060 podman[29096]: 2024-05-03 14:35:03.557006829 +0000 UTC m=+0.145926135 container create 5813c13738d02eba3243414f18be1b0e883ff7e44e629fece7ae8498dcafe66a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-05-03T14:35:03.840 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:03 smithi060 podman[29096]: 2024-05-03 14:35:03.777066528 +0000 UTC m=+0.365985887 container init 5813c13738d02eba3243414f18be1b0e883ff7e44e629fece7ae8498dcafe66a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430) 2024-05-03T14:35:03.840 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:03 smithi060 podman[29096]: 2024-05-03 14:35:03.78760725 +0000 UTC m=+0.376526583 container start 5813c13738d02eba3243414f18be1b0e883ff7e44e629fece7ae8498dcafe66a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_CLEAN=True, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=) 2024-05-03T14:35:03.901 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-03T14:35:03.901 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout disabled 2024-05-03T14:35:03.917 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-03T14:35:03.917 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout inactive 2024-05-03T14:35:03.917 INFO:teuthology.orchestra.run.smithi060.stdout:firewalld.service is not enabled 2024-05-03T14:35:03.918 INFO:teuthology.orchestra.run.smithi060.stdout:Not possible to enable service . firewalld.service is not available 2024-05-03T14:35:03.946 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-03T14:35:03.946 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout disabled 2024-05-03T14:35:03.967 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-03T14:35:03.967 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout inactive 2024-05-03T14:35:03.967 INFO:teuthology.orchestra.run.smithi060.stdout:firewalld.service is not enabled 2024-05-03T14:35:03.968 INFO:teuthology.orchestra.run.smithi060.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-05-03T14:35:03.968 INFO:teuthology.orchestra.run.smithi060.stdout:Waiting for mgr to start... 2024-05-03T14:35:03.968 INFO:teuthology.orchestra.run.smithi060.stdout:Waiting for mgr... 2024-05-03T14:35:04.127 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:03 smithi060 bash[29096]: 5813c13738d02eba3243414f18be1b0e883ff7e44e629fece7ae8498dcafe66a 2024-05-03T14:35:04.128 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:03 smithi060 systemd[1]: Started Ceph mgr.x for 285d64b2-095a-11ef-bc97-c7b262605968. 2024-05-03T14:35:04.128 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:04 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:04.042+0000 7f215aa00200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-03T14:35:04.449 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:04 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:04.312+0000 7f215aa00200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-03T14:35:04.701 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:04 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:04.449+0000 7f215aa00200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-03T14:35:04.999 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:04 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/2488849435' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-03T14:35:04.999 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:04 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:04.878+0000 7f215aa00200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-03T14:35:04.999 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:04 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:04.996+0000 7f215aa00200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-03T14:35:05.392 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-05-03T14:35:05.393 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout { 2024-05-03T14:35:05.393 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "fsid": "285d64b2-095a-11ef-bc97-c7b262605968", 2024-05-03T14:35:05.393 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "health": { 2024-05-03T14:35:05.393 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-03T14:35:05.393 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-03T14:35:05.394 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-03T14:35:05.394 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-05-03T14:35:05.394 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-03T14:35:05.394 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-03T14:35:05.396 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 0 2024-05-03T14:35:05.396 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-05-03T14:35:05.396 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-03T14:35:05.396 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "a" 2024-05-03T14:35:05.396 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-05-03T14:35:05.396 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-05-03T14:35:05.396 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-03T14:35:05.396 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T14:35:05.396 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-03T14:35:05.396 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-03T14:35:05.396 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-05-03T14:35:05.396 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-03T14:35:05.396 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T14:35:05.397 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-03T14:35:05.397 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-03T14:35:05.397 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-03T14:35:05.401 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-03T14:35:05.401 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-03T14:35:05.401 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-03T14:35:05.401 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-05-03T14:35:05.401 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-03T14:35:05.401 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-03T14:35:05.401 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-03T14:35:05.401 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-03T14:35:05.401 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-03T14:35:05.401 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-03T14:35:05.401 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-03T14:35:05.401 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-03T14:35:05.401 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-03T14:35:05.401 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-05-03T14:35:05.401 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-03T14:35:05.402 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T14:35:05.402 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-03T14:35:05.402 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-03T14:35:05.402 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-05-03T14:35:05.402 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-03T14:35:05.402 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-03T14:35:05.402 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-03T14:35:05.402 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-03T14:35:05.402 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-03T14:35:05.402 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-03T14:35:05.402 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "restful" 2024-05-03T14:35:05.402 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-05-03T14:35:05.402 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-03T14:35:05.403 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-05-03T14:35:05.403 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-03T14:35:05.403 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T14:35:05.403 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "modified": "2024-05-03T14:34:53.561346+0000", 2024-05-03T14:35:05.403 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-03T14:35:05.403 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-05-03T14:35:05.403 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-03T14:35:05.403 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout } 2024-05-03T14:35:05.403 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:04.606+0000 7f3143200700 1 Processor -- start 2024-05-03T14:35:05.403 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:04.606+0000 7f3143200700 1 -- start start 2024-05-03T14:35:05.403 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:04.607+0000 7f3143200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f314407ee50 0x7f314407d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:35:05.403 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:04.607+0000 7f3143200700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f314407d840 con 0x7f314407ee50 2024-05-03T14:35:05.403 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:04.607+0000 7f3141e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f314407ee50 0x7f314407d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:35:05.403 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:04.607+0000 7f3141e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f314407ee50 0x7f314407d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44966/0 (socket says 172.21.15.60:44966) 2024-05-03T14:35:05.403 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:04.607+0000 7f3141e00700 1 -- 172.21.15.60:0/739519118 learned_addr learned my addr 172.21.15.60:0/739519118 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:35:05.407 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:04.608+0000 7f3141e00700 1 -- 172.21.15.60:0/739519118 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f314407d9c0 con 0x7f314407ee50 2024-05-03T14:35:05.408 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:04.608+0000 7f3141e00700 1 --2- 172.21.15.60:0/739519118 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f314407ee50 0x7f314407d300 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f31340098a0 tx=0x7f3134009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e4abee5a14c5837 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:35:05.408 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:04.608+0000 7f3140a00700 1 -- 172.21.15.60:0/739519118 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3134004160 con 0x7f314407ee50 2024-05-03T14:35:05.408 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:04.609+0000 7f3140a00700 1 -- 172.21.15.60:0/739519118 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1139+0+0 (secure 0 0 0) 0x7f3134004300 con 0x7f314407ee50 2024-05-03T14:35:05.408 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:04.609+0000 7f3140a00700 1 -- 172.21.15.60:0/739519118 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3134011c00 con 0x7f314407ee50 2024-05-03T14:35:05.408 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:04.609+0000 7f3143200700 1 -- 172.21.15.60:0/739519118 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f314407ee50 msgr2=0x7f314407d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:35:05.408 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:04.609+0000 7f3143200700 1 --2- 172.21.15.60:0/739519118 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f314407ee50 0x7f314407d300 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f31340098a0 tx=0x7f3134009bb0 comp rx=0 tx=0).stop 2024-05-03T14:35:05.408 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:04.610+0000 7f3143200700 1 -- 172.21.15.60:0/739519118 shutdown_connections 2024-05-03T14:35:05.408 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:04.610+0000 7f3143200700 1 --2- 172.21.15.60:0/739519118 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f314407ee50 0x7f314407d300 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:35:05.408 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:04.610+0000 7f3143200700 1 -- 172.21.15.60:0/739519118 >> 172.21.15.60:0/739519118 conn(0x7f3144104500 msgr2=0x7f3144106920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:35:05.408 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:04.610+0000 7f3143200700 1 -- 172.21.15.60:0/739519118 shutdown_connections 2024-05-03T14:35:05.408 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:04.610+0000 7f3143200700 1 -- 172.21.15.60:0/739519118 wait complete. 2024-05-03T14:35:05.408 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:04.611+0000 7f3143200700 1 Processor -- start 2024-05-03T14:35:05.408 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:04.611+0000 7f3143200700 1 -- start start 2024-05-03T14:35:05.408 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:04.611+0000 7f3143200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f314407ee50 0x7f314419f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:35:05.409 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:04.611+0000 7f3143200700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f314410f570 con 0x7f314407ee50 2024-05-03T14:35:05.409 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:04.612+0000 7f3141e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f314407ee50 0x7f314419f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:35:05.409 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:04.612+0000 7f3141e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f314407ee50 0x7f314419f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44980/0 (socket says 172.21.15.60:44980) 2024-05-03T14:35:05.409 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:04.612+0000 7f3141e00700 1 -- 172.21.15.60:0/2488849435 learned_addr learned my addr 172.21.15.60:0/2488849435 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:35:05.409 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:04.612+0000 7f3141e00700 1 -- 172.21.15.60:0/2488849435 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f314419f6e0 con 0x7f314407ee50 2024-05-03T14:35:05.409 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:04.612+0000 7f3141e00700 1 --2- 172.21.15.60:0/2488849435 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f314407ee50 0x7f314419f1a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3134011bd0 tx=0x7f3134011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:35:05.409 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:04.613+0000 7f313aa00700 1 -- 172.21.15.60:0/2488849435 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3134003970 con 0x7f314407ee50 2024-05-03T14:35:05.409 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:04.613+0000 7f313aa00700 1 -- 172.21.15.60:0/2488849435 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1139+0+0 (secure 0 0 0) 0x7f3134003b10 con 0x7f314407ee50 2024-05-03T14:35:05.409 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:04.613+0000 7f3143200700 1 -- 172.21.15.60:0/2488849435 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f314419f970 con 0x7f314407ee50 2024-05-03T14:35:05.409 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:04.613+0000 7f313aa00700 1 -- 172.21.15.60:0/2488849435 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f31340245a0 con 0x7f314407ee50 2024-05-03T14:35:05.409 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:04.613+0000 7f3143200700 1 -- 172.21.15.60:0/2488849435 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31441a2660 con 0x7f314407ee50 2024-05-03T14:35:05.409 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:04.614+0000 7f313aa00700 1 -- 172.21.15.60:0/2488849435 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3134024c20 con 0x7f314407ee50 2024-05-03T14:35:05.410 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:04.614+0000 7f313aa00700 1 -- 172.21.15.60:0/2488849435 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3134019700 con 0x7f314407ee50 2024-05-03T14:35:05.410 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:04.614+0000 7f3143200700 1 -- 172.21.15.60:0/2488849435 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3144064b00 con 0x7f314407ee50 2024-05-03T14:35:05.410 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:04.619+0000 7f313aa00700 1 -- 172.21.15.60:0/2488849435 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f3134043e50 con 0x7f314407ee50 2024-05-03T14:35:05.410 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:04.698+0000 7f3143200700 1 -- 172.21.15.60:0/2488849435 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f31441a2830 con 0x7f314407ee50 2024-05-03T14:35:05.410 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:04.699+0000 7f313aa00700 1 -- 172.21.15.60:0/2488849435 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f3134032030 con 0x7f314407ee50 2024-05-03T14:35:05.410 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:04.700+0000 7f3143200700 1 -- 172.21.15.60:0/2488849435 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f314407ee50 msgr2=0x7f314419f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:35:05.410 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:04.700+0000 7f3143200700 1 --2- 172.21.15.60:0/2488849435 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f314407ee50 0x7f314419f1a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3134011bd0 tx=0x7f3134011cb0 comp rx=0 tx=0).stop 2024-05-03T14:35:05.410 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:04.700+0000 7f3143200700 1 -- 172.21.15.60:0/2488849435 shutdown_connections 2024-05-03T14:35:05.410 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:04.700+0000 7f3143200700 1 --2- 172.21.15.60:0/2488849435 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f314407ee50 0x7f314419f1a0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:35:05.410 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:04.700+0000 7f3143200700 1 -- 172.21.15.60:0/2488849435 >> 172.21.15.60:0/2488849435 conn(0x7f3144104500 msgr2=0x7f31441050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:35:05.410 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:04.701+0000 7f3143200700 1 -- 172.21.15.60:0/2488849435 shutdown_connections 2024-05-03T14:35:05.410 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:04.701+0000 7f3143200700 1 -- 172.21.15.60:0/2488849435 wait complete. 2024-05-03T14:35:05.410 INFO:teuthology.orchestra.run.smithi060.stdout:mgr not available, waiting (1/15)... 2024-05-03T14:35:07.087 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:06 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:06.643+0000 7f215aa00200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-03T14:35:07.088 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:06 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:06.878+0000 7f215aa00200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-03T14:35:07.502 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:07 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:07.097+0000 7f215aa00200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-03T14:35:07.758 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:07 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:07.499+0000 7f215aa00200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-03T14:35:07.758 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:07 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:07.630+0000 7f215aa00200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-03T14:35:08.019 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:07 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:07.895+0000 7f215aa00200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-03T14:35:08.337 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:08 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:08.163+0000 7f215aa00200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-03T14:35:08.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:08 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/1894468133' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-03T14:35:08.774 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-05-03T14:35:08.775 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout { 2024-05-03T14:35:08.775 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "fsid": "285d64b2-095a-11ef-bc97-c7b262605968", 2024-05-03T14:35:08.775 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "health": { 2024-05-03T14:35:08.775 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-03T14:35:08.775 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-03T14:35:08.776 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-03T14:35:08.777 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-05-03T14:35:08.777 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-03T14:35:08.777 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-03T14:35:08.777 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 0 2024-05-03T14:35:08.777 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-05-03T14:35:08.777 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-03T14:35:08.777 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "a" 2024-05-03T14:35:08.777 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-05-03T14:35:08.779 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-05-03T14:35:08.779 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-03T14:35:08.779 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T14:35:08.779 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-03T14:35:08.779 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-03T14:35:08.779 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-05-03T14:35:08.779 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-03T14:35:08.779 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T14:35:08.779 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-03T14:35:08.779 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-03T14:35:08.779 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-03T14:35:08.784 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-03T14:35:08.784 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-03T14:35:08.784 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-03T14:35:08.784 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-05-03T14:35:08.784 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-03T14:35:08.784 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-03T14:35:08.784 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-03T14:35:08.784 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-03T14:35:08.784 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-03T14:35:08.784 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-03T14:35:08.784 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-03T14:35:08.784 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-03T14:35:08.784 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-03T14:35:08.785 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-05-03T14:35:08.785 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-03T14:35:08.785 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T14:35:08.785 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-03T14:35:08.785 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-03T14:35:08.785 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-05-03T14:35:08.785 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-03T14:35:08.785 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-03T14:35:08.785 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-03T14:35:08.785 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-03T14:35:08.785 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-03T14:35:08.785 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-03T14:35:08.785 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "restful" 2024-05-03T14:35:08.785 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-05-03T14:35:08.786 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-03T14:35:08.786 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-05-03T14:35:08.786 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-03T14:35:08.786 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T14:35:08.786 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "modified": "2024-05-03T14:34:53.561346+0000", 2024-05-03T14:35:08.786 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-03T14:35:08.786 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-05-03T14:35:08.786 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-03T14:35:08.786 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout } 2024-05-03T14:35:08.786 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:08.016+0000 7f4fd8a00700 1 Processor -- start 2024-05-03T14:35:08.786 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:08.016+0000 7f4fd8a00700 1 -- start start 2024-05-03T14:35:08.786 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:08.016+0000 7f4fd8a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fd4109bd0 0x7f4fd410bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:35:08.786 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:08.016+0000 7f4fd8a00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd407ce70 con 0x7f4fd4109bd0 2024-05-03T14:35:08.786 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:08.017+0000 7f4fd1e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fd4109bd0 0x7f4fd410bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:35:08.787 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:08.017+0000 7f4fd1e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fd4109bd0 0x7f4fd410bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44982/0 (socket says 172.21.15.60:44982) 2024-05-03T14:35:08.791 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:08.017+0000 7f4fd1e00700 1 -- 172.21.15.60:0/2976496490 learned_addr learned my addr 172.21.15.60:0/2976496490 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:35:08.791 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:08.017+0000 7f4fd1e00700 1 -- 172.21.15.60:0/2976496490 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fd410c500 con 0x7f4fd4109bd0 2024-05-03T14:35:08.791 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:08.018+0000 7f4fd1e00700 1 --2- 172.21.15.60:0/2976496490 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fd4109bd0 0x7f4fd410bfc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4fc40098a0 tx=0x7f4fc4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e34dcd4e84903f7c server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:35:08.791 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:08.018+0000 7f4fd0a00700 1 -- 172.21.15.60:0/2976496490 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4fc4004160 con 0x7f4fd4109bd0 2024-05-03T14:35:08.791 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:08.018+0000 7f4fd0a00700 1 -- 172.21.15.60:0/2976496490 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1139+0+0 (secure 0 0 0) 0x7f4fc4004300 con 0x7f4fd4109bd0 2024-05-03T14:35:08.791 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:08.018+0000 7f4fd0a00700 1 -- 172.21.15.60:0/2976496490 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4fc4011c00 con 0x7f4fd4109bd0 2024-05-03T14:35:08.791 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:08.019+0000 7f4fd8a00700 1 -- 172.21.15.60:0/2976496490 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fd4109bd0 msgr2=0x7f4fd410bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:35:08.791 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:08.019+0000 7f4fd8a00700 1 --2- 172.21.15.60:0/2976496490 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fd4109bd0 0x7f4fd410bfc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4fc40098a0 tx=0x7f4fc4009bb0 comp rx=0 tx=0).stop 2024-05-03T14:35:08.791 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:08.019+0000 7f4fd8a00700 1 -- 172.21.15.60:0/2976496490 shutdown_connections 2024-05-03T14:35:08.792 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:08.019+0000 7f4fd8a00700 1 --2- 172.21.15.60:0/2976496490 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fd4109bd0 0x7f4fd410bfc0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:35:08.792 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:08.019+0000 7f4fd8a00700 1 -- 172.21.15.60:0/2976496490 >> 172.21.15.60:0/2976496490 conn(0x7f4fd41035c0 msgr2=0x7f4fd41059e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:35:08.792 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:08.019+0000 7f4fd8a00700 1 -- 172.21.15.60:0/2976496490 shutdown_connections 2024-05-03T14:35:08.792 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:08.019+0000 7f4fd8a00700 1 -- 172.21.15.60:0/2976496490 wait complete. 2024-05-03T14:35:08.792 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:08.021+0000 7f4fd8a00700 1 Processor -- start 2024-05-03T14:35:08.792 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:08.021+0000 7f4fd8a00700 1 -- start start 2024-05-03T14:35:08.792 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:08.021+0000 7f4fd8a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fd4109bd0 0x7f4fd4109450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:35:08.792 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:08.021+0000 7f4fd8a00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd410d4c0 con 0x7f4fd4109bd0 2024-05-03T14:35:08.792 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:08.022+0000 7f4fd1e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fd4109bd0 0x7f4fd4109450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:35:08.792 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:08.022+0000 7f4fd1e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fd4109bd0 0x7f4fd4109450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44994/0 (socket says 172.21.15.60:44994) 2024-05-03T14:35:08.792 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:08.022+0000 7f4fd1e00700 1 -- 172.21.15.60:0/1894468133 learned_addr learned my addr 172.21.15.60:0/1894468133 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:35:08.792 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:08.022+0000 7f4fd1e00700 1 -- 172.21.15.60:0/1894468133 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fd4109990 con 0x7f4fd4109bd0 2024-05-03T14:35:08.792 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:08.022+0000 7f4fd1e00700 1 --2- 172.21.15.60:0/1894468133 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fd4109bd0 0x7f4fd4109450 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4011bd0 tx=0x7f4fc4011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:35:08.793 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:08.022+0000 7f4fcaa00700 1 -- 172.21.15.60:0/1894468133 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4fc4003970 con 0x7f4fd4109bd0 2024-05-03T14:35:08.793 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:08.023+0000 7f4fcaa00700 1 -- 172.21.15.60:0/1894468133 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1139+0+0 (secure 0 0 0) 0x7f4fc4003b10 con 0x7f4fd4109bd0 2024-05-03T14:35:08.793 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:08.023+0000 7f4fcaa00700 1 -- 172.21.15.60:0/1894468133 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4fc40245a0 con 0x7f4fd4109bd0 2024-05-03T14:35:08.793 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:08.023+0000 7f4fd8a00700 1 -- 172.21.15.60:0/1894468133 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fd4107b80 con 0x7f4fd4109bd0 2024-05-03T14:35:08.793 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:08.023+0000 7f4fd8a00700 1 -- 172.21.15.60:0/1894468133 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fd41080c0 con 0x7f4fd4109bd0 2024-05-03T14:35:08.793 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:08.023+0000 7f4fcaa00700 1 -- 172.21.15.60:0/1894468133 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4fc4024c30 con 0x7f4fd4109bd0 2024-05-03T14:35:08.793 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:08.023+0000 7f4fcaa00700 1 -- 172.21.15.60:0/1894468133 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4fc4019700 con 0x7f4fd4109bd0 2024-05-03T14:35:08.793 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:08.024+0000 7f4fd8a00700 1 -- 172.21.15.60:0/1894468133 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fd4064b00 con 0x7f4fd4109bd0 2024-05-03T14:35:08.793 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:08.027+0000 7f4fcaa00700 1 -- 172.21.15.60:0/1894468133 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f4fc4043e50 con 0x7f4fd4109bd0 2024-05-03T14:35:08.793 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:08.106+0000 7f4fd8a00700 1 -- 172.21.15.60:0/1894468133 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f4fd4108400 con 0x7f4fd4109bd0 2024-05-03T14:35:08.793 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:08.107+0000 7f4fcaa00700 1 -- 172.21.15.60:0/1894468133 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f4fc4032030 con 0x7f4fd4109bd0 2024-05-03T14:35:08.793 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:08.108+0000 7f4fd8a00700 1 -- 172.21.15.60:0/1894468133 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fd4109bd0 msgr2=0x7f4fd4109450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:35:08.793 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:08.108+0000 7f4fd8a00700 1 --2- 172.21.15.60:0/1894468133 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fd4109bd0 0x7f4fd4109450 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4011bd0 tx=0x7f4fc4011cb0 comp rx=0 tx=0).stop 2024-05-03T14:35:08.793 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:08.108+0000 7f4fd8a00700 1 -- 172.21.15.60:0/1894468133 shutdown_connections 2024-05-03T14:35:08.794 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:08.108+0000 7f4fd8a00700 1 --2- 172.21.15.60:0/1894468133 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fd4109bd0 0x7f4fd4109450 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:35:08.794 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:08.108+0000 7f4fd8a00700 1 -- 172.21.15.60:0/1894468133 >> 172.21.15.60:0/1894468133 conn(0x7f4fd41035c0 msgr2=0x7f4fd41040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:35:08.794 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:08.108+0000 7f4fd8a00700 1 -- 172.21.15.60:0/1894468133 shutdown_connections 2024-05-03T14:35:08.794 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:08.109+0000 7f4fd8a00700 1 -- 172.21.15.60:0/1894468133 wait complete. 2024-05-03T14:35:08.794 INFO:teuthology.orchestra.run.smithi060.stdout:mgr not available, waiting (2/15)... 2024-05-03T14:35:09.089 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:08 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:08.987+0000 7f215aa00200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-03T14:35:09.587 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:09 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:09.284+0000 7f215aa00200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-03T14:35:10.338 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:09 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:09.854+0000 7f215aa00200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-03T14:35:10.338 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:09 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:09.986+0000 7f215aa00200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-03T14:35:10.837 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:10 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:10.460+0000 7f215aa00200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-03T14:35:11.196 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:10 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:10.931+0000 7f215aa00200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-03T14:35:11.196 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:11 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:11.059+0000 7f215aa00200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-03T14:35:11.452 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:11 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:11.398+0000 7f215aa00200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-03T14:35:11.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:11 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/588115161' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-03T14:35:11.838 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:11 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:11.585+0000 7f215aa00200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-03T14:35:12.272 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:11 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:11.976+0000 7f215aa00200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-03T14:35:12.272 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:12 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:12.142+0000 7f215aa00200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-03T14:35:12.272 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:12 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:12.268+0000 7f215aa00200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-03T14:35:12.384 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-05-03T14:35:12.384 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout { 2024-05-03T14:35:12.384 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "fsid": "285d64b2-095a-11ef-bc97-c7b262605968", 2024-05-03T14:35:12.384 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "health": { 2024-05-03T14:35:12.385 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-03T14:35:12.385 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-03T14:35:12.385 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-03T14:35:12.385 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-05-03T14:35:12.385 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-03T14:35:12.385 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-03T14:35:12.385 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 0 2024-05-03T14:35:12.385 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-05-03T14:35:12.385 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-03T14:35:12.385 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "a" 2024-05-03T14:35:12.385 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-05-03T14:35:12.385 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-05-03T14:35:12.385 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-03T14:35:12.385 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T14:35:12.386 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-03T14:35:12.386 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-03T14:35:12.386 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-05-03T14:35:12.386 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-03T14:35:12.386 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T14:35:12.386 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-03T14:35:12.386 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-03T14:35:12.394 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-03T14:35:12.394 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-03T14:35:12.394 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-03T14:35:12.394 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-03T14:35:12.394 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-05-03T14:35:12.394 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-03T14:35:12.394 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-03T14:35:12.394 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-03T14:35:12.394 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-03T14:35:12.394 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-03T14:35:12.395 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-03T14:35:12.395 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-03T14:35:12.395 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-03T14:35:12.395 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-03T14:35:12.395 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-05-03T14:35:12.395 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-03T14:35:12.395 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T14:35:12.395 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-03T14:35:12.395 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-03T14:35:12.395 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-05-03T14:35:12.395 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-03T14:35:12.395 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-03T14:35:12.395 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-03T14:35:12.395 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-03T14:35:12.396 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-03T14:35:12.396 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-03T14:35:12.396 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "restful" 2024-05-03T14:35:12.396 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-05-03T14:35:12.396 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-03T14:35:12.396 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-05-03T14:35:12.396 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-03T14:35:12.396 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T14:35:12.396 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "modified": "2024-05-03T14:34:53.561346+0000", 2024-05-03T14:35:12.396 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-03T14:35:12.396 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-05-03T14:35:12.396 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-03T14:35:12.396 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout } 2024-05-03T14:35:12.396 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:11.343+0000 7fd643a00700 1 Processor -- start 2024-05-03T14:35:12.396 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:11.344+0000 7fd643a00700 1 -- start start 2024-05-03T14:35:12.396 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:11.344+0000 7fd643a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd63c1090d0 0x7fd63c1094f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:35:12.397 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:11.344+0000 7fd643a00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd63c109a30 con 0x7fd63c1090d0 2024-05-03T14:35:12.397 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:11.345+0000 7fd641000700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd63c1090d0 0x7fd63c1094f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:35:12.397 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:11.345+0000 7fd641000700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd63c1090d0 0x7fd63c1094f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45004/0 (socket says 172.21.15.60:45004) 2024-05-03T14:35:12.397 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:11.345+0000 7fd641000700 1 -- 172.21.15.60:0/3763195339 learned_addr learned my addr 172.21.15.60:0/3763195339 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:35:12.397 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:11.345+0000 7fd641000700 1 -- 172.21.15.60:0/3763195339 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd63c109bb0 con 0x7fd63c1090d0 2024-05-03T14:35:12.397 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:11.345+0000 7fd641000700 1 --2- 172.21.15.60:0/3763195339 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd63c1090d0 0x7fd63c1094f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd6300098a0 tx=0x7fd630009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=281536f1714022b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:35:12.397 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:11.346+0000 7fd63b400700 1 -- 172.21.15.60:0/3763195339 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd630004160 con 0x7fd63c1090d0 2024-05-03T14:35:12.397 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:11.346+0000 7fd63b400700 1 -- 172.21.15.60:0/3763195339 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1139+0+0 (secure 0 0 0) 0x7fd630004300 con 0x7fd63c1090d0 2024-05-03T14:35:12.397 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:11.346+0000 7fd63b400700 1 -- 172.21.15.60:0/3763195339 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd630011c00 con 0x7fd63c1090d0 2024-05-03T14:35:12.397 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:11.346+0000 7fd643a00700 1 -- 172.21.15.60:0/3763195339 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd63c1090d0 msgr2=0x7fd63c1094f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:35:12.397 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:11.346+0000 7fd643a00700 1 --2- 172.21.15.60:0/3763195339 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd63c1090d0 0x7fd63c1094f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd6300098a0 tx=0x7fd630009bb0 comp rx=0 tx=0).stop 2024-05-03T14:35:12.397 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:11.346+0000 7fd643a00700 1 -- 172.21.15.60:0/3763195339 shutdown_connections 2024-05-03T14:35:12.397 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:11.346+0000 7fd643a00700 1 --2- 172.21.15.60:0/3763195339 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd63c1090d0 0x7fd63c1094f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:35:12.397 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:11.346+0000 7fd643a00700 1 -- 172.21.15.60:0/3763195339 >> 172.21.15.60:0/3763195339 conn(0x7fd63c104750 msgr2=0x7fd63c106b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:35:12.397 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:11.347+0000 7fd643a00700 1 -- 172.21.15.60:0/3763195339 shutdown_connections 2024-05-03T14:35:12.398 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:11.347+0000 7fd643a00700 1 -- 172.21.15.60:0/3763195339 wait complete. 2024-05-03T14:35:12.398 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:11.347+0000 7fd643a00700 1 Processor -- start 2024-05-03T14:35:12.398 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:11.348+0000 7fd643a00700 1 -- start start 2024-05-03T14:35:12.398 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:11.348+0000 7fd643a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd63c1090d0 0x7fd63c19d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:35:12.398 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:11.348+0000 7fd643a00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd63c10d540 con 0x7fd63c1090d0 2024-05-03T14:35:12.398 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:11.348+0000 7fd641000700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd63c1090d0 0x7fd63c19d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:35:12.398 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:11.348+0000 7fd641000700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd63c1090d0 0x7fd63c19d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45018/0 (socket says 172.21.15.60:45018) 2024-05-03T14:35:12.398 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:11.348+0000 7fd641000700 1 -- 172.21.15.60:0/588115161 learned_addr learned my addr 172.21.15.60:0/588115161 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:35:12.398 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:11.349+0000 7fd641000700 1 -- 172.21.15.60:0/588115161 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd63c19d5b0 con 0x7fd63c1090d0 2024-05-03T14:35:12.398 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:11.349+0000 7fd641000700 1 --2- 172.21.15.60:0/588115161 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd63c1090d0 0x7fd63c19d070 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd630011bd0 tx=0x7fd630011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:35:12.398 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:11.349+0000 7fd639600700 1 -- 172.21.15.60:0/588115161 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd630003b20 con 0x7fd63c1090d0 2024-05-03T14:35:12.398 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:11.349+0000 7fd639600700 1 -- 172.21.15.60:0/588115161 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1139+0+0 (secure 0 0 0) 0x7fd630003cc0 con 0x7fd63c1090d0 2024-05-03T14:35:12.398 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:11.349+0000 7fd639600700 1 -- 172.21.15.60:0/588115161 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd630024930 con 0x7fd63c1090d0 2024-05-03T14:35:12.399 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:11.349+0000 7fd643a00700 1 -- 172.21.15.60:0/588115161 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd63c19d840 con 0x7fd63c1090d0 2024-05-03T14:35:12.399 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:11.349+0000 7fd643a00700 1 -- 172.21.15.60:0/588115161 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd63c1a0530 con 0x7fd63c1090d0 2024-05-03T14:35:12.399 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:11.350+0000 7fd639600700 1 -- 172.21.15.60:0/588115161 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd630022020 con 0x7fd63c1090d0 2024-05-03T14:35:12.399 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:11.350+0000 7fd639600700 1 -- 172.21.15.60:0/588115161 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd6300193c0 con 0x7fd63c1090d0 2024-05-03T14:35:12.399 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:11.350+0000 7fd643a00700 1 -- 172.21.15.60:0/588115161 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd63c10d6c0 con 0x7fd63c1090d0 2024-05-03T14:35:12.399 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:11.353+0000 7fd639600700 1 -- 172.21.15.60:0/588115161 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fd63001aca0 con 0x7fd63c1090d0 2024-05-03T14:35:12.399 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:11.432+0000 7fd643a00700 1 -- 172.21.15.60:0/588115161 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fd63c1a0a60 con 0x7fd63c1090d0 2024-05-03T14:35:12.399 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:11.433+0000 7fd639600700 1 -- 172.21.15.60:0/588115161 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7fd63002d3f0 con 0x7fd63c1090d0 2024-05-03T14:35:12.399 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:11.434+0000 7fd643a00700 1 -- 172.21.15.60:0/588115161 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd63c1090d0 msgr2=0x7fd63c19d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:35:12.399 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:11.434+0000 7fd643a00700 1 --2- 172.21.15.60:0/588115161 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd63c1090d0 0x7fd63c19d070 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd630011bd0 tx=0x7fd630011cb0 comp rx=0 tx=0).stop 2024-05-03T14:35:12.399 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:11.434+0000 7fd643a00700 1 -- 172.21.15.60:0/588115161 shutdown_connections 2024-05-03T14:35:12.399 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:11.434+0000 7fd643a00700 1 --2- 172.21.15.60:0/588115161 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd63c1090d0 0x7fd63c19d070 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:35:12.399 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:11.434+0000 7fd643a00700 1 -- 172.21.15.60:0/588115161 >> 172.21.15.60:0/588115161 conn(0x7fd63c104750 msgr2=0x7fd63c105290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:35:12.399 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:11.435+0000 7fd643a00700 1 -- 172.21.15.60:0/588115161 shutdown_connections 2024-05-03T14:35:12.400 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:11.435+0000 7fd643a00700 1 -- 172.21.15.60:0/588115161 wait complete. 2024-05-03T14:35:12.400 INFO:teuthology.orchestra.run.smithi060.stdout:mgr not available, waiting (3/15)... 2024-05-03T14:35:13.337 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:13 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:13.053+0000 7f215aa00200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-03T14:35:13.838 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:13 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:13.356+0000 7f215aa00200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-03T14:35:13.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:13 smithi060 ceph-mon[28873]: Activating manager daemon x 2024-05-03T14:35:13.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:13 smithi060 ceph-mon[28873]: mgrmap e2: x(active, starting, since 0.0127245s) 2024-05-03T14:35:13.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:13 smithi060 ceph-mon[28873]: from='mgr.14100 172.21.15.60:0/2268593183' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-03T14:35:13.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:13 smithi060 ceph-mon[28873]: from='mgr.14100 172.21.15.60:0/2268593183' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-03T14:35:13.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:13 smithi060 ceph-mon[28873]: from='mgr.14100 172.21.15.60:0/2268593183' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-03T14:35:13.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:13 smithi060 ceph-mon[28873]: from='mgr.14100 172.21.15.60:0/2268593183' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T14:35:13.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:13 smithi060 ceph-mon[28873]: from='mgr.14100 172.21.15.60:0/2268593183' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-03T14:35:13.839 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:13 smithi060 ceph-mon[28873]: Manager daemon x is now available 2024-05-03T14:35:13.839 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:13 smithi060 ceph-mon[28873]: from='mgr.14100 172.21.15.60:0/2268593183' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-03T14:35:13.839 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:13 smithi060 ceph-mon[28873]: from='mgr.14100 172.21.15.60:0/2268593183' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-03T14:35:14.699 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:14 smithi060 ceph-mon[28873]: from='mgr.14100 172.21.15.60:0/2268593183' entity='mgr.x' 2024-05-03T14:35:14.699 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:14 smithi060 ceph-mon[28873]: from='mgr.14100 172.21.15.60:0/2268593183' entity='mgr.x' 2024-05-03T14:35:14.699 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:14 smithi060 ceph-mon[28873]: from='mgr.14100 172.21.15.60:0/2268593183' entity='mgr.x' 2024-05-03T14:35:14.699 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:14 smithi060 ceph-mon[28873]: mgrmap e3: x(active, since 1.02353s) 2024-05-03T14:35:15.716 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:15 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/3188317559' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-03T14:35:16.017 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-05-03T14:35:16.017 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout { 2024-05-03T14:35:16.018 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "fsid": "285d64b2-095a-11ef-bc97-c7b262605968", 2024-05-03T14:35:16.018 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "health": { 2024-05-03T14:35:16.018 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-03T14:35:16.018 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-03T14:35:16.019 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-03T14:35:16.019 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-05-03T14:35:16.019 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-03T14:35:16.019 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-03T14:35:16.020 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 0 2024-05-03T14:35:16.020 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-05-03T14:35:16.020 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-03T14:35:16.020 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "a" 2024-05-03T14:35:16.020 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-05-03T14:35:16.020 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-05-03T14:35:16.021 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-03T14:35:16.021 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T14:35:16.021 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-03T14:35:16.021 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-03T14:35:16.021 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-05-03T14:35:16.022 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-03T14:35:16.022 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T14:35:16.023 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-03T14:35:16.023 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-03T14:35:16.023 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-03T14:35:16.023 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-03T14:35:16.023 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-03T14:35:16.023 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-03T14:35:16.023 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-05-03T14:35:16.023 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-03T14:35:16.025 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-03T14:35:16.025 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-03T14:35:16.025 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-03T14:35:16.025 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-03T14:35:16.025 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-03T14:35:16.026 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-03T14:35:16.026 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-03T14:35:16.026 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-03T14:35:16.026 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-05-03T14:35:16.026 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-03T14:35:16.026 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T14:35:16.026 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-03T14:35:16.026 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-03T14:35:16.026 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-05-03T14:35:16.026 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-03T14:35:16.026 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-03T14:35:16.026 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-03T14:35:16.026 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-03T14:35:16.027 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-03T14:35:16.027 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-03T14:35:16.029 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "restful" 2024-05-03T14:35:16.029 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-05-03T14:35:16.029 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-03T14:35:16.029 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-05-03T14:35:16.030 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-03T14:35:16.030 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T14:35:16.030 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "modified": "2024-05-03T14:34:53.561346+0000", 2024-05-03T14:35:16.030 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-03T14:35:16.030 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-05-03T14:35:16.030 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-03T14:35:16.030 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout } 2024-05-03T14:35:16.030 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:14.991+0000 7f82d1800700 1 Processor -- start 2024-05-03T14:35:16.030 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:14.992+0000 7f82d1800700 1 -- start start 2024-05-03T14:35:16.030 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:14.992+0000 7f82d1800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82cc07edb0 0x7f82cc07d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:35:16.030 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:14.992+0000 7f82d1800700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f82cc07d860 con 0x7f82cc07edb0 2024-05-03T14:35:16.030 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:14.993+0000 7f82caa00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82cc07edb0 0x7f82cc07d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:35:16.030 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:14.993+0000 7f82caa00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82cc07edb0 0x7f82cc07d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48866/0 (socket says 172.21.15.60:48866) 2024-05-03T14:35:16.031 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:14.993+0000 7f82caa00700 1 -- 172.21.15.60:0/1686620628 learned_addr learned my addr 172.21.15.60:0/1686620628 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:35:16.031 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:14.993+0000 7f82caa00700 1 -- 172.21.15.60:0/1686620628 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82cc07d9e0 con 0x7f82cc07edb0 2024-05-03T14:35:16.031 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:14.994+0000 7f82caa00700 1 --2- 172.21.15.60:0/1686620628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82cc07edb0 0x7f82cc07d320 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f82bc009e80 tx=0x7f82bc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ccc017b37bd0c94 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:35:16.031 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:14.994+0000 7f82ca000700 1 -- 172.21.15.60:0/1686620628 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82bc017070 con 0x7f82cc07edb0 2024-05-03T14:35:16.031 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:14.994+0000 7f82ca000700 1 -- 172.21.15.60:0/1686620628 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1139+0+0 (secure 0 0 0) 0x7f82bc00b8c0 con 0x7f82cc07edb0 2024-05-03T14:35:16.031 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:14.994+0000 7f82ca000700 1 -- 172.21.15.60:0/1686620628 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82bc004620 con 0x7f82cc07edb0 2024-05-03T14:35:16.031 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:14.995+0000 7f82d1800700 1 -- 172.21.15.60:0/1686620628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82cc07edb0 msgr2=0x7f82cc07d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:35:16.033 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:14.995+0000 7f82d1800700 1 --2- 172.21.15.60:0/1686620628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82cc07edb0 0x7f82cc07d320 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f82bc009e80 tx=0x7f82bc00b330 comp rx=0 tx=0).stop 2024-05-03T14:35:16.033 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:14.995+0000 7f82d1800700 1 -- 172.21.15.60:0/1686620628 shutdown_connections 2024-05-03T14:35:16.033 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:14.995+0000 7f82d1800700 1 --2- 172.21.15.60:0/1686620628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82cc07edb0 0x7f82cc07d320 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:35:16.033 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:14.995+0000 7f82d1800700 1 -- 172.21.15.60:0/1686620628 >> 172.21.15.60:0/1686620628 conn(0x7f82cc104750 msgr2=0x7f82cc106b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:35:16.033 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:14.995+0000 7f82d1800700 1 -- 172.21.15.60:0/1686620628 shutdown_connections 2024-05-03T14:35:16.033 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:14.995+0000 7f82d1800700 1 -- 172.21.15.60:0/1686620628 wait complete. 2024-05-03T14:35:16.034 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:14.996+0000 7f82d1800700 1 Processor -- start 2024-05-03T14:35:16.034 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:14.996+0000 7f82d1800700 1 -- start start 2024-05-03T14:35:16.034 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:14.997+0000 7f82d1800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82cc07edb0 0x7f82cc19f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:35:16.034 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:14.997+0000 7f82d1800700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f82cc10f600 con 0x7f82cc07edb0 2024-05-03T14:35:16.034 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:14.997+0000 7f82caa00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82cc07edb0 0x7f82cc19f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:35:16.034 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:14.997+0000 7f82caa00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82cc07edb0 0x7f82cc19f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48882/0 (socket says 172.21.15.60:48882) 2024-05-03T14:35:16.034 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:14.997+0000 7f82caa00700 1 -- 172.21.15.60:0/3188317559 learned_addr learned my addr 172.21.15.60:0/3188317559 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:35:16.034 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:14.997+0000 7f82caa00700 1 -- 172.21.15.60:0/3188317559 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82cc19f750 con 0x7f82cc07edb0 2024-05-03T14:35:16.034 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:14.998+0000 7f82caa00700 1 --2- 172.21.15.60:0/3188317559 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82cc07edb0 0x7f82cc19f210 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f82bc00b7b0 tx=0x7f82bc003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:35:16.034 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:14.998+0000 7f82c3e00700 1 -- 172.21.15.60:0/3188317559 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82bc021020 con 0x7f82cc07edb0 2024-05-03T14:35:16.034 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:14.998+0000 7f82c3e00700 1 -- 172.21.15.60:0/3188317559 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1139+0+0 (secure 0 0 0) 0x7f82bc00be00 con 0x7f82cc07edb0 2024-05-03T14:35:16.034 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:14.998+0000 7f82c3e00700 1 -- 172.21.15.60:0/3188317559 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82bc017040 con 0x7f82cc07edb0 2024-05-03T14:35:16.034 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:14.998+0000 7f82d1800700 1 -- 172.21.15.60:0/3188317559 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82cc19f9e0 con 0x7f82cc07edb0 2024-05-03T14:35:16.034 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:14.998+0000 7f82d1800700 1 -- 172.21.15.60:0/3188317559 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82cc1a26d0 con 0x7f82cc07edb0 2024-05-03T14:35:16.035 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:14.999+0000 7f82c3e00700 1 -- 172.21.15.60:0/3188317559 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 3) ==== 50105+0+0 (secure 0 0 0) 0x7f82bc024020 con 0x7f82cc07edb0 2024-05-03T14:35:16.035 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:14.999+0000 7f82d1800700 1 -- 172.21.15.60:0/3188317559 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82cc064b00 con 0x7f82cc07edb0 2024-05-03T14:35:16.036 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:15.000+0000 7f82c3e00700 1 --2- 172.21.15.60:0/3188317559 >> [v2:172.21.15.60:6800/888447185,v1:172.21.15.60:6801/888447185] conn(0x7f82b003e350 0x7f82b0040810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:35:16.036 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:15.000+0000 7f82c3e00700 1 -- 172.21.15.60:0/3188317559 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f82bc00f050 con 0x7f82cc07edb0 2024-05-03T14:35:16.037 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:15.000+0000 7f82c2000700 1 --2- 172.21.15.60:0/3188317559 >> [v2:172.21.15.60:6800/888447185,v1:172.21.15.60:6801/888447185] conn(0x7f82b003e350 0x7f82b0040810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:35:16.037 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:15.001+0000 7f82c2000700 1 --2- 172.21.15.60:0/3188317559 >> [v2:172.21.15.60:6800/888447185,v1:172.21.15.60:6801/888447185] conn(0x7f82b003e350 0x7f82b0040810 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f82b4006fd0 tx=0x7f82b4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:35:16.037 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:15.006+0000 7f82c3e00700 1 -- 172.21.15.60:0/3188317559 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f82bc02b020 con 0x7f82cc07edb0 2024-05-03T14:35:16.037 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:15.344+0000 7f82d1800700 1 -- 172.21.15.60:0/3188317559 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f82cc1a28f0 con 0x7f82cc07edb0 2024-05-03T14:35:16.037 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:15.345+0000 7f82c3e00700 1 -- 172.21.15.60:0/3188317559 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f82bc01be50 con 0x7f82cc07edb0 2024-05-03T14:35:16.037 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:15.349+0000 7f82d1800700 1 -- 172.21.15.60:0/3188317559 >> [v2:172.21.15.60:6800/888447185,v1:172.21.15.60:6801/888447185] conn(0x7f82b003e350 msgr2=0x7f82b0040810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:35:16.037 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:15.349+0000 7f82d1800700 1 --2- 172.21.15.60:0/3188317559 >> [v2:172.21.15.60:6800/888447185,v1:172.21.15.60:6801/888447185] conn(0x7f82b003e350 0x7f82b0040810 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f82b4006fd0 tx=0x7f82b4006e40 comp rx=0 tx=0).stop 2024-05-03T14:35:16.037 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:15.349+0000 7f82d1800700 1 -- 172.21.15.60:0/3188317559 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82cc07edb0 msgr2=0x7f82cc19f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:35:16.037 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:15.349+0000 7f82d1800700 1 --2- 172.21.15.60:0/3188317559 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82cc07edb0 0x7f82cc19f210 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f82bc00b7b0 tx=0x7f82bc003d00 comp rx=0 tx=0).stop 2024-05-03T14:35:16.037 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:15.349+0000 7f82d1800700 1 -- 172.21.15.60:0/3188317559 shutdown_connections 2024-05-03T14:35:16.037 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:15.349+0000 7f82d1800700 1 --2- 172.21.15.60:0/3188317559 >> [v2:172.21.15.60:6800/888447185,v1:172.21.15.60:6801/888447185] conn(0x7f82b003e350 0x7f82b0040810 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:35:16.037 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:15.349+0000 7f82d1800700 1 --2- 172.21.15.60:0/3188317559 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82cc07edb0 0x7f82cc19f210 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:35:16.037 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:15.349+0000 7f82d1800700 1 -- 172.21.15.60:0/3188317559 >> 172.21.15.60:0/3188317559 conn(0x7f82cc104750 msgr2=0x7f82cc1051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:35:16.037 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:15.350+0000 7f82d1800700 1 -- 172.21.15.60:0/3188317559 shutdown_connections 2024-05-03T14:35:16.037 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:15.350+0000 7f82d1800700 1 -- 172.21.15.60:0/3188317559 wait complete. 2024-05-03T14:35:16.038 INFO:teuthology.orchestra.run.smithi060.stdout:mgr is available 2024-05-03T14:35:16.630 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:16 smithi060 ceph-mon[28873]: mgrmap e4: x(active, since 2s) 2024-05-03T14:35:17.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:17 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/2980327214' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-03T14:35:17.599 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-05-03T14:35:17.600 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout [global] 2024-05-03T14:35:17.600 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout fsid = 285d64b2-095a-11ef-bc97-c7b262605968 2024-05-03T14:35:17.600 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-03T14:35:17.601 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-03T14:35:17.601 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-03T14:35:17.601 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-03T14:35:17.601 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-03T14:35:17.603 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-05-03T14:35:17.603 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-03T14:35:17.603 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-03T14:35:17.604 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-05-03T14:35:17.604 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout [osd] 2024-05-03T14:35:17.604 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-03T14:35:17.604 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-03T14:35:17.604 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-03T14:35:17.604 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-03T14:35:17.604 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-03T14:35:17.604 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:16.699+0000 7fcfbc800700 1 Processor -- start 2024-05-03T14:35:17.604 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:16.699+0000 7fcfbc800700 1 -- start start 2024-05-03T14:35:17.604 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:16.700+0000 7fcfbc800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcfb4108000 0x7fcfb4108420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:35:17.604 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:16.700+0000 7fcfbc800700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb41089f0 con 0x7fcfb4108000 2024-05-03T14:35:17.604 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:16.700+0000 7fcfb9e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcfb4108000 0x7fcfb4108420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:35:17.604 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:16.700+0000 7fcfb9e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcfb4108000 0x7fcfb4108420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48898/0 (socket says 172.21.15.60:48898) 2024-05-03T14:35:17.604 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:16.700+0000 7fcfb9e00700 1 -- 172.21.15.60:0/213333370 learned_addr learned my addr 172.21.15.60:0/213333370 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:35:17.605 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:16.701+0000 7fcfb9e00700 1 -- 172.21.15.60:0/213333370 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfb4109200 con 0x7fcfb4108000 2024-05-03T14:35:17.605 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:16.701+0000 7fcfb9e00700 1 --2- 172.21.15.60:0/213333370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcfb4108000 0x7fcfb4108420 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcfa80098a0 tx=0x7fcfa8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e445bfbead4ba4da server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:35:17.605 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:16.701+0000 7fcfb8a00700 1 -- 172.21.15.60:0/213333370 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcfa8004160 con 0x7fcfb4108000 2024-05-03T14:35:17.605 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:16.701+0000 7fcfb8a00700 1 -- 172.21.15.60:0/213333370 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1139+0+0 (secure 0 0 0) 0x7fcfa8004300 con 0x7fcfb4108000 2024-05-03T14:35:17.605 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:16.701+0000 7fcfb8a00700 1 -- 172.21.15.60:0/213333370 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcfa8011c00 con 0x7fcfb4108000 2024-05-03T14:35:17.609 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:16.702+0000 7fcfbc800700 1 -- 172.21.15.60:0/213333370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcfb4108000 msgr2=0x7fcfb4108420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:35:17.609 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:16.702+0000 7fcfbc800700 1 --2- 172.21.15.60:0/213333370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcfb4108000 0x7fcfb4108420 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcfa80098a0 tx=0x7fcfa8009bb0 comp rx=0 tx=0).stop 2024-05-03T14:35:17.609 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:16.702+0000 7fcfbc800700 1 -- 172.21.15.60:0/213333370 shutdown_connections 2024-05-03T14:35:17.609 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:16.702+0000 7fcfbc800700 1 --2- 172.21.15.60:0/213333370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcfb4108000 0x7fcfb4108420 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:35:17.609 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:16.702+0000 7fcfbc800700 1 -- 172.21.15.60:0/213333370 >> 172.21.15.60:0/213333370 conn(0x7fcfb41035c0 msgr2=0x7fcfb41059e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:35:17.609 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:16.702+0000 7fcfbc800700 1 -- 172.21.15.60:0/213333370 shutdown_connections 2024-05-03T14:35:17.609 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:16.702+0000 7fcfbc800700 1 -- 172.21.15.60:0/213333370 wait complete. 2024-05-03T14:35:17.609 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:16.704+0000 7fcfbc800700 1 Processor -- start 2024-05-03T14:35:17.609 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:16.704+0000 7fcfbc800700 1 -- start start 2024-05-03T14:35:17.609 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:16.705+0000 7fcfbc800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcfb4108000 0x7fcfb419d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:35:17.610 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:16.705+0000 7fcfbc800700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb410c5b0 con 0x7fcfb4108000 2024-05-03T14:35:17.610 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:16.705+0000 7fcfb9e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcfb4108000 0x7fcfb419d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:35:17.610 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:16.705+0000 7fcfb9e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcfb4108000 0x7fcfb419d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48910/0 (socket says 172.21.15.60:48910) 2024-05-03T14:35:17.610 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:16.705+0000 7fcfb9e00700 1 -- 172.21.15.60:0/2980327214 learned_addr learned my addr 172.21.15.60:0/2980327214 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:35:17.610 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:16.705+0000 7fcfb9e00700 1 -- 172.21.15.60:0/2980327214 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfb419d6a0 con 0x7fcfb4108000 2024-05-03T14:35:17.610 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:16.706+0000 7fcfb9e00700 1 --2- 172.21.15.60:0/2980327214 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcfb4108000 0x7fcfb419d160 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fcfa80118c0 tx=0x7fcfa8011c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:35:17.610 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:16.706+0000 7fcfaea00700 1 -- 172.21.15.60:0/2980327214 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcfa8003af0 con 0x7fcfb4108000 2024-05-03T14:35:17.610 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:16.706+0000 7fcfaea00700 1 -- 172.21.15.60:0/2980327214 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1139+0+0 (secure 0 0 0) 0x7fcfa8020070 con 0x7fcfb4108000 2024-05-03T14:35:17.610 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:16.706+0000 7fcfaea00700 1 -- 172.21.15.60:0/2980327214 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcfa80195e0 con 0x7fcfb4108000 2024-05-03T14:35:17.610 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:16.706+0000 7fcfbc800700 1 -- 172.21.15.60:0/2980327214 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfb419d930 con 0x7fcfb4108000 2024-05-03T14:35:17.610 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:16.706+0000 7fcfbc800700 1 -- 172.21.15.60:0/2980327214 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfb41a0620 con 0x7fcfb4108000 2024-05-03T14:35:17.610 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:16.707+0000 7fcfbc800700 1 -- 172.21.15.60:0/2980327214 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfb4064b00 con 0x7fcfb4108000 2024-05-03T14:35:17.610 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:16.708+0000 7fcfaea00700 1 -- 172.21.15.60:0/2980327214 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7fcfa801a9a0 con 0x7fcfb4108000 2024-05-03T14:35:17.610 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:16.709+0000 7fcfaea00700 1 --2- 172.21.15.60:0/2980327214 >> [v2:172.21.15.60:6800/888447185,v1:172.21.15.60:6801/888447185] conn(0x7fcf9c03e420 0x7fcf9c0408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:35:17.610 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:16.709+0000 7fcfaea00700 1 -- 172.21.15.60:0/2980327214 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fcfa8053ec0 con 0x7fcfb4108000 2024-05-03T14:35:17.611 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:16.709+0000 7fcfb9400700 1 --2- 172.21.15.60:0/2980327214 >> [v2:172.21.15.60:6800/888447185,v1:172.21.15.60:6801/888447185] conn(0x7fcf9c03e420 0x7fcf9c0408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:35:17.611 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:16.710+0000 7fcfb9400700 1 --2- 172.21.15.60:0/2980327214 >> [v2:172.21.15.60:6800/888447185,v1:172.21.15.60:6801/888447185] conn(0x7fcf9c03e420 0x7fcf9c0408e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4006fd0 tx=0x7fcfa4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:35:17.611 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:16.714+0000 7fcfaea00700 1 -- 172.21.15.60:0/2980327214 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fcfa8005a20 con 0x7fcfb4108000 2024-05-03T14:35:17.611 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:16.951+0000 7fcfbc800700 1 -- 172.21.15.60:0/2980327214 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fcfb40008d0 con 0x7fcfb4108000 2024-05-03T14:35:17.611 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:16.952+0000 7fcfaea00700 1 -- 172.21.15.60:0/2980327214 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+409 (secure 0 0 0) 0x7fcfa80538e0 con 0x7fcfb4108000 2024-05-03T14:35:17.611 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:16.956+0000 7fcfbc800700 1 -- 172.21.15.60:0/2980327214 >> [v2:172.21.15.60:6800/888447185,v1:172.21.15.60:6801/888447185] conn(0x7fcf9c03e420 msgr2=0x7fcf9c0408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:35:17.611 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:16.956+0000 7fcfbc800700 1 --2- 172.21.15.60:0/2980327214 >> [v2:172.21.15.60:6800/888447185,v1:172.21.15.60:6801/888447185] conn(0x7fcf9c03e420 0x7fcf9c0408e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4006fd0 tx=0x7fcfa4006e40 comp rx=0 tx=0).stop 2024-05-03T14:35:17.611 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:16.956+0000 7fcfbc800700 1 -- 172.21.15.60:0/2980327214 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcfb4108000 msgr2=0x7fcfb419d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:35:17.611 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:16.956+0000 7fcfbc800700 1 --2- 172.21.15.60:0/2980327214 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcfb4108000 0x7fcfb419d160 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fcfa80118c0 tx=0x7fcfa8011c80 comp rx=0 tx=0).stop 2024-05-03T14:35:17.611 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:16.956+0000 7fcfbc800700 1 -- 172.21.15.60:0/2980327214 shutdown_connections 2024-05-03T14:35:17.611 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:16.956+0000 7fcfbc800700 1 --2- 172.21.15.60:0/2980327214 >> [v2:172.21.15.60:6800/888447185,v1:172.21.15.60:6801/888447185] conn(0x7fcf9c03e420 0x7fcf9c0408e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:35:17.611 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:16.956+0000 7fcfbc800700 1 --2- 172.21.15.60:0/2980327214 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcfb4108000 0x7fcfb419d160 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:35:17.611 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:16.956+0000 7fcfbc800700 1 -- 172.21.15.60:0/2980327214 >> 172.21.15.60:0/2980327214 conn(0x7fcfb41035c0 msgr2=0x7fcfb41044f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:35:17.611 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:16.957+0000 7fcfbc800700 1 -- 172.21.15.60:0/2980327214 shutdown_connections 2024-05-03T14:35:17.611 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:16.957+0000 7fcfbc800700 1 -- 172.21.15.60:0/2980327214 wait complete. 2024-05-03T14:35:17.611 INFO:teuthology.orchestra.run.smithi060.stdout:Enabling cephadm module... 2024-05-03T14:35:18.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:18 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/3336885130' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-03T14:35:18.837 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:18 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: ignoring --setuser ceph since I am not root 2024-05-03T14:35:18.838 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:18 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: ignoring --setgroup ceph since I am not root 2024-05-03T14:35:19.265 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:18 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:18.879+0000 7f7daf03c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-03T14:35:19.265 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:19 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:19.127+0000 7f7daf03c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-03T14:35:19.299 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:18.211+0000 7f2da9800700 1 Processor -- start 2024-05-03T14:35:19.299 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:18.211+0000 7f2da9800700 1 -- start start 2024-05-03T14:35:19.299 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:18.212+0000 7f2da9800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2da407f880 0x7f2da407fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:35:19.299 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:18.212+0000 7f2da9800700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f2da4080270 con 0x7f2da407f880 2024-05-03T14:35:19.299 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:18.212+0000 7f2da2a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2da407f880 0x7f2da407fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:35:19.299 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:18.213+0000 7f2da2a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2da407f880 0x7f2da407fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48916/0 (socket says 172.21.15.60:48916) 2024-05-03T14:35:19.300 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:18.213+0000 7f2da2a00700 1 -- 172.21.15.60:0/2523971757 learned_addr learned my addr 172.21.15.60:0/2523971757 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:35:19.300 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:18.213+0000 7f2da2a00700 1 -- 172.21.15.60:0/2523971757 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2da4080a80 con 0x7f2da407f880 2024-05-03T14:35:19.300 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:18.213+0000 7f2da2a00700 1 --2- 172.21.15.60:0/2523971757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2da407f880 0x7f2da407fca0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2d940098a0 tx=0x7f2d94009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a59ae3062107e62 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:35:19.300 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:18.213+0000 7f2da1600700 1 -- 172.21.15.60:0/2523971757 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d94004160 con 0x7f2da407f880 2024-05-03T14:35:19.300 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:18.214+0000 7f2da1600700 1 -- 172.21.15.60:0/2523971757 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1139+0+0 (secure 0 0 0) 0x7f2d94004300 con 0x7f2da407f880 2024-05-03T14:35:19.300 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:18.214+0000 7f2da1600700 1 -- 172.21.15.60:0/2523971757 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d94011c00 con 0x7f2da407f880 2024-05-03T14:35:19.302 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:18.214+0000 7f2da9800700 1 -- 172.21.15.60:0/2523971757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2da407f880 msgr2=0x7f2da407fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:35:19.302 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:18.214+0000 7f2da9800700 1 --2- 172.21.15.60:0/2523971757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2da407f880 0x7f2da407fca0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2d940098a0 tx=0x7f2d94009bb0 comp rx=0 tx=0).stop 2024-05-03T14:35:19.302 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:18.214+0000 7f2da9800700 1 -- 172.21.15.60:0/2523971757 shutdown_connections 2024-05-03T14:35:19.302 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:18.214+0000 7f2da9800700 1 --2- 172.21.15.60:0/2523971757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2da407f880 0x7f2da407fca0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:35:19.302 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:18.214+0000 7f2da9800700 1 -- 172.21.15.60:0/2523971757 >> 172.21.15.60:0/2523971757 conn(0x7f2da407e900 msgr2=0x7f2da41094f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:35:19.302 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:18.215+0000 7f2da9800700 1 -- 172.21.15.60:0/2523971757 shutdown_connections 2024-05-03T14:35:19.302 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:18.215+0000 7f2da9800700 1 -- 172.21.15.60:0/2523971757 wait complete. 2024-05-03T14:35:19.302 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:18.216+0000 7f2da9800700 1 Processor -- start 2024-05-03T14:35:19.303 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:18.216+0000 7f2da9800700 1 -- start start 2024-05-03T14:35:19.303 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:18.216+0000 7f2da9800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2da407f880 0x7f2da41a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:35:19.303 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:18.216+0000 7f2da9800700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f2da4081e40 con 0x7f2da407f880 2024-05-03T14:35:19.303 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:18.217+0000 7f2da2a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2da407f880 0x7f2da41a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:35:19.303 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:18.217+0000 7f2da2a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2da407f880 0x7f2da41a3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48926/0 (socket says 172.21.15.60:48926) 2024-05-03T14:35:19.303 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:18.217+0000 7f2da2a00700 1 -- 172.21.15.60:0/3336885130 learned_addr learned my addr 172.21.15.60:0/3336885130 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:35:19.303 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:18.217+0000 7f2da2a00700 1 -- 172.21.15.60:0/3336885130 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2da41a3c50 con 0x7f2da407f880 2024-05-03T14:35:19.303 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:18.217+0000 7f2da2a00700 1 --2- 172.21.15.60:0/3336885130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2da407f880 0x7f2da41a3710 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2d94009fb0 tx=0x7f2d94011c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:35:19.303 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:18.218+0000 7f2d9b400700 1 -- 172.21.15.60:0/3336885130 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d940038b0 con 0x7f2da407f880 2024-05-03T14:35:19.303 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:18.218+0000 7f2d9b400700 1 -- 172.21.15.60:0/3336885130 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1139+0+0 (secure 0 0 0) 0x7f2d9401a430 con 0x7f2da407f880 2024-05-03T14:35:19.306 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:18.218+0000 7f2da9800700 1 -- 172.21.15.60:0/3336885130 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2da41a3ee0 con 0x7f2da407f880 2024-05-03T14:35:19.306 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:18.218+0000 7f2da9800700 1 -- 172.21.15.60:0/3336885130 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2da41a6bd0 con 0x7f2da407f880 2024-05-03T14:35:19.306 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:18.218+0000 7f2d9b400700 1 -- 172.21.15.60:0/3336885130 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d94019630 con 0x7f2da407f880 2024-05-03T14:35:19.306 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:18.219+0000 7f2d9b400700 1 -- 172.21.15.60:0/3336885130 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f2d9401aa10 con 0x7f2da407f880 2024-05-03T14:35:19.306 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:18.219+0000 7f2da9800700 1 -- 172.21.15.60:0/3336885130 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2da4064b00 con 0x7f2da407f880 2024-05-03T14:35:19.306 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:18.220+0000 7f2d9b400700 1 --2- 172.21.15.60:0/3336885130 >> [v2:172.21.15.60:6800/888447185,v1:172.21.15.60:6801/888447185] conn(0x7f2d8803e420 0x7f2d880408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:35:19.306 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:18.220+0000 7f2d9b400700 1 -- 172.21.15.60:0/3336885130 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2d94053920 con 0x7f2da407f880 2024-05-03T14:35:19.306 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:18.220+0000 7f2da2000700 1 --2- 172.21.15.60:0/3336885130 >> [v2:172.21.15.60:6800/888447185,v1:172.21.15.60:6801/888447185] conn(0x7f2d8803e420 0x7f2d880408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:35:19.306 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:18.221+0000 7f2da2000700 1 --2- 172.21.15.60:0/3336885130 >> [v2:172.21.15.60:6800/888447185,v1:172.21.15.60:6801/888447185] conn(0x7f2d8803e420 0x7f2d880408e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2d90006fd0 tx=0x7f2d90006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:35:19.306 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:18.225+0000 7f2d9b400700 1 -- 172.21.15.60:0/3336885130 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f2d94021e80 con 0x7f2da407f880 2024-05-03T14:35:19.306 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:18.524+0000 7f2da9800700 1 -- 172.21.15.60:0/3336885130 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f2da40008d0 con 0x7f2da407f880 2024-05-03T14:35:19.307 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:18.579+0000 7f2d9b400700 1 -- 172.21.15.60:0/3336885130 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f2d9400fda0 con 0x7f2da407f880 2024-05-03T14:35:19.307 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:18.583+0000 7f2d9b400700 1 -- 172.21.15.60:0/3336885130 <== mon.0 v2:172.21.15.60:3300/0 8 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f2d94003a50 con 0x7f2da407f880 2024-05-03T14:35:19.307 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:18.583+0000 7f2da9800700 1 -- 172.21.15.60:0/3336885130 >> [v2:172.21.15.60:6800/888447185,v1:172.21.15.60:6801/888447185] conn(0x7f2d8803e420 msgr2=0x7f2d880408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:35:19.307 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:18.583+0000 7f2da9800700 1 --2- 172.21.15.60:0/3336885130 >> [v2:172.21.15.60:6800/888447185,v1:172.21.15.60:6801/888447185] conn(0x7f2d8803e420 0x7f2d880408e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2d90006fd0 tx=0x7f2d90006e40 comp rx=0 tx=0).stop 2024-05-03T14:35:19.307 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:18.583+0000 7f2da9800700 1 -- 172.21.15.60:0/3336885130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2da407f880 msgr2=0x7f2da41a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:35:19.307 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:18.583+0000 7f2da9800700 1 --2- 172.21.15.60:0/3336885130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2da407f880 0x7f2da41a3710 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2d94009fb0 tx=0x7f2d94011c60 comp rx=0 tx=0).stop 2024-05-03T14:35:19.307 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:18.585+0000 7f2da9800700 1 -- 172.21.15.60:0/3336885130 shutdown_connections 2024-05-03T14:35:19.307 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:18.585+0000 7f2da9800700 1 --2- 172.21.15.60:0/3336885130 >> [v2:172.21.15.60:6800/888447185,v1:172.21.15.60:6801/888447185] conn(0x7f2d8803e420 0x7f2d880408e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:35:19.307 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:18.585+0000 7f2da9800700 1 --2- 172.21.15.60:0/3336885130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2da407f880 0x7f2da41a3710 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:35:19.307 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:18.585+0000 7f2da9800700 1 -- 172.21.15.60:0/3336885130 >> 172.21.15.60:0/3336885130 conn(0x7f2da407e900 msgr2=0x7f2da419a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:35:19.307 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:18.585+0000 7f2da9800700 1 -- 172.21.15.60:0/3336885130 shutdown_connections 2024-05-03T14:35:19.307 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:18.585+0000 7f2da9800700 1 -- 172.21.15.60:0/3336885130 wait complete. 2024-05-03T14:35:19.579 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:19 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:19.261+0000 7f7daf03c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-03T14:35:19.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:19 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/3336885130' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-03T14:35:19.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:19 smithi060 ceph-mon[28873]: mgrmap e5: x(active, since 5s) 2024-05-03T14:35:19.838 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:19 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:19.690+0000 7f7daf03c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-03T14:35:19.838 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:19 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:19.807+0000 7f7daf03c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-03T14:35:20.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:20 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/1277629965' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-03T14:35:20.871 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout { 2024-05-03T14:35:20.872 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-03T14:35:20.872 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-03T14:35:20.872 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-03T14:35:20.872 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-03T14:35:20.873 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout } 2024-05-03T14:35:20.873 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:19.865+0000 7f3596a00700 1 Processor -- start 2024-05-03T14:35:20.873 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:19.865+0000 7f3596a00700 1 -- start start 2024-05-03T14:35:20.873 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:19.866+0000 7f3596a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35901090d0 0x7f35901094f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:35:20.873 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:19.866+0000 7f3596a00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f3590109a30 con 0x7f35901090d0 2024-05-03T14:35:20.873 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:19.866+0000 7f358fe00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35901090d0 0x7f35901094f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:35:20.875 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:19.866+0000 7f358fe00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35901090d0 0x7f35901094f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48956/0 (socket says 172.21.15.60:48956) 2024-05-03T14:35:20.875 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:19.866+0000 7f358fe00700 1 -- 172.21.15.60:0/224255183 learned_addr learned my addr 172.21.15.60:0/224255183 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:35:20.876 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:19.867+0000 7f358fe00700 1 -- 172.21.15.60:0/224255183 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3590109bb0 con 0x7f35901090d0 2024-05-03T14:35:20.876 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:19.867+0000 7f358fe00700 1 --2- 172.21.15.60:0/224255183 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35901090d0 0x7f35901094f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f35800098a0 tx=0x7f3580009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80ab823475368e8e server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:35:20.876 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:19.867+0000 7f358ea00700 1 -- 172.21.15.60:0/224255183 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3580004160 con 0x7f35901090d0 2024-05-03T14:35:20.876 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:19.867+0000 7f358ea00700 1 -- 172.21.15.60:0/224255183 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1139+0+0 (secure 0 0 0) 0x7f3580004300 con 0x7f35901090d0 2024-05-03T14:35:20.876 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:19.867+0000 7f358ea00700 1 -- 172.21.15.60:0/224255183 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3580011c00 con 0x7f35901090d0 2024-05-03T14:35:20.876 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:19.868+0000 7f3596a00700 1 -- 172.21.15.60:0/224255183 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35901090d0 msgr2=0x7f35901094f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:35:20.876 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:19.868+0000 7f3596a00700 1 --2- 172.21.15.60:0/224255183 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35901090d0 0x7f35901094f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f35800098a0 tx=0x7f3580009bb0 comp rx=0 tx=0).stop 2024-05-03T14:35:20.876 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:19.868+0000 7f3596a00700 1 -- 172.21.15.60:0/224255183 shutdown_connections 2024-05-03T14:35:20.876 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:19.868+0000 7f3596a00700 1 --2- 172.21.15.60:0/224255183 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35901090d0 0x7f35901094f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:35:20.876 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:19.868+0000 7f3596a00700 1 -- 172.21.15.60:0/224255183 >> 172.21.15.60:0/224255183 conn(0x7f3590104750 msgr2=0x7f3590106b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:35:20.876 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:19.868+0000 7f3596a00700 1 -- 172.21.15.60:0/224255183 shutdown_connections 2024-05-03T14:35:20.876 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:19.869+0000 7f3596a00700 1 -- 172.21.15.60:0/224255183 wait complete. 2024-05-03T14:35:20.877 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:19.870+0000 7f3596a00700 1 Processor -- start 2024-05-03T14:35:20.877 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:19.870+0000 7f3596a00700 1 -- start start 2024-05-03T14:35:20.880 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:19.870+0000 7f3596a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35901090d0 0x7f359007eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:35:20.881 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:19.870+0000 7f3596a00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f359010d540 con 0x7f35901090d0 2024-05-03T14:35:20.881 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:19.870+0000 7f358fe00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35901090d0 0x7f359007eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:35:20.881 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:19.870+0000 7f358fe00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35901090d0 0x7f359007eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48970/0 (socket says 172.21.15.60:48970) 2024-05-03T14:35:20.881 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:19.870+0000 7f358fe00700 1 -- 172.21.15.60:0/1277629965 learned_addr learned my addr 172.21.15.60:0/1277629965 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:35:20.881 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:19.871+0000 7f358fe00700 1 -- 172.21.15.60:0/1277629965 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f359007f0d0 con 0x7f35901090d0 2024-05-03T14:35:20.881 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:19.871+0000 7f358fe00700 1 --2- 172.21.15.60:0/1277629965 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35901090d0 0x7f359007eb90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3580003fb0 tx=0x7f3580011bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:35:20.881 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:19.871+0000 7f358cc00700 1 -- 172.21.15.60:0/1277629965 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35800039c0 con 0x7f35901090d0 2024-05-03T14:35:20.881 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:19.871+0000 7f358cc00700 1 -- 172.21.15.60:0/1277629965 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1139+0+0 (secure 0 0 0) 0x7f3580003b60 con 0x7f35901090d0 2024-05-03T14:35:20.881 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:19.871+0000 7f358cc00700 1 -- 172.21.15.60:0/1277629965 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f358001a6b0 con 0x7f35901090d0 2024-05-03T14:35:20.881 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:19.871+0000 7f3596a00700 1 -- 172.21.15.60:0/1277629965 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f359007d2c0 con 0x7f35901090d0 2024-05-03T14:35:20.881 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:19.872+0000 7f3596a00700 1 -- 172.21.15.60:0/1277629965 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f359007d800 con 0x7f35901090d0 2024-05-03T14:35:20.881 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:19.873+0000 7f358cc00700 1 -- 172.21.15.60:0/1277629965 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f3580024430 con 0x7f35901090d0 2024-05-03T14:35:20.881 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:19.873+0000 7f3596a00700 1 -- 172.21.15.60:0/1277629965 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3590064b00 con 0x7f35901090d0 2024-05-03T14:35:20.881 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:19.873+0000 7f358cc00700 1 --2- 172.21.15.60:0/1277629965 >> [v2:172.21.15.60:6800/888447185,v1:172.21.15.60:6801/888447185] conn(0x7f357403e470 0x7f3574040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:35:20.882 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:19.873+0000 7f358cc00700 1 -- 172.21.15.60:0/1277629965 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f35800539e0 con 0x7f35901090d0 2024-05-03T14:35:20.882 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:19.874+0000 7f358f400700 1 -- 172.21.15.60:0/1277629965 >> [v2:172.21.15.60:6800/888447185,v1:172.21.15.60:6801/888447185] conn(0x7f357403e470 msgr2=0x7f3574040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6800/888447185 2024-05-03T14:35:20.882 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:19.874+0000 7f358f400700 1 --2- 172.21.15.60:0/1277629965 >> [v2:172.21.15.60:6800/888447185,v1:172.21.15.60:6801/888447185] conn(0x7f357403e470 0x7f3574040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-03T14:35:20.882 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:19.879+0000 7f358cc00700 1 -- 172.21.15.60:0/1277629965 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f35800172b0 con 0x7f35901090d0 2024-05-03T14:35:20.882 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:20.075+0000 7f358f400700 1 -- 172.21.15.60:0/1277629965 >> [v2:172.21.15.60:6800/888447185,v1:172.21.15.60:6801/888447185] conn(0x7f357403e470 msgr2=0x7f3574040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6800/888447185 2024-05-03T14:35:20.882 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:20.075+0000 7f358f400700 1 --2- 172.21.15.60:0/1277629965 >> [v2:172.21.15.60:6800/888447185,v1:172.21.15.60:6801/888447185] conn(0x7f357403e470 0x7f3574040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-03T14:35:20.882 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:20.173+0000 7f3596a00700 1 -- 172.21.15.60:0/1277629965 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f35901a30c0 con 0x7f35901090d0 2024-05-03T14:35:20.882 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:20.174+0000 7f358cc00700 1 -- 172.21.15.60:0/1277629965 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f3580003d10 con 0x7f35901090d0 2024-05-03T14:35:20.882 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:20.177+0000 7f3596a00700 1 -- 172.21.15.60:0/1277629965 >> [v2:172.21.15.60:6800/888447185,v1:172.21.15.60:6801/888447185] conn(0x7f357403e470 msgr2=0x7f3574040930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-03T14:35:20.882 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:20.177+0000 7f3596a00700 1 --2- 172.21.15.60:0/1277629965 >> [v2:172.21.15.60:6800/888447185,v1:172.21.15.60:6801/888447185] conn(0x7f357403e470 0x7f3574040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:35:20.883 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:20.177+0000 7f3596a00700 1 -- 172.21.15.60:0/1277629965 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35901090d0 msgr2=0x7f359007eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:35:20.883 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:20.177+0000 7f3596a00700 1 --2- 172.21.15.60:0/1277629965 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35901090d0 0x7f359007eb90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3580003fb0 tx=0x7f3580011bd0 comp rx=0 tx=0).stop 2024-05-03T14:35:20.883 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:20.177+0000 7f3596a00700 1 -- 172.21.15.60:0/1277629965 shutdown_connections 2024-05-03T14:35:20.883 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:20.177+0000 7f3596a00700 1 --2- 172.21.15.60:0/1277629965 >> [v2:172.21.15.60:6800/888447185,v1:172.21.15.60:6801/888447185] conn(0x7f357403e470 0x7f3574040930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:35:20.883 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:20.177+0000 7f3596a00700 1 --2- 172.21.15.60:0/1277629965 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35901090d0 0x7f359007eb90 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:35:20.883 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:20.177+0000 7f3596a00700 1 -- 172.21.15.60:0/1277629965 >> 172.21.15.60:0/1277629965 conn(0x7f3590104750 msgr2=0x7f3590105290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:35:20.883 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:20.177+0000 7f3596a00700 1 -- 172.21.15.60:0/1277629965 shutdown_connections 2024-05-03T14:35:20.883 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:20.177+0000 7f3596a00700 1 -- 172.21.15.60:0/1277629965 wait complete. 2024-05-03T14:35:20.884 INFO:teuthology.orchestra.run.smithi060.stdout:Waiting for the mgr to restart... 2024-05-03T14:35:20.884 INFO:teuthology.orchestra.run.smithi060.stdout:Waiting for mgr epoch 5... 2024-05-03T14:35:21.482 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:21 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:21.420+0000 7f7daf03c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-03T14:35:21.837 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:21 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:21.656+0000 7f7daf03c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-03T14:35:22.274 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:21 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:21.876+0000 7f7daf03c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-03T14:35:22.587 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:22 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:22.270+0000 7f7daf03c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-03T14:35:22.588 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:22 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:22.395+0000 7f7daf03c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-03T14:35:22.919 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:22 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:22.652+0000 7f7daf03c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-03T14:35:23.337 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:22 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:22.915+0000 7f7daf03c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-03T14:35:24.026 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:23 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:23.726+0000 7f7daf03c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-03T14:35:24.337 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:24 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:24.022+0000 7f7daf03c200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-03T14:35:24.837 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:24 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:24.596+0000 7f7daf03c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-03T14:35:24.837 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:24 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:24.727+0000 7f7daf03c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-03T14:35:25.587 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:25 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:25.201+0000 7f7daf03c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-03T14:35:26.087 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:25 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:25.670+0000 7f7daf03c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-03T14:35:26.088 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:25 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:25.797+0000 7f7daf03c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-03T14:35:26.587 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:26 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:26.133+0000 7f7daf03c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-03T14:35:26.588 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:26 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:26.318+0000 7f7daf03c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-03T14:35:26.987 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:26 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:26.698+0000 7f7daf03c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-03T14:35:26.987 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:26 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:26.854+0000 7f7daf03c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-03T14:35:26.987 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:26 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:26.982+0000 7f7daf03c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-03T14:35:28.067 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:27 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:27.765+0000 7f7daf03c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-03T14:35:28.337 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:28 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:28.063+0000 7f7daf03c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-03T14:35:28.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:28 smithi060 ceph-mon[28873]: Active manager daemon x restarted 2024-05-03T14:35:28.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:28 smithi060 ceph-mon[28873]: Activating manager daemon x 2024-05-03T14:35:28.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:28 smithi060 ceph-mon[28873]: osdmap e2: 0 total, 0 up, 0 in 2024-05-03T14:35:28.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:28 smithi060 ceph-mon[28873]: mgrmap e6: x(active, starting, since 0.01005s) 2024-05-03T14:35:28.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:28 smithi060 ceph-mon[28873]: from='mgr.14120 172.21.15.60:0/1113217389' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T14:35:28.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:28 smithi060 ceph-mon[28873]: from='mgr.14120 172.21.15.60:0/1113217389' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-03T14:35:28.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:28 smithi060 ceph-mon[28873]: from='mgr.14120 172.21.15.60:0/1113217389' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-03T14:35:28.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:28 smithi060 ceph-mon[28873]: from='mgr.14120 172.21.15.60:0/1113217389' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-03T14:35:28.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:28 smithi060 ceph-mon[28873]: from='mgr.14120 172.21.15.60:0/1113217389' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-03T14:35:28.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:28 smithi060 ceph-mon[28873]: Manager daemon x is now available 2024-05-03T14:35:28.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:28 smithi060 ceph-mon[28873]: from='mgr.14120 172.21.15.60:0/1113217389' entity='mgr.x' 2024-05-03T14:35:29.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:29 smithi060 ceph-mon[28873]: Found migration_current of "None". Setting to last migration. 2024-05-03T14:35:29.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:29 smithi060 ceph-mon[28873]: from='mgr.14120 172.21.15.60:0/1113217389' entity='mgr.x' 2024-05-03T14:35:29.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:29 smithi060 ceph-mon[28873]: from='mgr.14120 172.21.15.60:0/1113217389' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:35:29.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:29 smithi060 ceph-mon[28873]: from='mgr.14120 172.21.15.60:0/1113217389' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:35:29.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:29 smithi060 ceph-mon[28873]: from='mgr.14120 172.21.15.60:0/1113217389' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-03T14:35:29.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:29 smithi060 ceph-mon[28873]: from='mgr.14120 172.21.15.60:0/1113217389' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-03T14:35:29.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:29 smithi060 ceph-mon[28873]: from='mgr.14120 172.21.15.60:0/1113217389' entity='mgr.x' 2024-05-03T14:35:29.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:29 smithi060 ceph-mon[28873]: from='mgr.14120 172.21.15.60:0/1113217389' entity='mgr.x' 2024-05-03T14:35:29.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:29 smithi060 ceph-mon[28873]: mgrmap e7: x(active, since 1.01201s) 2024-05-03T14:35:30.227 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:30 smithi060 ceph-mon[28873]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-03T14:35:30.227 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:30 smithi060 ceph-mon[28873]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-03T14:35:30.227 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:30 smithi060 ceph-mon[28873]: [03/May/2024:14:35:29] ENGINE Bus STARTING 2024-05-03T14:35:30.227 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:30 smithi060 ceph-mon[28873]: [03/May/2024:14:35:29] ENGINE Serving on https://172.21.15.60:7150 2024-05-03T14:35:30.227 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:30 smithi060 ceph-mon[28873]: [03/May/2024:14:35:29] ENGINE Serving on http://172.21.15.60:8765 2024-05-03T14:35:30.227 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:30 smithi060 ceph-mon[28873]: [03/May/2024:14:35:29] ENGINE Bus STARTED 2024-05-03T14:35:30.227 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:30 smithi060 ceph-mon[28873]: from='mgr.14120 172.21.15.60:0/1113217389' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:35:30.235 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout { 2024-05-03T14:35:30.237 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-03T14:35:30.237 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-03T14:35:30.237 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout } 2024-05-03T14:35:30.237 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:21.478+0000 7f7df9400700 1 Processor -- start 2024-05-03T14:35:30.237 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:21.478+0000 7f7df9400700 1 -- start start 2024-05-03T14:35:30.237 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:21.479+0000 7f7df9400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7df407ee50 0x7f7df407d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:35:30.237 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:21.479+0000 7f7df9400700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f7df407d840 con 0x7f7df407ee50 2024-05-03T14:35:30.237 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:21.479+0000 7f7df3e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7df407ee50 0x7f7df407d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:35:30.237 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:21.479+0000 7f7df3e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7df407ee50 0x7f7df407d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48980/0 (socket says 172.21.15.60:48980) 2024-05-03T14:35:30.237 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:21.479+0000 7f7df3e00700 1 -- 172.21.15.60:0/25368456 learned_addr learned my addr 172.21.15.60:0/25368456 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:35:30.237 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:21.480+0000 7f7df3e00700 1 -- 172.21.15.60:0/25368456 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df407d9c0 con 0x7f7df407ee50 2024-05-03T14:35:30.244 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:21.480+0000 7f7df3e00700 1 --2- 172.21.15.60:0/25368456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7df407ee50 0x7f7df407d300 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7de40098a0 tx=0x7f7de4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2994f3db3dd83c41 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:35:30.244 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:21.481+0000 7f7df2a00700 1 -- 172.21.15.60:0/25368456 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7de4004160 con 0x7f7df407ee50 2024-05-03T14:35:30.244 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:21.481+0000 7f7df2a00700 1 -- 172.21.15.60:0/25368456 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1139+0+0 (secure 0 0 0) 0x7f7de4004300 con 0x7f7df407ee50 2024-05-03T14:35:30.244 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:21.481+0000 7f7df2a00700 1 -- 172.21.15.60:0/25368456 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7de4011c00 con 0x7f7df407ee50 2024-05-03T14:35:30.244 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:21.481+0000 7f7df9400700 1 -- 172.21.15.60:0/25368456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7df407ee50 msgr2=0x7f7df407d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:35:30.244 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:21.481+0000 7f7df9400700 1 --2- 172.21.15.60:0/25368456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7df407ee50 0x7f7df407d300 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7de40098a0 tx=0x7f7de4009bb0 comp rx=0 tx=0).stop 2024-05-03T14:35:30.244 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:21.482+0000 7f7df9400700 1 -- 172.21.15.60:0/25368456 shutdown_connections 2024-05-03T14:35:30.244 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:21.482+0000 7f7df9400700 1 --2- 172.21.15.60:0/25368456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7df407ee50 0x7f7df407d300 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:35:30.244 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:21.482+0000 7f7df9400700 1 -- 172.21.15.60:0/25368456 >> 172.21.15.60:0/25368456 conn(0x7f7df4104500 msgr2=0x7f7df4106920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:35:30.244 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:21.482+0000 7f7df9400700 1 -- 172.21.15.60:0/25368456 shutdown_connections 2024-05-03T14:35:30.244 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:21.482+0000 7f7df9400700 1 -- 172.21.15.60:0/25368456 wait complete. 2024-05-03T14:35:30.245 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:21.483+0000 7f7df9400700 1 Processor -- start 2024-05-03T14:35:30.245 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:21.483+0000 7f7df9400700 1 -- start start 2024-05-03T14:35:30.245 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:21.483+0000 7f7df9400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7df407ee50 0x7f7df419f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:35:30.245 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:21.483+0000 7f7df9400700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f7df410f570 con 0x7f7df407ee50 2024-05-03T14:35:30.245 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:21.484+0000 7f7df3e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7df407ee50 0x7f7df419f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:35:30.245 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:21.484+0000 7f7df3e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7df407ee50 0x7f7df419f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48984/0 (socket says 172.21.15.60:48984) 2024-05-03T14:35:30.245 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:21.484+0000 7f7df3e00700 1 -- 172.21.15.60:0/3369450438 learned_addr learned my addr 172.21.15.60:0/3369450438 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:35:30.245 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:21.484+0000 7f7df3e00700 1 -- 172.21.15.60:0/3369450438 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df419f6e0 con 0x7f7df407ee50 2024-05-03T14:35:30.245 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:21.484+0000 7f7df3e00700 1 --2- 172.21.15.60:0/3369450438 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7df407ee50 0x7f7df419f1a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7de4011bd0 tx=0x7f7de4011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:35:30.245 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:21.485+0000 7f7df0c00700 1 -- 172.21.15.60:0/3369450438 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7de4003970 con 0x7f7df407ee50 2024-05-03T14:35:30.245 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:21.485+0000 7f7df0c00700 1 -- 172.21.15.60:0/3369450438 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1139+0+0 (secure 0 0 0) 0x7f7de4003b10 con 0x7f7df407ee50 2024-05-03T14:35:30.245 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:21.485+0000 7f7df0c00700 1 -- 172.21.15.60:0/3369450438 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7de40245a0 con 0x7f7df407ee50 2024-05-03T14:35:30.245 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:21.485+0000 7f7df9400700 1 -- 172.21.15.60:0/3369450438 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7df419f970 con 0x7f7df407ee50 2024-05-03T14:35:30.245 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:21.485+0000 7f7df9400700 1 -- 172.21.15.60:0/3369450438 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7df41a2660 con 0x7f7df407ee50 2024-05-03T14:35:30.245 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:21.486+0000 7f7df0c00700 1 -- 172.21.15.60:0/3369450438 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f7de401a9a0 con 0x7f7df407ee50 2024-05-03T14:35:30.246 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:21.486+0000 7f7df0c00700 1 --2- 172.21.15.60:0/3369450438 >> [v2:172.21.15.60:6800/888447185,v1:172.21.15.60:6801/888447185] conn(0x7f7dd803e470 0x7f7dd8040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:35:30.246 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:21.487+0000 7f7df0c00700 1 -- 172.21.15.60:0/3369450438 --> [v2:172.21.15.60:6800/888447185,v1:172.21.15.60:6801/888447185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7dd8041040 con 0x7f7dd803e470 2024-05-03T14:35:30.246 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:21.487+0000 7f7df0c00700 1 -- 172.21.15.60:0/3369450438 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7de4053820 con 0x7f7df407ee50 2024-05-03T14:35:30.246 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:21.487+0000 7f7df3400700 1 -- 172.21.15.60:0/3369450438 >> [v2:172.21.15.60:6800/888447185,v1:172.21.15.60:6801/888447185] conn(0x7f7dd803e470 msgr2=0x7f7dd8040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6800/888447185 2024-05-03T14:35:30.246 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:21.487+0000 7f7df3400700 1 --2- 172.21.15.60:0/3369450438 >> [v2:172.21.15.60:6800/888447185,v1:172.21.15.60:6801/888447185] conn(0x7f7dd803e470 0x7f7dd8040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-03T14:35:30.246 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:21.688+0000 7f7df3400700 1 -- 172.21.15.60:0/3369450438 >> [v2:172.21.15.60:6800/888447185,v1:172.21.15.60:6801/888447185] conn(0x7f7dd803e470 msgr2=0x7f7dd8040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6800/888447185 2024-05-03T14:35:30.246 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:21.688+0000 7f7df3400700 1 --2- 172.21.15.60:0/3369450438 >> [v2:172.21.15.60:6800/888447185,v1:172.21.15.60:6801/888447185] conn(0x7f7dd803e470 0x7f7dd8040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-03T14:35:30.246 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:22.089+0000 7f7df3400700 1 -- 172.21.15.60:0/3369450438 >> [v2:172.21.15.60:6800/888447185,v1:172.21.15.60:6801/888447185] conn(0x7f7dd803e470 msgr2=0x7f7dd8040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6800/888447185 2024-05-03T14:35:30.246 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:22.089+0000 7f7df3400700 1 --2- 172.21.15.60:0/3369450438 >> [v2:172.21.15.60:6800/888447185,v1:172.21.15.60:6801/888447185] conn(0x7f7dd803e470 0x7f7dd8040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-03T14:35:30.246 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:22.890+0000 7f7df3400700 1 -- 172.21.15.60:0/3369450438 >> [v2:172.21.15.60:6800/888447185,v1:172.21.15.60:6801/888447185] conn(0x7f7dd803e470 msgr2=0x7f7dd8040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6800/888447185 2024-05-03T14:35:30.246 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:22.890+0000 7f7df3400700 1 --2- 172.21.15.60:0/3369450438 >> [v2:172.21.15.60:6800/888447185,v1:172.21.15.60:6801/888447185] conn(0x7f7dd803e470 0x7f7dd8040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-03T14:35:30.246 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:24.492+0000 7f7df3400700 1 -- 172.21.15.60:0/3369450438 >> [v2:172.21.15.60:6800/888447185,v1:172.21.15.60:6801/888447185] conn(0x7f7dd803e470 msgr2=0x7f7dd8040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6800/888447185 2024-05-03T14:35:30.246 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:24.492+0000 7f7df3400700 1 --2- 172.21.15.60:0/3369450438 >> [v2:172.21.15.60:6800/888447185,v1:172.21.15.60:6801/888447185] conn(0x7f7dd803e470 0x7f7dd8040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-03T14:35:30.247 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:27.695+0000 7f7df3400700 1 -- 172.21.15.60:0/3369450438 >> [v2:172.21.15.60:6800/888447185,v1:172.21.15.60:6801/888447185] conn(0x7f7dd803e470 msgr2=0x7f7dd8040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6800/888447185 2024-05-03T14:35:30.247 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:27.696+0000 7f7df3400700 1 --2- 172.21.15.60:0/3369450438 >> [v2:172.21.15.60:6800/888447185,v1:172.21.15.60:6801/888447185] conn(0x7f7dd803e470 0x7f7dd8040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-05-03T14:35:30.247 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:28.072+0000 7f7df0c00700 1 -- 172.21.15.60:0/3369450438 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mgrmap(e 6) ==== 49989+0+0 (secure 0 0 0) 0x7f7de4024c20 con 0x7f7df407ee50 2024-05-03T14:35:30.247 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:28.073+0000 7f7df0c00700 1 -- 172.21.15.60:0/3369450438 >> [v2:172.21.15.60:6800/888447185,v1:172.21.15.60:6801/888447185] conn(0x7f7dd803e470 msgr2=0x7f7dd8040930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-03T14:35:30.247 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:28.073+0000 7f7df0c00700 1 --2- 172.21.15.60:0/3369450438 >> [v2:172.21.15.60:6800/888447185,v1:172.21.15.60:6801/888447185] conn(0x7f7dd803e470 0x7f7dd8040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:35:30.247 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:29.075+0000 7f7df0c00700 1 -- 172.21.15.60:0/3369450438 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mgrmap(e 7) ==== 50116+0+0 (secure 0 0 0) 0x7f7de400b3e0 con 0x7f7df407ee50 2024-05-03T14:35:30.247 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:29.076+0000 7f7df0c00700 1 --2- 172.21.15.60:0/3369450438 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f7dd8041d60 0x7f7dd8044150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:35:30.247 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:29.076+0000 7f7df0c00700 1 -- 172.21.15.60:0/3369450438 --> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7de4051db0 con 0x7f7dd8041d60 2024-05-03T14:35:30.247 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:29.079+0000 7f7df3400700 1 --2- 172.21.15.60:0/3369450438 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f7dd8041d60 0x7f7dd8044150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:35:30.247 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:29.083+0000 7f7df3400700 1 --2- 172.21.15.60:0/3369450438 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f7dd8041d60 0x7f7dd8044150 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f7de0003a10 tx=0x7f7de0009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:35:30.247 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:29.086+0000 7f7df0c00700 1 -- 172.21.15.60:0/3369450438 <== mgr.14120 v2:172.21.15.60:6800/424391417 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f7de4051db0 con 0x7f7dd8041d60 2024-05-03T14:35:30.247 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:29.101+0000 7f7dea000700 1 -- 172.21.15.60:0/3369450438 --> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f7dcc002600 con 0x7f7dd8041d60 2024-05-03T14:35:30.247 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:29.102+0000 7f7df0c00700 1 -- 172.21.15.60:0/3369450438 <== mgr.14120 v2:172.21.15.60:6800/424391417 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f7dcc002600 con 0x7f7dd8041d60 2024-05-03T14:35:30.247 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:29.103+0000 7f7dea000700 1 -- 172.21.15.60:0/3369450438 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f7dd8041d60 msgr2=0x7f7dd8044150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:35:30.247 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:29.103+0000 7f7dea000700 1 --2- 172.21.15.60:0/3369450438 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f7dd8041d60 0x7f7dd8044150 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f7de0003a10 tx=0x7f7de0009290 comp rx=0 tx=0).stop 2024-05-03T14:35:30.248 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:29.103+0000 7f7dea000700 1 -- 172.21.15.60:0/3369450438 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7df407ee50 msgr2=0x7f7df419f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:35:30.248 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:29.103+0000 7f7dea000700 1 --2- 172.21.15.60:0/3369450438 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7df407ee50 0x7f7df419f1a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7de4011bd0 tx=0x7f7de4011cb0 comp rx=0 tx=0).stop 2024-05-03T14:35:30.248 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:29.103+0000 7f7dea000700 1 -- 172.21.15.60:0/3369450438 shutdown_connections 2024-05-03T14:35:30.248 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:29.103+0000 7f7dea000700 1 --2- 172.21.15.60:0/3369450438 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f7dd8041d60 0x7f7dd8044150 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:35:30.248 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:29.103+0000 7f7dea000700 1 --2- 172.21.15.60:0/3369450438 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7df407ee50 0x7f7df419f1a0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:35:30.248 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:29.103+0000 7f7dea000700 1 --2- 172.21.15.60:0/3369450438 >> [v2:172.21.15.60:6800/888447185,v1:172.21.15.60:6801/888447185] conn(0x7f7dd803e470 0x7f7dd8040930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:35:30.248 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:29.103+0000 7f7dea000700 1 -- 172.21.15.60:0/3369450438 >> 172.21.15.60:0/3369450438 conn(0x7f7df4104500 msgr2=0x7f7df41050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:35:30.248 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:29.104+0000 7f7dea000700 1 -- 172.21.15.60:0/3369450438 shutdown_connections 2024-05-03T14:35:30.248 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:29.104+0000 7f7dea000700 1 -- 172.21.15.60:0/3369450438 wait complete. 2024-05-03T14:35:30.248 INFO:teuthology.orchestra.run.smithi060.stdout:mgr epoch 5 is available 2024-05-03T14:35:30.248 INFO:teuthology.orchestra.run.smithi060.stdout:Setting orchestrator backend to cephadm... 2024-05-03T14:35:31.487 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:31 smithi060 ceph-mon[28873]: mgrmap e8: x(active, since 2s) 2024-05-03T14:35:31.837 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:30.889+0000 7f91c1600700 1 Processor -- start 2024-05-03T14:35:31.838 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:30.889+0000 7f91c1600700 1 -- start start 2024-05-03T14:35:31.838 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:30.890+0000 7f91c1600700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91bc108f20 0x7f91bc109340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:35:31.838 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:30.890+0000 7f91c1600700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f91bc109880 con 0x7f91bc108f20 2024-05-03T14:35:31.838 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:30.890+0000 7f91bbe00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91bc108f20 0x7f91bc109340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:35:31.838 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:30.890+0000 7f91bbe00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91bc108f20 0x7f91bc109340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55296/0 (socket says 172.21.15.60:55296) 2024-05-03T14:35:31.838 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:30.890+0000 7f91bbe00700 1 -- 172.21.15.60:0/186243910 learned_addr learned my addr 172.21.15.60:0/186243910 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:35:31.838 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:30.891+0000 7f91bbe00700 1 -- 172.21.15.60:0/186243910 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91bc109a00 con 0x7f91bc108f20 2024-05-03T14:35:31.838 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:30.891+0000 7f91bbe00700 1 --2- 172.21.15.60:0/186243910 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91bc108f20 0x7f91bc109340 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f91ac0098a0 tx=0x7f91ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=681f72d530dbe39e server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:35:31.838 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:30.891+0000 7f91baa00700 1 -- 172.21.15.60:0/186243910 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f91ac004160 con 0x7f91bc108f20 2024-05-03T14:35:31.838 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:30.892+0000 7f91baa00700 1 -- 172.21.15.60:0/186243910 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1139+0+0 (secure 0 0 0) 0x7f91ac004300 con 0x7f91bc108f20 2024-05-03T14:35:31.838 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:30.892+0000 7f91baa00700 1 -- 172.21.15.60:0/186243910 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f91ac011c00 con 0x7f91bc108f20 2024-05-03T14:35:31.838 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:30.892+0000 7f91c1600700 1 -- 172.21.15.60:0/186243910 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91bc108f20 msgr2=0x7f91bc109340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:35:31.839 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:30.892+0000 7f91c1600700 1 --2- 172.21.15.60:0/186243910 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91bc108f20 0x7f91bc109340 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f91ac0098a0 tx=0x7f91ac009bb0 comp rx=0 tx=0).stop 2024-05-03T14:35:31.839 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:30.892+0000 7f91c1600700 1 -- 172.21.15.60:0/186243910 shutdown_connections 2024-05-03T14:35:31.839 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:30.892+0000 7f91c1600700 1 --2- 172.21.15.60:0/186243910 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91bc108f20 0x7f91bc109340 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:35:31.839 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:30.892+0000 7f91c1600700 1 -- 172.21.15.60:0/186243910 >> 172.21.15.60:0/186243910 conn(0x7f91bc104580 msgr2=0x7f91bc1069a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:35:31.839 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:30.892+0000 7f91c1600700 1 -- 172.21.15.60:0/186243910 shutdown_connections 2024-05-03T14:35:31.839 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:30.892+0000 7f91c1600700 1 -- 172.21.15.60:0/186243910 wait complete. 2024-05-03T14:35:31.839 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:30.893+0000 7f91c1600700 1 Processor -- start 2024-05-03T14:35:31.839 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:30.894+0000 7f91c1600700 1 -- start start 2024-05-03T14:35:31.839 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:30.894+0000 7f91c1600700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91bc108f20 0x7f91bc07eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:35:31.839 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:30.894+0000 7f91c1600700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f91bc10d390 con 0x7f91bc108f20 2024-05-03T14:35:31.839 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:30.894+0000 7f91bbe00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91bc108f20 0x7f91bc07eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:35:31.839 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:30.894+0000 7f91bbe00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91bc108f20 0x7f91bc07eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55306/0 (socket says 172.21.15.60:55306) 2024-05-03T14:35:31.839 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:30.894+0000 7f91bbe00700 1 -- 172.21.15.60:0/94356705 learned_addr learned my addr 172.21.15.60:0/94356705 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:35:31.839 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:30.895+0000 7f91bbe00700 1 -- 172.21.15.60:0/94356705 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91bc07f0d0 con 0x7f91bc108f20 2024-05-03T14:35:31.840 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:30.895+0000 7f91bbe00700 1 --2- 172.21.15.60:0/94356705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91bc108f20 0x7f91bc07eb90 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f91ac003fb0 tx=0x7f91ac011bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:35:31.840 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:30.895+0000 7f91b8c00700 1 -- 172.21.15.60:0/94356705 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f91ac0039c0 con 0x7f91bc108f20 2024-05-03T14:35:31.840 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:30.895+0000 7f91b8c00700 1 -- 172.21.15.60:0/94356705 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1139+0+0 (secure 0 0 0) 0x7f91ac003b60 con 0x7f91bc108f20 2024-05-03T14:35:31.840 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:30.895+0000 7f91b8c00700 1 -- 172.21.15.60:0/94356705 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f91ac01a6d0 con 0x7f91bc108f20 2024-05-03T14:35:31.840 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:30.895+0000 7f91c1600700 1 -- 172.21.15.60:0/94356705 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91bc07d2c0 con 0x7f91bc108f20 2024-05-03T14:35:31.840 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:30.895+0000 7f91c1600700 1 -- 172.21.15.60:0/94356705 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91bc07d800 con 0x7f91bc108f20 2024-05-03T14:35:31.840 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:30.896+0000 7f91c1600700 1 -- 172.21.15.60:0/94356705 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91bc064b00 con 0x7f91bc108f20 2024-05-03T14:35:31.840 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:30.897+0000 7f91b8c00700 1 -- 172.21.15.60:0/94356705 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f91ac0249a0 con 0x7f91bc108f20 2024-05-03T14:35:31.840 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:30.897+0000 7f91b8c00700 1 --2- 172.21.15.60:0/94356705 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f91a003e470 0x7f91a0040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:35:31.840 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:30.897+0000 7f91b8c00700 1 -- 172.21.15.60:0/94356705 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f91ac053a00 con 0x7f91bc108f20 2024-05-03T14:35:31.840 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:30.897+0000 7f91bb400700 1 --2- 172.21.15.60:0/94356705 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f91a003e470 0x7f91a0040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:35:31.840 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:30.898+0000 7f91bb400700 1 --2- 172.21.15.60:0/94356705 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f91a003e470 0x7f91a0040930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f91a8006fd0 tx=0x7f91a8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:35:31.840 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:30.903+0000 7f91b8c00700 1 -- 172.21.15.60:0/94356705 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f91ac00b3e0 con 0x7f91bc108f20 2024-05-03T14:35:31.841 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:31.146+0000 7f91c1600700 1 -- 172.21.15.60:0/94356705 --> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f91bc1057c0 con 0x7f91a003e470 2024-05-03T14:35:31.841 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:31.159+0000 7f91b8c00700 1 -- 172.21.15.60:0/94356705 <== mgr.14120 v2:172.21.15.60:6800/424391417 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f91bc1057c0 con 0x7f91a003e470 2024-05-03T14:35:31.841 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:31.162+0000 7f91c1600700 1 -- 172.21.15.60:0/94356705 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f91a003e470 msgr2=0x7f91a0040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:35:31.841 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:31.162+0000 7f91c1600700 1 --2- 172.21.15.60:0/94356705 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f91a003e470 0x7f91a0040930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f91a8006fd0 tx=0x7f91a8006e40 comp rx=0 tx=0).stop 2024-05-03T14:35:31.841 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:31.162+0000 7f91c1600700 1 -- 172.21.15.60:0/94356705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91bc108f20 msgr2=0x7f91bc07eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:35:31.841 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:31.162+0000 7f91c1600700 1 --2- 172.21.15.60:0/94356705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91bc108f20 0x7f91bc07eb90 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f91ac003fb0 tx=0x7f91ac011bd0 comp rx=0 tx=0).stop 2024-05-03T14:35:31.841 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:31.162+0000 7f91c1600700 1 -- 172.21.15.60:0/94356705 shutdown_connections 2024-05-03T14:35:31.841 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:31.162+0000 7f91c1600700 1 --2- 172.21.15.60:0/94356705 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f91a003e470 0x7f91a0040930 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:35:31.841 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:31.162+0000 7f91c1600700 1 --2- 172.21.15.60:0/94356705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91bc108f20 0x7f91bc07eb90 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:35:31.841 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:31.162+0000 7f91c1600700 1 -- 172.21.15.60:0/94356705 >> 172.21.15.60:0/94356705 conn(0x7f91bc104580 msgr2=0x7f91bc1050b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:35:31.841 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:31.162+0000 7f91c1600700 1 -- 172.21.15.60:0/94356705 shutdown_connections 2024-05-03T14:35:31.841 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:31.163+0000 7f91c1600700 1 -- 172.21.15.60:0/94356705 wait complete. 2024-05-03T14:35:32.425 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:32 smithi060 ceph-mon[28873]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:35:32.425 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:32 smithi060 ceph-mon[28873]: from='mgr.14120 172.21.15.60:0/1113217389' entity='mgr.x' 2024-05-03T14:35:32.425 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:32 smithi060 ceph-mon[28873]: from='mgr.14120 172.21.15.60:0/1113217389' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:35:33.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:33 smithi060 ceph-mon[28873]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:35:33.399 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-03T14:35:33.400 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:32.420+0000 7f40da600700 1 Processor -- start 2024-05-03T14:35:33.400 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:32.420+0000 7f40da600700 1 -- start start 2024-05-03T14:35:33.400 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:32.421+0000 7f40da600700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40d407edd0 0x7f40d407d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:35:33.400 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:32.421+0000 7f40da600700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f40d407d880 con 0x7f40d407edd0 2024-05-03T14:35:33.400 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:32.421+0000 7f40d3400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40d407edd0 0x7f40d407d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:35:33.400 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:32.421+0000 7f40d3400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40d407edd0 0x7f40d407d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50998/0 (socket says 172.21.15.60:50998) 2024-05-03T14:35:33.400 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:32.421+0000 7f40d3400700 1 -- 172.21.15.60:0/2157023204 learned_addr learned my addr 172.21.15.60:0/2157023204 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:35:33.405 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:32.422+0000 7f40d3400700 1 -- 172.21.15.60:0/2157023204 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40d407da00 con 0x7f40d407edd0 2024-05-03T14:35:33.405 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:32.422+0000 7f40d3400700 1 --2- 172.21.15.60:0/2157023204 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40d407edd0 0x7f40d407d340 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f40c40098a0 tx=0x7f40c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea6ab7eb5efa6f2a server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:35:33.406 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:32.423+0000 7f40d2000700 1 -- 172.21.15.60:0/2157023204 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40c4004160 con 0x7f40d407edd0 2024-05-03T14:35:33.406 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:32.423+0000 7f40d2000700 1 -- 172.21.15.60:0/2157023204 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1139+0+0 (secure 0 0 0) 0x7f40c4004300 con 0x7f40d407edd0 2024-05-03T14:35:33.406 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:32.423+0000 7f40d2000700 1 -- 172.21.15.60:0/2157023204 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40c4011c00 con 0x7f40d407edd0 2024-05-03T14:35:33.406 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:32.423+0000 7f40da600700 1 -- 172.21.15.60:0/2157023204 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40d407edd0 msgr2=0x7f40d407d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:35:33.406 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:32.423+0000 7f40da600700 1 --2- 172.21.15.60:0/2157023204 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40d407edd0 0x7f40d407d340 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f40c40098a0 tx=0x7f40c4009bb0 comp rx=0 tx=0).stop 2024-05-03T14:35:33.406 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:32.424+0000 7f40da600700 1 -- 172.21.15.60:0/2157023204 shutdown_connections 2024-05-03T14:35:33.406 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:32.424+0000 7f40da600700 1 --2- 172.21.15.60:0/2157023204 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40d407edd0 0x7f40d407d340 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:35:33.406 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:32.424+0000 7f40da600700 1 -- 172.21.15.60:0/2157023204 >> 172.21.15.60:0/2157023204 conn(0x7f40d4104750 msgr2=0x7f40d4106b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:35:33.406 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:32.424+0000 7f40da600700 1 -- 172.21.15.60:0/2157023204 shutdown_connections 2024-05-03T14:35:33.406 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:32.424+0000 7f40da600700 1 -- 172.21.15.60:0/2157023204 wait complete. 2024-05-03T14:35:33.406 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:32.425+0000 7f40da600700 1 Processor -- start 2024-05-03T14:35:33.406 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:32.425+0000 7f40da600700 1 -- start start 2024-05-03T14:35:33.406 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:32.425+0000 7f40da600700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40d407edd0 0x7f40d419f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:35:33.407 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:32.425+0000 7f40da600700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f40d410f5e0 con 0x7f40d407edd0 2024-05-03T14:35:33.407 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:32.426+0000 7f40d3400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40d407edd0 0x7f40d419f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:35:33.407 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:32.426+0000 7f40d3400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40d407edd0 0x7f40d419f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51004/0 (socket says 172.21.15.60:51004) 2024-05-03T14:35:33.407 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:32.426+0000 7f40d3400700 1 -- 172.21.15.60:0/1040519533 learned_addr learned my addr 172.21.15.60:0/1040519533 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:35:33.407 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:32.426+0000 7f40d3400700 1 -- 172.21.15.60:0/1040519533 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40d419f840 con 0x7f40d407edd0 2024-05-03T14:35:33.407 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:32.426+0000 7f40d3400700 1 --2- 172.21.15.60:0/1040519533 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40d407edd0 0x7f40d419f300 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f40c4009fb0 tx=0x7f40c4011c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:35:33.407 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:32.427+0000 7f40cbe00700 1 -- 172.21.15.60:0/1040519533 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40c40038b0 con 0x7f40d407edd0 2024-05-03T14:35:33.407 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:32.427+0000 7f40cbe00700 1 -- 172.21.15.60:0/1040519533 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1139+0+0 (secure 0 0 0) 0x7f40c401a430 con 0x7f40d407edd0 2024-05-03T14:35:33.407 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:32.427+0000 7f40cbe00700 1 -- 172.21.15.60:0/1040519533 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40c4019630 con 0x7f40d407edd0 2024-05-03T14:35:33.407 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:32.427+0000 7f40da600700 1 -- 172.21.15.60:0/1040519533 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40d419fad0 con 0x7f40d407edd0 2024-05-03T14:35:33.407 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:32.427+0000 7f40da600700 1 -- 172.21.15.60:0/1040519533 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40d41a27c0 con 0x7f40d407edd0 2024-05-03T14:35:33.407 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:32.428+0000 7f40da600700 1 -- 172.21.15.60:0/1040519533 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40d4064b00 con 0x7f40d407edd0 2024-05-03T14:35:33.407 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:32.428+0000 7f40cbe00700 1 -- 172.21.15.60:0/1040519533 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f40c401aa10 con 0x7f40d407edd0 2024-05-03T14:35:33.407 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:32.429+0000 7f40cbe00700 1 --2- 172.21.15.60:0/1040519533 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f40b803e470 0x7f40b8040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:35:33.407 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:32.429+0000 7f40cbe00700 1 -- 172.21.15.60:0/1040519533 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f40c4053a10 con 0x7f40d407edd0 2024-05-03T14:35:33.408 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:32.429+0000 7f40d2a00700 1 --2- 172.21.15.60:0/1040519533 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f40b803e470 0x7f40b8040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:35:33.408 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:32.430+0000 7f40d2a00700 1 --2- 172.21.15.60:0/1040519533 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f40b803e470 0x7f40b8040930 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f40c0006fd0 tx=0x7f40c0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:35:33.408 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:32.434+0000 7f40cbe00700 1 -- 172.21.15.60:0/1040519533 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f40c4023e10 con 0x7f40d407edd0 2024-05-03T14:35:33.408 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:32.665+0000 7f40da600700 1 -- 172.21.15.60:0/1040519533 --> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f40d41059e0 con 0x7f40b803e470 2024-05-03T14:35:33.408 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:32.667+0000 7f40cbe00700 1 -- 172.21.15.60:0/1040519533 <== mgr.14120 v2:172.21.15.60:6800/424391417 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f40d41059e0 con 0x7f40b803e470 2024-05-03T14:35:33.408 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:32.669+0000 7f40da600700 1 -- 172.21.15.60:0/1040519533 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f40b803e470 msgr2=0x7f40b8040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:35:33.408 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:32.669+0000 7f40da600700 1 --2- 172.21.15.60:0/1040519533 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f40b803e470 0x7f40b8040930 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f40c0006fd0 tx=0x7f40c0006e40 comp rx=0 tx=0).stop 2024-05-03T14:35:33.408 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:32.669+0000 7f40da600700 1 -- 172.21.15.60:0/1040519533 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40d407edd0 msgr2=0x7f40d419f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:35:33.408 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:32.669+0000 7f40da600700 1 --2- 172.21.15.60:0/1040519533 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40d407edd0 0x7f40d419f300 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f40c4009fb0 tx=0x7f40c4011c60 comp rx=0 tx=0).stop 2024-05-03T14:35:33.408 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:32.670+0000 7f40da600700 1 -- 172.21.15.60:0/1040519533 shutdown_connections 2024-05-03T14:35:33.408 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:32.670+0000 7f40da600700 1 --2- 172.21.15.60:0/1040519533 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f40b803e470 0x7f40b8040930 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:35:33.408 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:32.670+0000 7f40da600700 1 --2- 172.21.15.60:0/1040519533 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40d407edd0 0x7f40d419f300 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:35:33.408 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:32.670+0000 7f40da600700 1 -- 172.21.15.60:0/1040519533 >> 172.21.15.60:0/1040519533 conn(0x7f40d4104750 msgr2=0x7f40d41052d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:35:33.409 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:32.670+0000 7f40da600700 1 -- 172.21.15.60:0/1040519533 shutdown_connections 2024-05-03T14:35:33.409 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:32.670+0000 7f40da600700 1 -- 172.21.15.60:0/1040519533 wait complete. 2024-05-03T14:35:33.409 INFO:teuthology.orchestra.run.smithi060.stdout:Generating ssh key... 2024-05-03T14:35:35.021 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:34 smithi060 ceph-mon[28873]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:35:35.022 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:34 smithi060 ceph-mon[28873]: Generating ssh key... 2024-05-03T14:35:35.022 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:34 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: Generating public/private rsa key pair. 2024-05-03T14:35:35.022 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:34 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: Your identification has been saved in /tmp/tmpd6kduw_p/key. 2024-05-03T14:35:35.022 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:34 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: Your public key has been saved in /tmp/tmpd6kduw_p/key.pub. 2024-05-03T14:35:35.022 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:34 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: The key fingerprint is: 2024-05-03T14:35:35.023 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:34 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: SHA256:VvjC+BFTd5uBp7d1aZ490P9ukKXZiA+/dkSk1B67zEc ceph-285d64b2-095a-11ef-bc97-c7b262605968 2024-05-03T14:35:35.023 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:34 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: The key's randomart image is: 2024-05-03T14:35:35.023 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:34 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: +---[RSA 3072]----+ 2024-05-03T14:35:35.023 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:34 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: | . ..o. | 2024-05-03T14:35:35.023 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:34 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: | o ...o=o| 2024-05-03T14:35:35.023 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:34 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: | + . ++++| 2024-05-03T14:35:35.023 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:34 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: | o = ..o=E| 2024-05-03T14:35:35.023 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:34 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: | . S . o*%*| 2024-05-03T14:35:35.023 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:34 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: | o o o BB*| 2024-05-03T14:35:35.023 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:34 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: | . + o+| 2024-05-03T14:35:35.023 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:34 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: | + +| 2024-05-03T14:35:35.023 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:34 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: | ..=.| 2024-05-03T14:35:35.023 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:34 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: +----[SHA256]-----+ 2024-05-03T14:35:35.640 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:34.051+0000 7f6adae00700 1 Processor -- start 2024-05-03T14:35:35.640 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:34.051+0000 7f6adae00700 1 -- start start 2024-05-03T14:35:35.640 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:34.052+0000 7f6adae00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ad410fcf0 0x7f6ad41120e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:35:35.640 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:34.052+0000 7f6adae00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad4112620 con 0x7f6ad410fcf0 2024-05-03T14:35:35.641 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:34.052+0000 7f6ad9a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ad410fcf0 0x7f6ad41120e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:35:35.641 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:34.053+0000 7f6ad9a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ad410fcf0 0x7f6ad41120e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51008/0 (socket says 172.21.15.60:51008) 2024-05-03T14:35:35.641 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:34.053+0000 7f6ad9a00700 1 -- 172.21.15.60:0/4256218973 learned_addr learned my addr 172.21.15.60:0/4256218973 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:35:35.641 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:34.053+0000 7f6ad9a00700 1 -- 172.21.15.60:0/4256218973 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ad41127a0 con 0x7f6ad410fcf0 2024-05-03T14:35:35.641 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:34.053+0000 7f6ad9a00700 1 --2- 172.21.15.60:0/4256218973 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ad410fcf0 0x7f6ad41120e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6ac80098a0 tx=0x7f6ac8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e0340ab570ae1b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:35:35.641 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:34.054+0000 7f6ad3e00700 1 -- 172.21.15.60:0/4256218973 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ac8004160 con 0x7f6ad410fcf0 2024-05-03T14:35:35.641 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:34.054+0000 7f6ad3e00700 1 -- 172.21.15.60:0/4256218973 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1139+0+0 (secure 0 0 0) 0x7f6ac8004300 con 0x7f6ad410fcf0 2024-05-03T14:35:35.641 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:34.054+0000 7f6ad3e00700 1 -- 172.21.15.60:0/4256218973 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ac8011c00 con 0x7f6ad410fcf0 2024-05-03T14:35:35.641 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:34.054+0000 7f6adae00700 1 -- 172.21.15.60:0/4256218973 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ad410fcf0 msgr2=0x7f6ad41120e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:35:35.641 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:34.054+0000 7f6adae00700 1 --2- 172.21.15.60:0/4256218973 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ad410fcf0 0x7f6ad41120e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6ac80098a0 tx=0x7f6ac8009bb0 comp rx=0 tx=0).stop 2024-05-03T14:35:35.641 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:34.054+0000 7f6adae00700 1 -- 172.21.15.60:0/4256218973 shutdown_connections 2024-05-03T14:35:35.641 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:34.054+0000 7f6adae00700 1 --2- 172.21.15.60:0/4256218973 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ad410fcf0 0x7f6ad41120e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:35:35.642 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:34.054+0000 7f6adae00700 1 -- 172.21.15.60:0/4256218973 >> 172.21.15.60:0/4256218973 conn(0x7f6ad407ec10 msgr2=0x7f6ad4081030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:35:35.642 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:34.055+0000 7f6adae00700 1 -- 172.21.15.60:0/4256218973 shutdown_connections 2024-05-03T14:35:35.642 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:34.055+0000 7f6adae00700 1 -- 172.21.15.60:0/4256218973 wait complete. 2024-05-03T14:35:35.642 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:34.056+0000 7f6adae00700 1 Processor -- start 2024-05-03T14:35:35.642 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:34.056+0000 7f6adae00700 1 -- start start 2024-05-03T14:35:35.642 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:34.056+0000 7f6adae00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ad410fcf0 0x7f6ad41a34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:35:35.642 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:34.056+0000 7f6adae00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad4113890 con 0x7f6ad410fcf0 2024-05-03T14:35:35.642 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:34.056+0000 7f6ad9a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ad410fcf0 0x7f6ad41a34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:35:35.642 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:34.056+0000 7f6ad9a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ad410fcf0 0x7f6ad41a34f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51016/0 (socket says 172.21.15.60:51016) 2024-05-03T14:35:35.642 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:34.056+0000 7f6ad9a00700 1 -- 172.21.15.60:0/867276608 learned_addr learned my addr 172.21.15.60:0/867276608 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:35:35.642 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:34.057+0000 7f6ad9a00700 1 -- 172.21.15.60:0/867276608 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ad41a3a30 con 0x7f6ad410fcf0 2024-05-03T14:35:35.642 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:34.057+0000 7f6ad9a00700 1 --2- 172.21.15.60:0/867276608 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ad410fcf0 0x7f6ad41a34f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6ac80118c0 tx=0x7f6ac8011c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:35:35.642 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:34.057+0000 7f6ad2000700 1 -- 172.21.15.60:0/867276608 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ac8003af0 con 0x7f6ad410fcf0 2024-05-03T14:35:35.643 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:34.057+0000 7f6ad2000700 1 -- 172.21.15.60:0/867276608 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1139+0+0 (secure 0 0 0) 0x7f6ac8020070 con 0x7f6ad410fcf0 2024-05-03T14:35:35.643 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:34.057+0000 7f6adae00700 1 -- 172.21.15.60:0/867276608 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ad41a3cc0 con 0x7f6ad410fcf0 2024-05-03T14:35:35.643 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:34.057+0000 7f6ad2000700 1 -- 172.21.15.60:0/867276608 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ac80195e0 con 0x7f6ad410fcf0 2024-05-03T14:35:35.643 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:34.058+0000 7f6adae00700 1 -- 172.21.15.60:0/867276608 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ad41a69b0 con 0x7f6ad410fcf0 2024-05-03T14:35:35.643 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:34.059+0000 7f6ad2000700 1 -- 172.21.15.60:0/867276608 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f6ac801a9a0 con 0x7f6ad410fcf0 2024-05-03T14:35:35.643 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:34.059+0000 7f6adae00700 1 -- 172.21.15.60:0/867276608 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ad4064b00 con 0x7f6ad410fcf0 2024-05-03T14:35:35.643 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:34.059+0000 7f6ad2000700 1 --2- 172.21.15.60:0/867276608 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f6abc03e4c0 0x7f6abc040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:35:35.643 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:34.059+0000 7f6ad2000700 1 -- 172.21.15.60:0/867276608 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6ac8058580 con 0x7f6ad410fcf0 2024-05-03T14:35:35.643 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:34.060+0000 7f6ad9000700 1 --2- 172.21.15.60:0/867276608 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f6abc03e4c0 0x7f6abc040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:35:35.643 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:34.060+0000 7f6ad9000700 1 --2- 172.21.15.60:0/867276608 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f6abc03e4c0 0x7f6abc040980 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6ac0006fd0 tx=0x7f6ac0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:35:35.643 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:34.065+0000 7f6ad2000700 1 -- 172.21.15.60:0/867276608 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f6ac80172b0 con 0x7f6ad410fcf0 2024-05-03T14:35:35.643 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:34.303+0000 7f6adae00700 1 -- 172.21.15.60:0/867276608 --> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f6ad4080e80 con 0x7f6abc03e4c0 2024-05-03T14:35:35.643 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:34.999+0000 7f6ad2000700 1 -- 172.21.15.60:0/867276608 <== mgr.14120 v2:172.21.15.60:6800/424391417 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6ad4080e80 con 0x7f6abc03e4c0 2024-05-03T14:35:35.643 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:35.002+0000 7f6adae00700 1 -- 172.21.15.60:0/867276608 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f6abc03e4c0 msgr2=0x7f6abc040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:35:35.644 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:35.002+0000 7f6adae00700 1 --2- 172.21.15.60:0/867276608 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f6abc03e4c0 0x7f6abc040980 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6ac0006fd0 tx=0x7f6ac0006e40 comp rx=0 tx=0).stop 2024-05-03T14:35:35.644 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:35.002+0000 7f6adae00700 1 -- 172.21.15.60:0/867276608 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ad410fcf0 msgr2=0x7f6ad41a34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:35:35.644 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:35.002+0000 7f6adae00700 1 --2- 172.21.15.60:0/867276608 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ad410fcf0 0x7f6ad41a34f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6ac80118c0 tx=0x7f6ac8011c80 comp rx=0 tx=0).stop 2024-05-03T14:35:35.644 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:35.002+0000 7f6adae00700 1 -- 172.21.15.60:0/867276608 shutdown_connections 2024-05-03T14:35:35.644 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:35.002+0000 7f6adae00700 1 --2- 172.21.15.60:0/867276608 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f6abc03e4c0 0x7f6abc040980 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:35:35.644 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:35.002+0000 7f6adae00700 1 --2- 172.21.15.60:0/867276608 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ad410fcf0 0x7f6ad41a34f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:35:35.644 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:35.002+0000 7f6adae00700 1 -- 172.21.15.60:0/867276608 >> 172.21.15.60:0/867276608 conn(0x7f6ad407ec10 msgr2=0x7f6ad4080770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:35:35.644 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:35.002+0000 7f6adae00700 1 -- 172.21.15.60:0/867276608 shutdown_connections 2024-05-03T14:35:35.644 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:35.002+0000 7f6adae00700 1 -- 172.21.15.60:0/867276608 wait complete. 2024-05-03T14:35:36.263 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:35 smithi060 ceph-mon[28873]: from='mgr.14120 172.21.15.60:0/1113217389' entity='mgr.x' 2024-05-03T14:35:36.263 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:35 smithi060 ceph-mon[28873]: from='mgr.14120 172.21.15.60:0/1113217389' entity='mgr.x' 2024-05-03T14:35:37.225 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:36 smithi060 ceph-mon[28873]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:35:37.237 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGj+Vyqd8aAQiJp4VPt9ZKpUtijfX5oIjqped5rn0oI9rdDS2uD8reuxqzIi5HZaW21BYBG25pAunMBdHpW2r5Iw6t17emAMGMh0AxPsQzJ8tG8J+mjWz2bX36Y6w+/8yyDvBziiWJYZXIRRB6bxxgHUQuO29wcg5bvG2M82wmguHSBQ+ZE5CQ7b7Tl8+nTbnif/4kD7lDxF74bsRQyMxbUf2gnC/XhKSEZpHXzWzD3/6+zaUZ7fm7gFxI4PgMo96XXQ7fYfCvpiUoqWPUnKLyh9/gQEZfT4Zmpagc+HHbjUoGiNWQlh1L/gsi8IpJOsb1T0tAUGd9jB+uxUMbcSTc2CfvrVf8S6qKB5rngS5D65Sy32no0JbBISShCsvoW6NZViImbtLc1mQZ6kcXmjXRbTxVuabIgz7rLT/GToKbnr7HmZAPiPCVcaC2Jn+FyFfMx6WMwPKwSkj5WqOKUDiE7gkzp9Zp/8YAbx9UwqMJgmSapet11CvOhKeXCVKqGlE= ceph-285d64b2-095a-11ef-bc97-c7b262605968 2024-05-03T14:35:37.238 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:36.259+0000 7fedd9800700 1 Processor -- start 2024-05-03T14:35:37.238 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:36.259+0000 7fedd9800700 1 -- start start 2024-05-03T14:35:37.238 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:36.260+0000 7fedd9800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fedd4109a20 0x7fedd410be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:35:37.243 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:36.260+0000 7fedd9800700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fedd407ce10 con 0x7fedd4109a20 2024-05-03T14:35:37.243 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:36.260+0000 7fedd3e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fedd4109a20 0x7fedd410be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:35:37.243 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:36.260+0000 7fedd3e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fedd4109a20 0x7fedd410be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51026/0 (socket says 172.21.15.60:51026) 2024-05-03T14:35:37.243 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:36.260+0000 7fedd3e00700 1 -- 172.21.15.60:0/260137150 learned_addr learned my addr 172.21.15.60:0/260137150 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:35:37.243 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:36.261+0000 7fedd3e00700 1 -- 172.21.15.60:0/260137150 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedd410c350 con 0x7fedd4109a20 2024-05-03T14:35:37.243 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:36.261+0000 7fedd3e00700 1 --2- 172.21.15.60:0/260137150 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fedd4109a20 0x7fedd410be10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fedc40098a0 tx=0x7fedc4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa77127f54e66a68 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:35:37.243 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:36.262+0000 7fedd2a00700 1 -- 172.21.15.60:0/260137150 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fedc4004160 con 0x7fedd4109a20 2024-05-03T14:35:37.244 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:36.262+0000 7fedd2a00700 1 -- 172.21.15.60:0/260137150 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1139+0+0 (secure 0 0 0) 0x7fedc4004300 con 0x7fedd4109a20 2024-05-03T14:35:37.244 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:36.262+0000 7fedd2a00700 1 -- 172.21.15.60:0/260137150 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fedc4011c00 con 0x7fedd4109a20 2024-05-03T14:35:37.244 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:36.262+0000 7fedd9800700 1 -- 172.21.15.60:0/260137150 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fedd4109a20 msgr2=0x7fedd410be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:35:37.244 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:36.262+0000 7fedd9800700 1 --2- 172.21.15.60:0/260137150 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fedd4109a20 0x7fedd410be10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fedc40098a0 tx=0x7fedc4009bb0 comp rx=0 tx=0).stop 2024-05-03T14:35:37.244 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:36.262+0000 7fedd9800700 1 -- 172.21.15.60:0/260137150 shutdown_connections 2024-05-03T14:35:37.244 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:36.263+0000 7fedd9800700 1 --2- 172.21.15.60:0/260137150 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fedd4109a20 0x7fedd410be10 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:35:37.244 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:36.263+0000 7fedd9800700 1 -- 172.21.15.60:0/260137150 >> 172.21.15.60:0/260137150 conn(0x7fedd41033d0 msgr2=0x7fedd41057f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:35:37.244 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:36.263+0000 7fedd9800700 1 -- 172.21.15.60:0/260137150 shutdown_connections 2024-05-03T14:35:37.244 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:36.263+0000 7fedd9800700 1 -- 172.21.15.60:0/260137150 wait complete. 2024-05-03T14:35:37.244 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:36.264+0000 7fedd9800700 1 Processor -- start 2024-05-03T14:35:37.244 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:36.264+0000 7fedd9800700 1 -- start start 2024-05-03T14:35:37.244 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:36.264+0000 7fedd9800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fedd4109a20 0x7fedd419f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:35:37.244 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:36.264+0000 7fedd9800700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fedd410d320 con 0x7fedd4109a20 2024-05-03T14:35:37.244 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:36.265+0000 7fedd3e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fedd4109a20 0x7fedd419f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:35:37.245 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:36.265+0000 7fedd3e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fedd4109a20 0x7fedd419f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51030/0 (socket says 172.21.15.60:51030) 2024-05-03T14:35:37.245 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:36.265+0000 7fedd3e00700 1 -- 172.21.15.60:0/563974478 learned_addr learned my addr 172.21.15.60:0/563974478 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:35:37.245 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:36.265+0000 7fedd3e00700 1 -- 172.21.15.60:0/563974478 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedd419f690 con 0x7fedd4109a20 2024-05-03T14:35:37.245 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:36.265+0000 7fedd3e00700 1 --2- 172.21.15.60:0/563974478 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fedd4109a20 0x7fedd419f150 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fedc4011bd0 tx=0x7fedc4011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:35:37.245 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:36.266+0000 7fedd0c00700 1 -- 172.21.15.60:0/563974478 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fedc4003970 con 0x7fedd4109a20 2024-05-03T14:35:37.245 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:36.266+0000 7fedd0c00700 1 -- 172.21.15.60:0/563974478 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1139+0+0 (secure 0 0 0) 0x7fedc4003b10 con 0x7fedd4109a20 2024-05-03T14:35:37.245 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:36.266+0000 7fedd0c00700 1 -- 172.21.15.60:0/563974478 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fedc40245a0 con 0x7fedd4109a20 2024-05-03T14:35:37.245 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:36.266+0000 7fedd9800700 1 -- 172.21.15.60:0/563974478 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedd419f920 con 0x7fedd4109a20 2024-05-03T14:35:37.245 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:36.266+0000 7fedd9800700 1 -- 172.21.15.60:0/563974478 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedd41a2610 con 0x7fedd4109a20 2024-05-03T14:35:37.245 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:36.267+0000 7fedd9800700 1 -- 172.21.15.60:0/563974478 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedd4064b00 con 0x7fedd4109a20 2024-05-03T14:35:37.245 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:36.267+0000 7fedd0c00700 1 -- 172.21.15.60:0/563974478 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fedc401a9a0 con 0x7fedd4109a20 2024-05-03T14:35:37.245 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:36.268+0000 7fedd0c00700 1 --2- 172.21.15.60:0/563974478 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7fedb803e4c0 0x7fedb8040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:35:37.245 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:36.268+0000 7fedd0c00700 1 -- 172.21.15.60:0/563974478 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fedc40537c0 con 0x7fedd4109a20 2024-05-03T14:35:37.245 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:36.268+0000 7fedd3400700 1 --2- 172.21.15.60:0/563974478 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7fedb803e4c0 0x7fedb8040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:35:37.246 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:36.269+0000 7fedd3400700 1 --2- 172.21.15.60:0/563974478 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7fedb803e4c0 0x7fedb8040980 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fedc0006fd0 tx=0x7fedc0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:35:37.246 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:36.274+0000 7fedd0c00700 1 -- 172.21.15.60:0/563974478 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fedc400b3e0 con 0x7fedd4109a20 2024-05-03T14:35:37.246 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:36.509+0000 7fedd9800700 1 -- 172.21.15.60:0/563974478 --> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fedd41045e0 con 0x7fedb803e4c0 2024-05-03T14:35:37.246 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:36.510+0000 7fedd0c00700 1 -- 172.21.15.60:0/563974478 <== mgr.14120 v2:172.21.15.60:6800/424391417 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fedd41045e0 con 0x7fedb803e4c0 2024-05-03T14:35:37.246 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:36.514+0000 7fedd9800700 1 -- 172.21.15.60:0/563974478 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7fedb803e4c0 msgr2=0x7fedb8040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:35:37.246 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:36.514+0000 7fedd9800700 1 --2- 172.21.15.60:0/563974478 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7fedb803e4c0 0x7fedb8040980 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fedc0006fd0 tx=0x7fedc0006e40 comp rx=0 tx=0).stop 2024-05-03T14:35:37.246 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:36.514+0000 7fedd9800700 1 -- 172.21.15.60:0/563974478 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fedd4109a20 msgr2=0x7fedd419f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:35:37.246 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:36.514+0000 7fedd9800700 1 --2- 172.21.15.60:0/563974478 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fedd4109a20 0x7fedd419f150 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fedc4011bd0 tx=0x7fedc4011cb0 comp rx=0 tx=0).stop 2024-05-03T14:35:37.246 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:36.514+0000 7fedd9800700 1 -- 172.21.15.60:0/563974478 shutdown_connections 2024-05-03T14:35:37.246 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:36.514+0000 7fedd9800700 1 --2- 172.21.15.60:0/563974478 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7fedb803e4c0 0x7fedb8040980 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:35:37.246 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:36.514+0000 7fedd9800700 1 --2- 172.21.15.60:0/563974478 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fedd4109a20 0x7fedd419f150 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:35:37.246 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:36.514+0000 7fedd9800700 1 -- 172.21.15.60:0/563974478 >> 172.21.15.60:0/563974478 conn(0x7fedd41033d0 msgr2=0x7fedd4103ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:35:37.246 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:36.514+0000 7fedd9800700 1 -- 172.21.15.60:0/563974478 shutdown_connections 2024-05-03T14:35:37.246 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:36.514+0000 7fedd9800700 1 -- 172.21.15.60:0/563974478 wait complete. 2024-05-03T14:35:37.247 INFO:teuthology.orchestra.run.smithi060.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-03T14:35:37.247 INFO:teuthology.orchestra.run.smithi060.stdout:Adding key to root@localhost authorized_keys... 2024-05-03T14:35:37.247 INFO:teuthology.orchestra.run.smithi060.stdout:Adding host smithi060... 2024-05-03T14:35:39.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:38 smithi060 ceph-mon[28873]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi060", "addr": "172.21.15.60", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:35:39.815 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:39 smithi060 ceph-mon[28873]: Deploying cephadm binary to smithi060 2024-05-03T14:35:45.091 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout Added host 'smithi060' with addr '172.21.15.60' 2024-05-03T14:35:45.091 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:37.845+0000 7f060e400700 1 Processor -- start 2024-05-03T14:35:45.091 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:37.845+0000 7f060e400700 1 -- start start 2024-05-03T14:35:45.091 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:37.846+0000 7f060e400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f060807ed60 0x7f060807d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:35:45.092 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:37.846+0000 7f060e400700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f060807f180 con 0x7f060807ed60 2024-05-03T14:35:45.092 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:37.846+0000 7f0607400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f060807ed60 0x7f060807d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:35:45.092 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:37.846+0000 7f0607400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f060807ed60 0x7f060807d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51042/0 (socket says 172.21.15.60:51042) 2024-05-03T14:35:45.092 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:37.846+0000 7f0607400700 1 -- 172.21.15.60:0/3815454525 learned_addr learned my addr 172.21.15.60:0/3815454525 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:35:45.092 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:37.847+0000 7f0607400700 1 -- 172.21.15.60:0/3815454525 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f060807d860 con 0x7f060807ed60 2024-05-03T14:35:45.092 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:37.847+0000 7f0607400700 1 --2- 172.21.15.60:0/3815454525 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f060807ed60 0x7f060807d320 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f05f80098a0 tx=0x7f05f8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc026460d2121934 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:35:45.092 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:37.848+0000 7f0606000700 1 -- 172.21.15.60:0/3815454525 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05f8004160 con 0x7f060807ed60 2024-05-03T14:35:45.092 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:37.848+0000 7f0606000700 1 -- 172.21.15.60:0/3815454525 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1139+0+0 (secure 0 0 0) 0x7f05f8004300 con 0x7f060807ed60 2024-05-03T14:35:45.092 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:37.848+0000 7f0606000700 1 -- 172.21.15.60:0/3815454525 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05f8011c00 con 0x7f060807ed60 2024-05-03T14:35:45.092 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:37.848+0000 7f060e400700 1 -- 172.21.15.60:0/3815454525 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f060807ed60 msgr2=0x7f060807d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:35:45.092 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:37.848+0000 7f060e400700 1 --2- 172.21.15.60:0/3815454525 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f060807ed60 0x7f060807d320 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f05f80098a0 tx=0x7f05f8009bb0 comp rx=0 tx=0).stop 2024-05-03T14:35:45.092 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:37.849+0000 7f060e400700 1 -- 172.21.15.60:0/3815454525 shutdown_connections 2024-05-03T14:35:45.092 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:37.849+0000 7f060e400700 1 --2- 172.21.15.60:0/3815454525 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f060807ed60 0x7f060807d320 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:35:45.093 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:37.849+0000 7f060e400700 1 -- 172.21.15.60:0/3815454525 >> 172.21.15.60:0/3815454525 conn(0x7f06081047b0 msgr2=0x7f0608106bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:35:45.093 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:37.849+0000 7f060e400700 1 -- 172.21.15.60:0/3815454525 shutdown_connections 2024-05-03T14:35:45.093 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:37.849+0000 7f060e400700 1 -- 172.21.15.60:0/3815454525 wait complete. 2024-05-03T14:35:45.093 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:37.850+0000 7f060e400700 1 Processor -- start 2024-05-03T14:35:45.093 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:37.850+0000 7f060e400700 1 -- start start 2024-05-03T14:35:45.093 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:37.850+0000 7f060e400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f060807ed60 0x7f060819f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:35:45.093 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:37.850+0000 7f060e400700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f060810f5e0 con 0x7f060807ed60 2024-05-03T14:35:45.093 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:37.851+0000 7f0607400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f060807ed60 0x7f060819f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:35:45.093 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:37.851+0000 7f0607400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f060807ed60 0x7f060819f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51046/0 (socket says 172.21.15.60:51046) 2024-05-03T14:35:45.093 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:37.851+0000 7f0607400700 1 -- 172.21.15.60:0/3470029912 learned_addr learned my addr 172.21.15.60:0/3470029912 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:35:45.093 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:37.851+0000 7f0607400700 1 -- 172.21.15.60:0/3470029912 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f060819f7f0 con 0x7f060807ed60 2024-05-03T14:35:45.093 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:37.851+0000 7f0607400700 1 --2- 172.21.15.60:0/3470029912 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f060807ed60 0x7f060819f2b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f05f8011bd0 tx=0x7f05f8011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:35:45.093 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:37.852+0000 7f05ffe00700 1 -- 172.21.15.60:0/3470029912 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05f8003b20 con 0x7f060807ed60 2024-05-03T14:35:45.093 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:37.852+0000 7f05ffe00700 1 -- 172.21.15.60:0/3470029912 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1139+0+0 (secure 0 0 0) 0x7f05f8003cc0 con 0x7f060807ed60 2024-05-03T14:35:45.094 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:37.852+0000 7f05ffe00700 1 -- 172.21.15.60:0/3470029912 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05f8024930 con 0x7f060807ed60 2024-05-03T14:35:45.094 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:37.852+0000 7f060e400700 1 -- 172.21.15.60:0/3470029912 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f060819fa80 con 0x7f060807ed60 2024-05-03T14:35:45.094 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:37.852+0000 7f060e400700 1 -- 172.21.15.60:0/3470029912 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06081a2770 con 0x7f060807ed60 2024-05-03T14:35:45.094 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:37.853+0000 7f060e400700 1 -- 172.21.15.60:0/3470029912 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0608064b00 con 0x7f060807ed60 2024-05-03T14:35:45.094 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:37.853+0000 7f05ffe00700 1 -- 172.21.15.60:0/3470029912 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f05f8022020 con 0x7f060807ed60 2024-05-03T14:35:45.094 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:37.854+0000 7f05ffe00700 1 --2- 172.21.15.60:0/3470029912 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f05ec03e4c0 0x7f05ec040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:35:45.094 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:37.854+0000 7f05ffe00700 1 -- 172.21.15.60:0/3470029912 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f05f80536f0 con 0x7f060807ed60 2024-05-03T14:35:45.094 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:37.854+0000 7f0606a00700 1 --2- 172.21.15.60:0/3470029912 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f05ec03e4c0 0x7f05ec040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:35:45.094 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:37.855+0000 7f0606a00700 1 --2- 172.21.15.60:0/3470029912 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f05ec03e4c0 0x7f05ec040980 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f05f4006fd0 tx=0x7f05f4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:35:45.094 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:37.859+0000 7f05ffe00700 1 -- 172.21.15.60:0/3470029912 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f05f801aa00 con 0x7f060807ed60 2024-05-03T14:35:45.094 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:38.102+0000 7f060e400700 1 -- 172.21.15.60:0/3470029912 --> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi060", "addr": "172.21.15.60", "target": ["mon-mgr", ""]}) -- 0x7f0608105a90 con 0x7f05ec03e4c0 2024-05-03T14:35:45.094 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:44.420+0000 7f05ffe00700 1 -- 172.21.15.60:0/3470029912 <== mgr.14120 v2:172.21.15.60:6800/424391417 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f0608105a90 con 0x7f05ec03e4c0 2024-05-03T14:35:45.095 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:44.424+0000 7f060e400700 1 -- 172.21.15.60:0/3470029912 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f05ec03e4c0 msgr2=0x7f05ec040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:35:45.095 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:44.424+0000 7f060e400700 1 --2- 172.21.15.60:0/3470029912 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f05ec03e4c0 0x7f05ec040980 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f05f4006fd0 tx=0x7f05f4006e40 comp rx=0 tx=0).stop 2024-05-03T14:35:45.095 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:44.424+0000 7f060e400700 1 -- 172.21.15.60:0/3470029912 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f060807ed60 msgr2=0x7f060819f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:35:45.095 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:44.424+0000 7f060e400700 1 --2- 172.21.15.60:0/3470029912 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f060807ed60 0x7f060819f2b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f05f8011bd0 tx=0x7f05f8011cb0 comp rx=0 tx=0).stop 2024-05-03T14:35:45.095 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:44.424+0000 7f060e400700 1 -- 172.21.15.60:0/3470029912 shutdown_connections 2024-05-03T14:35:45.095 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:44.424+0000 7f060e400700 1 --2- 172.21.15.60:0/3470029912 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f05ec03e4c0 0x7f05ec040980 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:35:45.095 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:44.424+0000 7f060e400700 1 --2- 172.21.15.60:0/3470029912 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f060807ed60 0x7f060819f2b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:35:45.095 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:44.424+0000 7f060e400700 1 -- 172.21.15.60:0/3470029912 >> 172.21.15.60:0/3470029912 conn(0x7f06081047b0 msgr2=0x7f0608105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:35:45.095 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:44.424+0000 7f060e400700 1 -- 172.21.15.60:0/3470029912 shutdown_connections 2024-05-03T14:35:45.095 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:44.425+0000 7f060e400700 1 -- 172.21.15.60:0/3470029912 wait complete. 2024-05-03T14:35:45.095 INFO:teuthology.orchestra.run.smithi060.stdout:Deploying unmanaged mon service... 2024-05-03T14:35:45.719 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:45 smithi060 ceph-mon[28873]: from='mgr.14120 172.21.15.60:0/1113217389' entity='mgr.x' 2024-05-03T14:35:45.719 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:45 smithi060 ceph-mon[28873]: Added host smithi060 2024-05-03T14:35:45.719 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:45 smithi060 ceph-mon[28873]: from='mgr.14120 172.21.15.60:0/1113217389' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:35:46.679 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-03T14:35:46.680 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:45.784+0000 7f545fc00700 1 Processor -- start 2024-05-03T14:35:46.680 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:45.784+0000 7f545fc00700 1 -- start start 2024-05-03T14:35:46.680 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:45.784+0000 7f545fc00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f545810b2d0 0x7f545810b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:35:46.680 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:45.784+0000 7f545fc00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f545810bcc0 con 0x7f545810b2d0 2024-05-03T14:35:46.680 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:45.785+0000 7f545d200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f545810b2d0 0x7f545810b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:35:46.680 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:45.785+0000 7f545d200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f545810b2d0 0x7f545810b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40836/0 (socket says 172.21.15.60:40836) 2024-05-03T14:35:46.680 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:45.785+0000 7f545d200700 1 -- 172.21.15.60:0/2210663480 learned_addr learned my addr 172.21.15.60:0/2210663480 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:35:46.680 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:45.785+0000 7f545d200700 1 -- 172.21.15.60:0/2210663480 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f545810c480 con 0x7f545810b2d0 2024-05-03T14:35:46.680 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:45.786+0000 7f545d200700 1 --2- 172.21.15.60:0/2210663480 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f545810b2d0 0x7f545810b6f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f544c0098a0 tx=0x7f544c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b123b07977a1f283 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:35:46.680 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:45.786+0000 7f5457400700 1 -- 172.21.15.60:0/2210663480 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f544c004160 con 0x7f545810b2d0 2024-05-03T14:35:46.680 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:45.786+0000 7f5457400700 1 -- 172.21.15.60:0/2210663480 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1139+0+0 (secure 0 0 0) 0x7f544c004300 con 0x7f545810b2d0 2024-05-03T14:35:46.680 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:45.786+0000 7f5457400700 1 -- 172.21.15.60:0/2210663480 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f544c011c00 con 0x7f545810b2d0 2024-05-03T14:35:46.681 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:45.786+0000 7f545fc00700 1 -- 172.21.15.60:0/2210663480 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f545810b2d0 msgr2=0x7f545810b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:35:46.681 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:45.787+0000 7f545fc00700 1 --2- 172.21.15.60:0/2210663480 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f545810b2d0 0x7f545810b6f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f544c0098a0 tx=0x7f544c009bb0 comp rx=0 tx=0).stop 2024-05-03T14:35:46.681 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:45.787+0000 7f545fc00700 1 -- 172.21.15.60:0/2210663480 shutdown_connections 2024-05-03T14:35:46.681 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:45.787+0000 7f545fc00700 1 --2- 172.21.15.60:0/2210663480 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f545810b2d0 0x7f545810b6f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:35:46.681 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:45.787+0000 7f545fc00700 1 -- 172.21.15.60:0/2210663480 >> 172.21.15.60:0/2210663480 conn(0x7f545807e900 msgr2=0x7f54581094f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:35:46.681 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:45.787+0000 7f545fc00700 1 -- 172.21.15.60:0/2210663480 shutdown_connections 2024-05-03T14:35:46.681 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:45.787+0000 7f545fc00700 1 -- 172.21.15.60:0/2210663480 wait complete. 2024-05-03T14:35:46.681 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:45.788+0000 7f545fc00700 1 Processor -- start 2024-05-03T14:35:46.681 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:45.788+0000 7f545fc00700 1 -- start start 2024-05-03T14:35:46.681 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:45.788+0000 7f545fc00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f545810b2d0 0x7f545819f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:35:46.681 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:45.788+0000 7f545fc00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f545810f830 con 0x7f545810b2d0 2024-05-03T14:35:46.681 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:45.789+0000 7f545d200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f545810b2d0 0x7f545819f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:35:46.681 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:45.789+0000 7f545d200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f545810b2d0 0x7f545819f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40838/0 (socket says 172.21.15.60:40838) 2024-05-03T14:35:46.681 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:45.789+0000 7f545d200700 1 -- 172.21.15.60:0/487763537 learned_addr learned my addr 172.21.15.60:0/487763537 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:35:46.682 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:45.789+0000 7f545d200700 1 -- 172.21.15.60:0/487763537 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f545819f7e0 con 0x7f545810b2d0 2024-05-03T14:35:46.682 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:45.789+0000 7f545d200700 1 --2- 172.21.15.60:0/487763537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f545810b2d0 0x7f545819f2a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f544c000c00 tx=0x7f544c011c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:35:46.682 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:45.790+0000 7f5455600700 1 -- 172.21.15.60:0/487763537 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f544c003b20 con 0x7f545810b2d0 2024-05-03T14:35:46.682 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:45.790+0000 7f5455600700 1 -- 172.21.15.60:0/487763537 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1139+0+0 (secure 0 0 0) 0x7f544c003cc0 con 0x7f545810b2d0 2024-05-03T14:35:46.682 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:45.790+0000 7f545fc00700 1 -- 172.21.15.60:0/487763537 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f545819fa70 con 0x7f545810b2d0 2024-05-03T14:35:46.682 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:45.790+0000 7f5455600700 1 -- 172.21.15.60:0/487763537 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f544c024970 con 0x7f545810b2d0 2024-05-03T14:35:46.682 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:45.790+0000 7f545fc00700 1 -- 172.21.15.60:0/487763537 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54581a2760 con 0x7f545810b2d0 2024-05-03T14:35:46.682 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:45.791+0000 7f545fc00700 1 -- 172.21.15.60:0/487763537 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f545810f9b0 con 0x7f545810b2d0 2024-05-03T14:35:46.682 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:45.791+0000 7f5455600700 1 -- 172.21.15.60:0/487763537 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f544c022020 con 0x7f545810b2d0 2024-05-03T14:35:46.682 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:45.792+0000 7f5455600700 1 --2- 172.21.15.60:0/487763537 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f544003e4c0 0x7f5440040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:35:46.682 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:45.792+0000 7f5455600700 1 -- 172.21.15.60:0/487763537 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f544c0536e0 con 0x7f545810b2d0 2024-05-03T14:35:46.682 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:45.792+0000 7f5457e00700 1 --2- 172.21.15.60:0/487763537 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f544003e4c0 0x7f5440040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:35:46.683 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:45.793+0000 7f5457e00700 1 --2- 172.21.15.60:0/487763537 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f544003e4c0 0x7f5440040980 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5448006fd0 tx=0x7f5448006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:35:46.683 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:45.798+0000 7f5455600700 1 -- 172.21.15.60:0/487763537 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f544c00f2e0 con 0x7f545810b2d0 2024-05-03T14:35:46.683 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:46.056+0000 7f545fc00700 1 -- 172.21.15.60:0/487763537 --> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f54581094b0 con 0x7f544003e4c0 2024-05-03T14:35:46.683 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:46.062+0000 7f5455600700 1 -- 172.21.15.60:0/487763537 <== mgr.14120 v2:172.21.15.60:6800/424391417 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f54581094b0 con 0x7f544003e4c0 2024-05-03T14:35:46.683 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:46.065+0000 7f545fc00700 1 -- 172.21.15.60:0/487763537 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f544003e4c0 msgr2=0x7f5440040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:35:46.683 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:46.065+0000 7f545fc00700 1 --2- 172.21.15.60:0/487763537 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f544003e4c0 0x7f5440040980 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5448006fd0 tx=0x7f5448006e40 comp rx=0 tx=0).stop 2024-05-03T14:35:46.683 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:46.065+0000 7f545fc00700 1 -- 172.21.15.60:0/487763537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f545810b2d0 msgr2=0x7f545819f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:35:46.683 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:46.065+0000 7f545fc00700 1 --2- 172.21.15.60:0/487763537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f545810b2d0 0x7f545819f2a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f544c000c00 tx=0x7f544c011c80 comp rx=0 tx=0).stop 2024-05-03T14:35:46.683 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:46.065+0000 7f545fc00700 1 -- 172.21.15.60:0/487763537 shutdown_connections 2024-05-03T14:35:46.683 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:46.065+0000 7f545fc00700 1 --2- 172.21.15.60:0/487763537 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f544003e4c0 0x7f5440040980 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:35:46.683 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:46.065+0000 7f545fc00700 1 --2- 172.21.15.60:0/487763537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f545810b2d0 0x7f545819f2a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:35:46.683 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:46.065+0000 7f545fc00700 1 -- 172.21.15.60:0/487763537 >> 172.21.15.60:0/487763537 conn(0x7f545807e900 msgr2=0x7f5458108da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:35:46.683 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:46.065+0000 7f545fc00700 1 -- 172.21.15.60:0/487763537 shutdown_connections 2024-05-03T14:35:46.683 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:46.065+0000 7f545fc00700 1 -- 172.21.15.60:0/487763537 wait complete. 2024-05-03T14:35:46.684 INFO:teuthology.orchestra.run.smithi060.stdout:Deploying unmanaged mgr service... 2024-05-03T14:35:47.313 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:47 smithi060 ceph-mon[28873]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:35:47.314 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:47 smithi060 ceph-mon[28873]: Saving service mon spec with placement count:5 2024-05-03T14:35:47.314 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:47 smithi060 ceph-mon[28873]: from='mgr.14120 172.21.15.60:0/1113217389' entity='mgr.x' 2024-05-03T14:35:48.545 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-03T14:35:48.545 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:47.358+0000 7f4200000700 1 Processor -- start 2024-05-03T14:35:48.545 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:47.358+0000 7f4200000700 1 -- start start 2024-05-03T14:35:48.545 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:47.358+0000 7f4200000700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41f810fec0 0x7f41f81122b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:35:48.545 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:47.359+0000 7f4200000700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f41f8112880 con 0x7f41f810fec0 2024-05-03T14:35:48.546 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:47.359+0000 7f41fd600700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41f810fec0 0x7f41f81122b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:35:48.546 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:47.359+0000 7f41fd600700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41f810fec0 0x7f41f81122b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40840/0 (socket says 172.21.15.60:40840) 2024-05-03T14:35:48.546 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:47.359+0000 7f41fd600700 1 -- 172.21.15.60:0/1074100807 learned_addr learned my addr 172.21.15.60:0/1074100807 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:35:48.546 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:47.359+0000 7f41fd600700 1 -- 172.21.15.60:0/1074100807 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41f8112a00 con 0x7f41f810fec0 2024-05-03T14:35:48.546 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:47.360+0000 7f41fd600700 1 --2- 172.21.15.60:0/1074100807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41f810fec0 0x7f41f81122b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f41ec0098a0 tx=0x7f41ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3facd08dd6eac00 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:35:48.546 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:47.360+0000 7f41f7e00700 1 -- 172.21.15.60:0/1074100807 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41ec004160 con 0x7f41f810fec0 2024-05-03T14:35:48.546 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:47.360+0000 7f41f7e00700 1 -- 172.21.15.60:0/1074100807 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1139+0+0 (secure 0 0 0) 0x7f41ec004300 con 0x7f41f810fec0 2024-05-03T14:35:48.546 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:47.360+0000 7f41f7e00700 1 -- 172.21.15.60:0/1074100807 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41ec011c00 con 0x7f41f810fec0 2024-05-03T14:35:48.546 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:47.361+0000 7f4200000700 1 -- 172.21.15.60:0/1074100807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41f810fec0 msgr2=0x7f41f81122b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:35:48.546 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:47.361+0000 7f4200000700 1 --2- 172.21.15.60:0/1074100807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41f810fec0 0x7f41f81122b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f41ec0098a0 tx=0x7f41ec009bb0 comp rx=0 tx=0).stop 2024-05-03T14:35:48.546 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:47.361+0000 7f4200000700 1 -- 172.21.15.60:0/1074100807 shutdown_connections 2024-05-03T14:35:48.546 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:47.361+0000 7f4200000700 1 --2- 172.21.15.60:0/1074100807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41f810fec0 0x7f41f81122b0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:35:48.547 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:47.361+0000 7f4200000700 1 -- 172.21.15.60:0/1074100807 >> 172.21.15.60:0/1074100807 conn(0x7f41f807e900 msgr2=0x7f41f8080d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:35:48.547 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:47.361+0000 7f4200000700 1 -- 172.21.15.60:0/1074100807 shutdown_connections 2024-05-03T14:35:48.547 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:47.361+0000 7f4200000700 1 -- 172.21.15.60:0/1074100807 wait complete. 2024-05-03T14:35:48.547 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:47.362+0000 7f4200000700 1 Processor -- start 2024-05-03T14:35:48.547 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:47.362+0000 7f4200000700 1 -- start start 2024-05-03T14:35:48.547 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:47.363+0000 7f4200000700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41f810fec0 0x7f41f81a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:35:48.547 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:47.363+0000 7f4200000700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f41f8113a70 con 0x7f41f810fec0 2024-05-03T14:35:48.547 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:47.363+0000 7f41fd600700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41f810fec0 0x7f41f81a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:35:48.547 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:47.363+0000 7f41fd600700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41f810fec0 0x7f41f81a3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40854/0 (socket says 172.21.15.60:40854) 2024-05-03T14:35:48.547 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:47.363+0000 7f41fd600700 1 -- 172.21.15.60:0/4119305054 learned_addr learned my addr 172.21.15.60:0/4119305054 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:35:48.547 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:47.363+0000 7f41fd600700 1 -- 172.21.15.60:0/4119305054 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41f81a3ba0 con 0x7f41f810fec0 2024-05-03T14:35:48.547 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:47.364+0000 7f41fd600700 1 --2- 172.21.15.60:0/4119305054 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41f810fec0 0x7f41f81a3660 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f41ec009fb0 tx=0x7f41ec011c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:35:48.547 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:47.364+0000 7f41f6000700 1 -- 172.21.15.60:0/4119305054 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41ec0038b0 con 0x7f41f810fec0 2024-05-03T14:35:48.547 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:47.364+0000 7f41f6000700 1 -- 172.21.15.60:0/4119305054 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1139+0+0 (secure 0 0 0) 0x7f41ec01a430 con 0x7f41f810fec0 2024-05-03T14:35:48.548 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:47.364+0000 7f4200000700 1 -- 172.21.15.60:0/4119305054 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41f81a3e90 con 0x7f41f810fec0 2024-05-03T14:35:48.548 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:47.364+0000 7f41f6000700 1 -- 172.21.15.60:0/4119305054 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41ec019590 con 0x7f41f810fec0 2024-05-03T14:35:48.548 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:47.364+0000 7f4200000700 1 -- 172.21.15.60:0/4119305054 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41f81a6b20 con 0x7f41f810fec0 2024-05-03T14:35:48.548 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:47.365+0000 7f4200000700 1 -- 172.21.15.60:0/4119305054 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41f8064b00 con 0x7f41f810fec0 2024-05-03T14:35:48.548 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:47.365+0000 7f41f6000700 1 -- 172.21.15.60:0/4119305054 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f41ec01a9f0 con 0x7f41f810fec0 2024-05-03T14:35:48.548 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:47.366+0000 7f41f6000700 1 --2- 172.21.15.60:0/4119305054 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f41e003e470 0x7f41e0040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:35:48.548 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:47.366+0000 7f41f6000700 1 -- 172.21.15.60:0/4119305054 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f41ec053a00 con 0x7f41f810fec0 2024-05-03T14:35:48.548 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:47.366+0000 7f41fcc00700 1 --2- 172.21.15.60:0/4119305054 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f41e003e470 0x7f41e0040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:35:48.548 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:47.367+0000 7f41fcc00700 1 --2- 172.21.15.60:0/4119305054 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f41e003e470 0x7f41e0040930 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f41e4006fd0 tx=0x7f41e4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:35:48.548 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:47.372+0000 7f41f6000700 1 -- 172.21.15.60:0/4119305054 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f41ec01a690 con 0x7f41f810fec0 2024-05-03T14:35:48.548 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:47.635+0000 7f4200000700 1 -- 172.21.15.60:0/4119305054 --> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f41f81a6cf0 con 0x7f41e003e470 2024-05-03T14:35:48.548 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:47.643+0000 7f41f6000700 1 -- 172.21.15.60:0/4119305054 <== mgr.14120 v2:172.21.15.60:6800/424391417 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f41f81a6cf0 con 0x7f41e003e470 2024-05-03T14:35:48.548 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:47.646+0000 7f4200000700 1 -- 172.21.15.60:0/4119305054 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f41e003e470 msgr2=0x7f41e0040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:35:48.548 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:47.646+0000 7f4200000700 1 --2- 172.21.15.60:0/4119305054 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f41e003e470 0x7f41e0040930 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f41e4006fd0 tx=0x7f41e4006e40 comp rx=0 tx=0).stop 2024-05-03T14:35:48.548 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:47.646+0000 7f4200000700 1 -- 172.21.15.60:0/4119305054 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41f810fec0 msgr2=0x7f41f81a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:35:48.549 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:47.646+0000 7f4200000700 1 --2- 172.21.15.60:0/4119305054 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41f810fec0 0x7f41f81a3660 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f41ec009fb0 tx=0x7f41ec011c80 comp rx=0 tx=0).stop 2024-05-03T14:35:48.549 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:47.646+0000 7f4200000700 1 -- 172.21.15.60:0/4119305054 shutdown_connections 2024-05-03T14:35:48.549 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:47.646+0000 7f4200000700 1 --2- 172.21.15.60:0/4119305054 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f41e003e470 0x7f41e0040930 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:35:48.549 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:47.646+0000 7f4200000700 1 --2- 172.21.15.60:0/4119305054 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41f810fec0 0x7f41f81a3660 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:35:48.549 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:47.646+0000 7f4200000700 1 -- 172.21.15.60:0/4119305054 >> 172.21.15.60:0/4119305054 conn(0x7f41f807e900 msgr2=0x7f41f8080510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:35:48.549 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:47.647+0000 7f4200000700 1 -- 172.21.15.60:0/4119305054 shutdown_connections 2024-05-03T14:35:48.549 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:47.647+0000 7f4200000700 1 -- 172.21.15.60:0/4119305054 wait complete. 2024-05-03T14:35:48.805 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:48 smithi060 ceph-mon[28873]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:35:48.805 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:48 smithi060 ceph-mon[28873]: Saving service mgr spec with placement count:2 2024-05-03T14:35:48.805 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:48 smithi060 ceph-mon[28873]: from='mgr.14120 172.21.15.60:0/1113217389' entity='mgr.x' 2024-05-03T14:35:49.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:49 smithi060 ceph-mon[28873]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:35:49.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:49 smithi060 ceph-mon[28873]: from='mgr.14120 172.21.15.60:0/1113217389' entity='mgr.x' 2024-05-03T14:35:50.618 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:49.647+0000 7f1324c00700 1 Processor -- start 2024-05-03T14:35:50.618 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:49.647+0000 7f1324c00700 1 -- start start 2024-05-03T14:35:50.619 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:49.647+0000 7f1324c00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1320109bc0 0x7f132010bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:35:50.619 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:49.647+0000 7f1324c00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f132007ce50 con 0x7f1320109bc0 2024-05-03T14:35:50.619 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:49.647+0000 7f131de00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1320109bc0 0x7f132010bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:35:50.619 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:49.647+0000 7f131de00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1320109bc0 0x7f132010bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40870/0 (socket says 172.21.15.60:40870) 2024-05-03T14:35:50.619 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:49.647+0000 7f131de00700 1 -- 172.21.15.60:0/1558096283 learned_addr learned my addr 172.21.15.60:0/1558096283 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:35:50.619 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:49.648+0000 7f131de00700 1 -- 172.21.15.60:0/1558096283 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f132010c4f0 con 0x7f1320109bc0 2024-05-03T14:35:50.619 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:49.648+0000 7f131de00700 1 --2- 172.21.15.60:0/1558096283 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1320109bc0 0x7f132010bfb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1310009e80 tx=0x7f131000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b7eb71a0a53fbd7 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:35:50.619 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:49.649+0000 7f131d400700 1 -- 172.21.15.60:0/1558096283 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1310017070 con 0x7f1320109bc0 2024-05-03T14:35:50.619 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:49.649+0000 7f131d400700 1 -- 172.21.15.60:0/1558096283 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f131000b8c0 con 0x7f1320109bc0 2024-05-03T14:35:50.619 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:49.649+0000 7f131d400700 1 -- 172.21.15.60:0/1558096283 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1310004580 con 0x7f1320109bc0 2024-05-03T14:35:50.619 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:49.649+0000 7f1324c00700 1 -- 172.21.15.60:0/1558096283 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1320109bc0 msgr2=0x7f132010bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:35:50.619 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:49.649+0000 7f1324c00700 1 --2- 172.21.15.60:0/1558096283 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1320109bc0 0x7f132010bfb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1310009e80 tx=0x7f131000b330 comp rx=0 tx=0).stop 2024-05-03T14:35:50.619 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:49.650+0000 7f1324c00700 1 -- 172.21.15.60:0/1558096283 shutdown_connections 2024-05-03T14:35:50.620 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:49.650+0000 7f1324c00700 1 --2- 172.21.15.60:0/1558096283 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1320109bc0 0x7f132010bfb0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:35:50.620 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:49.650+0000 7f1324c00700 1 -- 172.21.15.60:0/1558096283 >> 172.21.15.60:0/1558096283 conn(0x7f13201035b0 msgr2=0x7f13201059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:35:50.620 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:49.650+0000 7f1324c00700 1 -- 172.21.15.60:0/1558096283 shutdown_connections 2024-05-03T14:35:50.620 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:49.650+0000 7f1324c00700 1 -- 172.21.15.60:0/1558096283 wait complete. 2024-05-03T14:35:50.620 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:49.651+0000 7f1324c00700 1 Processor -- start 2024-05-03T14:35:50.620 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:49.651+0000 7f1324c00700 1 -- start start 2024-05-03T14:35:50.620 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:49.652+0000 7f1324c00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1320109bc0 0x7f1320109440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:35:50.620 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:49.652+0000 7f1324c00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f132010d550 con 0x7f1320109bc0 2024-05-03T14:35:50.620 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:49.652+0000 7f131de00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1320109bc0 0x7f1320109440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:35:50.620 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:49.652+0000 7f131de00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1320109bc0 0x7f1320109440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40878/0 (socket says 172.21.15.60:40878) 2024-05-03T14:35:50.620 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:49.652+0000 7f131de00700 1 -- 172.21.15.60:0/1417946328 learned_addr learned my addr 172.21.15.60:0/1417946328 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:35:50.620 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:49.652+0000 7f131de00700 1 -- 172.21.15.60:0/1417946328 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1320109980 con 0x7f1320109bc0 2024-05-03T14:35:50.620 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:49.653+0000 7f131de00700 1 --2- 172.21.15.60:0/1417946328 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1320109bc0 0x7f1320109440 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f131000b7b0 tx=0x7f1310003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:35:50.620 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:49.653+0000 7f1316a00700 1 -- 172.21.15.60:0/1417946328 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1310021020 con 0x7f1320109bc0 2024-05-03T14:35:50.620 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:49.653+0000 7f1324c00700 1 -- 172.21.15.60:0/1417946328 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1320107b70 con 0x7f1320109bc0 2024-05-03T14:35:50.621 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:49.653+0000 7f1316a00700 1 -- 172.21.15.60:0/1417946328 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f131000be00 con 0x7f1320109bc0 2024-05-03T14:35:50.621 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:49.653+0000 7f1316a00700 1 -- 172.21.15.60:0/1417946328 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1310017040 con 0x7f1320109bc0 2024-05-03T14:35:50.621 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:49.653+0000 7f1324c00700 1 -- 172.21.15.60:0/1417946328 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13201080b0 con 0x7f1320109bc0 2024-05-03T14:35:50.621 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:49.654+0000 7f1324c00700 1 -- 172.21.15.60:0/1417946328 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1320064b00 con 0x7f1320109bc0 2024-05-03T14:35:50.621 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:49.655+0000 7f1316a00700 1 -- 172.21.15.60:0/1417946328 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f1310024020 con 0x7f1320109bc0 2024-05-03T14:35:50.621 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:49.655+0000 7f1316a00700 1 --2- 172.21.15.60:0/1417946328 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f130403e470 0x7f1304040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:35:50.621 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:49.655+0000 7f1316a00700 1 -- 172.21.15.60:0/1417946328 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f131000f050 con 0x7f1320109bc0 2024-05-03T14:35:50.621 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:49.655+0000 7f1317e00700 1 --2- 172.21.15.60:0/1417946328 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f130403e470 0x7f1304040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:35:50.621 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:49.656+0000 7f1317e00700 1 --2- 172.21.15.60:0/1417946328 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f130403e470 0x7f1304040930 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1308006fd0 tx=0x7f1308006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:35:50.621 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:49.661+0000 7f1316a00700 1 -- 172.21.15.60:0/1417946328 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f131002b020 con 0x7f1320109bc0 2024-05-03T14:35:50.621 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:49.910+0000 7f1324c00700 1 -- 172.21.15.60:0/1417946328 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f1320108280 con 0x7f1320109bc0 2024-05-03T14:35:50.621 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:49.916+0000 7f1316a00700 1 -- 172.21.15.60:0/1417946328 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f131000a040 con 0x7f1320109bc0 2024-05-03T14:35:50.621 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:49.919+0000 7f1324c00700 1 -- 172.21.15.60:0/1417946328 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f130403e470 msgr2=0x7f1304040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:35:50.621 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:49.919+0000 7f1324c00700 1 --2- 172.21.15.60:0/1417946328 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f130403e470 0x7f1304040930 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1308006fd0 tx=0x7f1308006e40 comp rx=0 tx=0).stop 2024-05-03T14:35:50.621 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:49.919+0000 7f1324c00700 1 -- 172.21.15.60:0/1417946328 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1320109bc0 msgr2=0x7f1320109440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:35:50.622 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:49.919+0000 7f1324c00700 1 --2- 172.21.15.60:0/1417946328 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1320109bc0 0x7f1320109440 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f131000b7b0 tx=0x7f1310003d00 comp rx=0 tx=0).stop 2024-05-03T14:35:50.622 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:49.920+0000 7f1324c00700 1 -- 172.21.15.60:0/1417946328 shutdown_connections 2024-05-03T14:35:50.622 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:49.920+0000 7f1324c00700 1 --2- 172.21.15.60:0/1417946328 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f130403e470 0x7f1304040930 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:35:50.622 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:49.920+0000 7f1324c00700 1 --2- 172.21.15.60:0/1417946328 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1320109bc0 0x7f1320109440 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:35:50.622 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:49.920+0000 7f1324c00700 1 -- 172.21.15.60:0/1417946328 >> 172.21.15.60:0/1417946328 conn(0x7f13201035b0 msgr2=0x7f1320104070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:35:50.622 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:49.920+0000 7f1324c00700 1 -- 172.21.15.60:0/1417946328 shutdown_connections 2024-05-03T14:35:50.622 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:49.920+0000 7f1324c00700 1 -- 172.21.15.60:0/1417946328 wait complete. 2024-05-03T14:35:51.217 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:50 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/1417946328' entity='client.admin' 2024-05-03T14:35:51.217 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:50 smithi060 ceph-mon[28873]: from='mgr.14120 172.21.15.60:0/1113217389' entity='mgr.x' 2024-05-03T14:35:51.217 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:50 smithi060 ceph-mon[28873]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:35:52.267 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:51.213+0000 7f1311600700 1 Processor -- start 2024-05-03T14:35:52.267 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:51.213+0000 7f1311600700 1 -- start start 2024-05-03T14:35:52.267 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:51.213+0000 7f1311600700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f130c07edf0 0x7f130c07d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:35:52.267 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:51.213+0000 7f1311600700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f130c07d8a0 con 0x7f130c07edf0 2024-05-03T14:35:52.267 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:51.214+0000 7f130aa00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f130c07edf0 0x7f130c07d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:35:52.267 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:51.214+0000 7f130aa00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f130c07edf0 0x7f130c07d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40886/0 (socket says 172.21.15.60:40886) 2024-05-03T14:35:52.267 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:51.214+0000 7f130aa00700 1 -- 172.21.15.60:0/2834235140 learned_addr learned my addr 172.21.15.60:0/2834235140 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:35:52.267 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:51.214+0000 7f130aa00700 1 -- 172.21.15.60:0/2834235140 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f130c07da20 con 0x7f130c07edf0 2024-05-03T14:35:52.267 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:51.215+0000 7f130aa00700 1 --2- 172.21.15.60:0/2834235140 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f130c07edf0 0x7f130c07d360 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f12fc0098a0 tx=0x7f12fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76187d5dab3d188e server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:35:52.268 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:51.215+0000 7f1309600700 1 -- 172.21.15.60:0/2834235140 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f12fc004160 con 0x7f130c07edf0 2024-05-03T14:35:52.268 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:51.215+0000 7f1309600700 1 -- 172.21.15.60:0/2834235140 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f12fc004300 con 0x7f130c07edf0 2024-05-03T14:35:52.268 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:51.215+0000 7f1309600700 1 -- 172.21.15.60:0/2834235140 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f12fc011c20 con 0x7f130c07edf0 2024-05-03T14:35:52.268 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:51.216+0000 7f1311600700 1 -- 172.21.15.60:0/2834235140 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f130c07edf0 msgr2=0x7f130c07d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:35:52.268 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:51.216+0000 7f1311600700 1 --2- 172.21.15.60:0/2834235140 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f130c07edf0 0x7f130c07d360 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f12fc0098a0 tx=0x7f12fc009bb0 comp rx=0 tx=0).stop 2024-05-03T14:35:52.268 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:51.216+0000 7f1311600700 1 -- 172.21.15.60:0/2834235140 shutdown_connections 2024-05-03T14:35:52.268 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:51.216+0000 7f1311600700 1 --2- 172.21.15.60:0/2834235140 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f130c07edf0 0x7f130c07d360 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:35:52.268 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:51.216+0000 7f1311600700 1 -- 172.21.15.60:0/2834235140 >> 172.21.15.60:0/2834235140 conn(0x7f130c104750 msgr2=0x7f130c106b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:35:52.268 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:51.216+0000 7f1311600700 1 -- 172.21.15.60:0/2834235140 shutdown_connections 2024-05-03T14:35:52.268 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:51.216+0000 7f1311600700 1 -- 172.21.15.60:0/2834235140 wait complete. 2024-05-03T14:35:52.268 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:51.217+0000 7f1311600700 1 Processor -- start 2024-05-03T14:35:52.268 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:51.217+0000 7f1311600700 1 -- start start 2024-05-03T14:35:52.268 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:51.218+0000 7f1311600700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f130c07edf0 0x7f130c19f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:35:52.268 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:51.218+0000 7f1311600700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f130c10f660 con 0x7f130c07edf0 2024-05-03T14:35:52.268 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:51.218+0000 7f130aa00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f130c07edf0 0x7f130c19f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:35:52.269 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:51.218+0000 7f130aa00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f130c07edf0 0x7f130c19f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40894/0 (socket says 172.21.15.60:40894) 2024-05-03T14:35:52.269 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:51.218+0000 7f130aa00700 1 -- 172.21.15.60:0/3393307396 learned_addr learned my addr 172.21.15.60:0/3393307396 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:35:52.269 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:51.218+0000 7f130aa00700 1 -- 172.21.15.60:0/3393307396 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f130c19f810 con 0x7f130c07edf0 2024-05-03T14:35:52.269 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:51.219+0000 7f130aa00700 1 --2- 172.21.15.60:0/3393307396 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f130c07edf0 0x7f130c19f2d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f12fc011bf0 tx=0x7f12fc011cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:35:52.269 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:51.219+0000 7f1303400700 1 -- 172.21.15.60:0/3393307396 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f12fc0038b0 con 0x7f130c07edf0 2024-05-03T14:35:52.269 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:51.219+0000 7f1303400700 1 -- 172.21.15.60:0/3393307396 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f12fc003a50 con 0x7f130c07edf0 2024-05-03T14:35:52.269 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:51.219+0000 7f1303400700 1 -- 172.21.15.60:0/3393307396 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f12fc0243f0 con 0x7f130c07edf0 2024-05-03T14:35:52.269 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:51.219+0000 7f1311600700 1 -- 172.21.15.60:0/3393307396 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f130c19faa0 con 0x7f130c07edf0 2024-05-03T14:35:52.269 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:51.219+0000 7f1311600700 1 -- 172.21.15.60:0/3393307396 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f130c1a2790 con 0x7f130c07edf0 2024-05-03T14:35:52.269 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:51.220+0000 7f1311600700 1 -- 172.21.15.60:0/3393307396 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f130c064b00 con 0x7f130c07edf0 2024-05-03T14:35:52.269 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:51.221+0000 7f1303400700 1 -- 172.21.15.60:0/3393307396 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f12fc003c00 con 0x7f130c07edf0 2024-05-03T14:35:52.269 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:51.221+0000 7f1303400700 1 --2- 172.21.15.60:0/3393307396 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f12f003e470 0x7f12f0040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:35:52.269 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:51.221+0000 7f1303400700 1 -- 172.21.15.60:0/3393307396 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f12fc0535c0 con 0x7f130c07edf0 2024-05-03T14:35:52.269 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:51.222+0000 7f130a000700 1 --2- 172.21.15.60:0/3393307396 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f12f003e470 0x7f12f0040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:35:52.270 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:51.222+0000 7f130a000700 1 --2- 172.21.15.60:0/3393307396 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f12f003e470 0x7f12f0040930 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f12f8006fd0 tx=0x7f12f8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:35:52.270 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:51.227+0000 7f1303400700 1 -- 172.21.15.60:0/3393307396 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f12fc01a430 con 0x7f130c07edf0 2024-05-03T14:35:52.270 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:51.487+0000 7f1311600700 1 -- 172.21.15.60:0/3393307396 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f130c1a2960 con 0x7f130c07edf0 2024-05-03T14:35:52.270 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:51.495+0000 7f1303400700 1 -- 172.21.15.60:0/3393307396 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f12fc01a760 con 0x7f130c07edf0 2024-05-03T14:35:52.270 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:51.499+0000 7f1311600700 1 -- 172.21.15.60:0/3393307396 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f12f003e470 msgr2=0x7f12f0040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:35:52.270 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:51.499+0000 7f1311600700 1 --2- 172.21.15.60:0/3393307396 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f12f003e470 0x7f12f0040930 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f12f8006fd0 tx=0x7f12f8006e40 comp rx=0 tx=0).stop 2024-05-03T14:35:52.270 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:51.499+0000 7f1311600700 1 -- 172.21.15.60:0/3393307396 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f130c07edf0 msgr2=0x7f130c19f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:35:52.270 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:51.499+0000 7f1311600700 1 --2- 172.21.15.60:0/3393307396 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f130c07edf0 0x7f130c19f2d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f12fc011bf0 tx=0x7f12fc011cd0 comp rx=0 tx=0).stop 2024-05-03T14:35:52.270 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:51.499+0000 7f1311600700 1 -- 172.21.15.60:0/3393307396 shutdown_connections 2024-05-03T14:35:52.270 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:51.499+0000 7f1311600700 1 --2- 172.21.15.60:0/3393307396 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f12f003e470 0x7f12f0040930 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:35:52.270 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:51.499+0000 7f1311600700 1 --2- 172.21.15.60:0/3393307396 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f130c07edf0 0x7f130c19f2d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:35:52.270 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:51.499+0000 7f1311600700 1 -- 172.21.15.60:0/3393307396 >> 172.21.15.60:0/3393307396 conn(0x7f130c104750 msgr2=0x7f130c1052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:35:52.270 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:51.500+0000 7f1311600700 1 -- 172.21.15.60:0/3393307396 shutdown_connections 2024-05-03T14:35:52.270 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:51.500+0000 7f1311600700 1 -- 172.21.15.60:0/3393307396 wait complete. 2024-05-03T14:35:52.271 INFO:teuthology.orchestra.run.smithi060.stdout:Enabling the dashboard module... 2024-05-03T14:35:52.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:52 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/3393307396' entity='client.admin' 2024-05-03T14:35:53.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:53 smithi060 ceph-mon[28873]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:35:53.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:53 smithi060 ceph-mon[28873]: from='mgr.14120 172.21.15.60:0/1113217389' entity='mgr.x' 2024-05-03T14:35:54.833 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:54 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/3217630771' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-03T14:35:54.833 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:54 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: ignoring --setuser ceph since I am not root 2024-05-03T14:35:54.833 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:54 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: ignoring --setgroup ceph since I am not root 2024-05-03T14:35:54.833 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:54 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:54.830+0000 7fd4c00ca200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-03T14:35:55.240 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:53.197+0000 7fde02200700 1 Processor -- start 2024-05-03T14:35:55.240 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:53.197+0000 7fde02200700 1 -- start start 2024-05-03T14:35:55.241 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:53.198+0000 7fde02200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddfc108000 0x7fddfc108420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:35:55.241 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:53.198+0000 7fde02200700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fddfc1089f0 con 0x7fddfc108000 2024-05-03T14:35:55.241 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:53.198+0000 7fddfb400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddfc108000 0x7fddfc108420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:35:55.242 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:53.198+0000 7fddfb400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddfc108000 0x7fddfc108420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41008/0 (socket says 172.21.15.60:41008) 2024-05-03T14:35:55.242 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:53.198+0000 7fddfb400700 1 -- 172.21.15.60:0/90206927 learned_addr learned my addr 172.21.15.60:0/90206927 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:35:55.242 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:53.199+0000 7fddfb400700 1 -- 172.21.15.60:0/90206927 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddfc109200 con 0x7fddfc108000 2024-05-03T14:35:55.242 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:53.199+0000 7fddfb400700 1 --2- 172.21.15.60:0/90206927 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddfc108000 0x7fddfc108420 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fddec0098a0 tx=0x7fddec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86dcf6a20e1686b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:35:55.242 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:53.199+0000 7fddfaa00700 1 -- 172.21.15.60:0/90206927 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fddec004160 con 0x7fddfc108000 2024-05-03T14:35:55.242 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:53.199+0000 7fddfaa00700 1 -- 172.21.15.60:0/90206927 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7fddec004300 con 0x7fddfc108000 2024-05-03T14:35:55.242 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:53.199+0000 7fddfaa00700 1 -- 172.21.15.60:0/90206927 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fddec011c20 con 0x7fddfc108000 2024-05-03T14:35:55.244 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:53.200+0000 7fde02200700 1 -- 172.21.15.60:0/90206927 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddfc108000 msgr2=0x7fddfc108420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:35:55.244 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:53.200+0000 7fde02200700 1 --2- 172.21.15.60:0/90206927 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddfc108000 0x7fddfc108420 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fddec0098a0 tx=0x7fddec009bb0 comp rx=0 tx=0).stop 2024-05-03T14:35:55.245 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:53.200+0000 7fde02200700 1 -- 172.21.15.60:0/90206927 shutdown_connections 2024-05-03T14:35:55.245 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:53.200+0000 7fde02200700 1 --2- 172.21.15.60:0/90206927 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddfc108000 0x7fddfc108420 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:35:55.245 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:53.200+0000 7fde02200700 1 -- 172.21.15.60:0/90206927 >> 172.21.15.60:0/90206927 conn(0x7fddfc1035c0 msgr2=0x7fddfc1059e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:35:55.245 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:53.200+0000 7fde02200700 1 -- 172.21.15.60:0/90206927 shutdown_connections 2024-05-03T14:35:55.245 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:53.200+0000 7fde02200700 1 -- 172.21.15.60:0/90206927 wait complete. 2024-05-03T14:35:55.245 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:53.201+0000 7fde02200700 1 Processor -- start 2024-05-03T14:35:55.245 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:53.202+0000 7fde02200700 1 -- start start 2024-05-03T14:35:55.245 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:53.202+0000 7fde02200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddfc108000 0x7fddfc193180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:35:55.245 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:53.202+0000 7fde02200700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fddfc10c5b0 con 0x7fddfc108000 2024-05-03T14:35:55.245 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:53.202+0000 7fddfb400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddfc108000 0x7fddfc193180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:35:55.245 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:53.202+0000 7fddfb400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddfc108000 0x7fddfc193180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41012/0 (socket says 172.21.15.60:41012) 2024-05-03T14:35:55.245 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:53.202+0000 7fddfb400700 1 -- 172.21.15.60:0/3217630771 learned_addr learned my addr 172.21.15.60:0/3217630771 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:35:55.245 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:53.203+0000 7fddfb400700 1 -- 172.21.15.60:0/3217630771 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddfc1936c0 con 0x7fddfc108000 2024-05-03T14:35:55.246 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:53.203+0000 7fddfb400700 1 --2- 172.21.15.60:0/3217630771 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddfc108000 0x7fddfc193180 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fddec0118e0 tx=0x7fddec003980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:35:55.246 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:53.203+0000 7fddf8c00700 1 -- 172.21.15.60:0/3217630771 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fddec004520 con 0x7fddfc108000 2024-05-03T14:35:55.246 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:53.203+0000 7fddf8c00700 1 -- 172.21.15.60:0/3217630771 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7fddec003e30 con 0x7fddfc108000 2024-05-03T14:35:55.246 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:53.203+0000 7fddf8c00700 1 -- 172.21.15.60:0/3217630771 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fddec0194b0 con 0x7fddfc108000 2024-05-03T14:35:55.248 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:53.203+0000 7fde02200700 1 -- 172.21.15.60:0/3217630771 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddfc191900 con 0x7fddfc108000 2024-05-03T14:35:55.248 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:53.203+0000 7fde02200700 1 -- 172.21.15.60:0/3217630771 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddfc191de0 con 0x7fddfc108000 2024-05-03T14:35:55.248 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:53.205+0000 7fde02200700 1 -- 172.21.15.60:0/3217630771 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddfc064b00 con 0x7fddfc108000 2024-05-03T14:35:55.248 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:53.205+0000 7fddf8c00700 1 -- 172.21.15.60:0/3217630771 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fddec01a9c0 con 0x7fddfc108000 2024-05-03T14:35:55.248 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:53.205+0000 7fddf8c00700 1 --2- 172.21.15.60:0/3217630771 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7fdde003e4c0 0x7fdde0040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:35:55.248 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:53.205+0000 7fddf8c00700 1 -- 172.21.15.60:0/3217630771 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fddec053770 con 0x7fddfc108000 2024-05-03T14:35:55.248 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:53.205+0000 7fddf3e00700 1 --2- 172.21.15.60:0/3217630771 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7fdde003e4c0 0x7fdde0040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:35:55.248 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:53.206+0000 7fddf3e00700 1 --2- 172.21.15.60:0/3217630771 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7fdde003e4c0 0x7fdde0040980 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdde4006fd0 tx=0x7fdde4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:35:55.249 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:53.211+0000 7fddf8c00700 1 -- 172.21.15.60:0/3217630771 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fddec00b3e0 con 0x7fddfc108000 2024-05-03T14:35:55.249 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:53.519+0000 7fde02200700 1 -- 172.21.15.60:0/3217630771 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fddfc196f50 con 0x7fddfc108000 2024-05-03T14:35:55.249 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:54.507+0000 7fddf8c00700 1 -- 172.21.15.60:0/3217630771 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fddec004090 con 0x7fddfc108000 2024-05-03T14:35:55.249 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:54.511+0000 7fddf8c00700 1 -- 172.21.15.60:0/3217630771 <== mon.0 v2:172.21.15.60:3300/0 8 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7fddec02dcf0 con 0x7fddfc108000 2024-05-03T14:35:55.249 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:54.513+0000 7fde02200700 1 -- 172.21.15.60:0/3217630771 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7fdde003e4c0 msgr2=0x7fdde0040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:35:55.249 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:54.513+0000 7fde02200700 1 --2- 172.21.15.60:0/3217630771 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7fdde003e4c0 0x7fdde0040980 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdde4006fd0 tx=0x7fdde4006e40 comp rx=0 tx=0).stop 2024-05-03T14:35:55.249 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:54.513+0000 7fde02200700 1 -- 172.21.15.60:0/3217630771 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddfc108000 msgr2=0x7fddfc193180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:35:55.249 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:54.513+0000 7fde02200700 1 --2- 172.21.15.60:0/3217630771 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddfc108000 0x7fddfc193180 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fddec0118e0 tx=0x7fddec003980 comp rx=0 tx=0).stop 2024-05-03T14:35:55.249 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:54.513+0000 7fde02200700 1 -- 172.21.15.60:0/3217630771 shutdown_connections 2024-05-03T14:35:55.249 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:54.513+0000 7fde02200700 1 --2- 172.21.15.60:0/3217630771 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7fdde003e4c0 0x7fdde0040980 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:35:55.249 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:54.513+0000 7fde02200700 1 --2- 172.21.15.60:0/3217630771 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddfc108000 0x7fddfc193180 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:35:55.249 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:54.513+0000 7fde02200700 1 -- 172.21.15.60:0/3217630771 >> 172.21.15.60:0/3217630771 conn(0x7fddfc1035c0 msgr2=0x7fddfc104500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:35:55.249 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:54.514+0000 7fde02200700 1 -- 172.21.15.60:0/3217630771 shutdown_connections 2024-05-03T14:35:55.249 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:54.514+0000 7fde02200700 1 -- 172.21.15.60:0/3217630771 wait complete. 2024-05-03T14:35:55.410 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:55 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:55.096+0000 7fd4c00ca200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-03T14:35:55.410 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:55 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:55.239+0000 7fd4c00ca200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-03T14:35:55.665 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:55 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/3217630771' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-03T14:35:55.665 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:55 smithi060 ceph-mon[28873]: mgrmap e9: x(active, since 26s) 2024-05-03T14:35:56.074 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:55 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:55.662+0000 7fd4c00ca200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-03T14:35:56.075 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:55 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:55.780+0000 7fd4c00ca200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-03T14:35:56.826 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:35:56 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/3223144314' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-03T14:35:56.839 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout { 2024-05-03T14:35:56.846 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-03T14:35:56.846 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-03T14:35:56.846 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-03T14:35:56.846 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-03T14:35:56.846 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout } 2024-05-03T14:35:56.846 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:55.862+0000 7fa205c00700 1 Processor -- start 2024-05-03T14:35:56.847 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:55.863+0000 7fa205c00700 1 -- start start 2024-05-03T14:35:56.847 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:55.863+0000 7fa205c00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa20007ed90 0x7fa20007d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:35:56.847 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:55.863+0000 7fa205c00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa20007d840 con 0x7fa20007ed90 2024-05-03T14:35:56.847 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:55.864+0000 7fa1fea00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa20007ed90 0x7fa20007d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:35:56.847 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:55.864+0000 7fa1fea00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa20007ed90 0x7fa20007d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41048/0 (socket says 172.21.15.60:41048) 2024-05-03T14:35:56.847 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:55.864+0000 7fa1fea00700 1 -- 172.21.15.60:0/551618366 learned_addr learned my addr 172.21.15.60:0/551618366 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:35:56.847 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:55.864+0000 7fa1fea00700 1 -- 172.21.15.60:0/551618366 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa20007d9c0 con 0x7fa20007ed90 2024-05-03T14:35:56.847 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:55.865+0000 7fa1fea00700 1 --2- 172.21.15.60:0/551618366 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa20007ed90 0x7fa20007d300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa1f00098a0 tx=0x7fa1f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9f89e0efac32fe1 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:35:56.847 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:55.865+0000 7fa1fd600700 1 -- 172.21.15.60:0/551618366 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1f0004160 con 0x7fa20007ed90 2024-05-03T14:35:56.847 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:55.865+0000 7fa1fd600700 1 -- 172.21.15.60:0/551618366 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7fa1f0004300 con 0x7fa20007ed90 2024-05-03T14:35:56.847 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:55.865+0000 7fa1fd600700 1 -- 172.21.15.60:0/551618366 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1f0011c20 con 0x7fa20007ed90 2024-05-03T14:35:56.847 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:55.866+0000 7fa205c00700 1 -- 172.21.15.60:0/551618366 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa20007ed90 msgr2=0x7fa20007d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:35:56.847 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:55.866+0000 7fa205c00700 1 --2- 172.21.15.60:0/551618366 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa20007ed90 0x7fa20007d300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa1f00098a0 tx=0x7fa1f0009bb0 comp rx=0 tx=0).stop 2024-05-03T14:35:56.848 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:55.866+0000 7fa205c00700 1 -- 172.21.15.60:0/551618366 shutdown_connections 2024-05-03T14:35:56.848 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:55.866+0000 7fa205c00700 1 --2- 172.21.15.60:0/551618366 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa20007ed90 0x7fa20007d300 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:35:56.848 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:55.866+0000 7fa205c00700 1 -- 172.21.15.60:0/551618366 >> 172.21.15.60:0/551618366 conn(0x7fa200104750 msgr2=0x7fa200106b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:35:56.848 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:55.866+0000 7fa205c00700 1 -- 172.21.15.60:0/551618366 shutdown_connections 2024-05-03T14:35:56.848 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:55.866+0000 7fa205c00700 1 -- 172.21.15.60:0/551618366 wait complete. 2024-05-03T14:35:56.848 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:55.867+0000 7fa205c00700 1 Processor -- start 2024-05-03T14:35:56.848 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:55.867+0000 7fa205c00700 1 -- start start 2024-05-03T14:35:56.848 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:55.867+0000 7fa205c00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa20007ed90 0x7fa20019f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:35:56.848 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:55.867+0000 7fa205c00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa20010f660 con 0x7fa20007ed90 2024-05-03T14:35:56.848 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:55.868+0000 7fa1fea00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa20007ed90 0x7fa20019f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:35:56.848 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:55.868+0000 7fa1fea00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa20007ed90 0x7fa20019f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41054/0 (socket says 172.21.15.60:41054) 2024-05-03T14:35:56.848 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:55.868+0000 7fa1fea00700 1 -- 172.21.15.60:0/3223144314 learned_addr learned my addr 172.21.15.60:0/3223144314 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:35:56.848 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:55.868+0000 7fa1fea00700 1 -- 172.21.15.60:0/3223144314 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa20019f810 con 0x7fa20007ed90 2024-05-03T14:35:56.848 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:55.868+0000 7fa1fea00700 1 --2- 172.21.15.60:0/3223144314 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa20007ed90 0x7fa20019f2d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0011bf0 tx=0x7fa1f0011cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:35:56.849 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:55.869+0000 7fa1f7400700 1 -- 172.21.15.60:0/3223144314 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1f00038b0 con 0x7fa20007ed90 2024-05-03T14:35:56.849 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:55.869+0000 7fa1f7400700 1 -- 172.21.15.60:0/3223144314 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7fa1f0003a50 con 0x7fa20007ed90 2024-05-03T14:35:56.849 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:55.869+0000 7fa1f7400700 1 -- 172.21.15.60:0/3223144314 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1f00243f0 con 0x7fa20007ed90 2024-05-03T14:35:56.849 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:55.869+0000 7fa205c00700 1 -- 172.21.15.60:0/3223144314 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa20019faa0 con 0x7fa20007ed90 2024-05-03T14:35:56.849 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:55.869+0000 7fa205c00700 1 -- 172.21.15.60:0/3223144314 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2001a2790 con 0x7fa20007ed90 2024-05-03T14:35:56.849 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:55.870+0000 7fa1f7400700 1 -- 172.21.15.60:0/3223144314 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7fa1f0003c00 con 0x7fa20007ed90 2024-05-03T14:35:56.849 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:55.870+0000 7fa205c00700 1 -- 172.21.15.60:0/3223144314 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa200064b00 con 0x7fa20007ed90 2024-05-03T14:35:56.849 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:55.871+0000 7fa1f7400700 1 --2- 172.21.15.60:0/3223144314 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7fa1e403e510 0x7fa1e40409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:35:56.849 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:55.871+0000 7fa1f7400700 1 -- 172.21.15.60:0/3223144314 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa1f00535e0 con 0x7fa20007ed90 2024-05-03T14:35:56.849 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:55.871+0000 7fa1fe000700 1 -- 172.21.15.60:0/3223144314 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7fa1e403e510 msgr2=0x7fa1e40409d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6800/424391417 2024-05-03T14:35:56.849 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:55.871+0000 7fa1fe000700 1 --2- 172.21.15.60:0/3223144314 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7fa1e403e510 0x7fa1e40409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-03T14:35:56.849 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:55.877+0000 7fa1f7400700 1 -- 172.21.15.60:0/3223144314 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fa1f001a4e0 con 0x7fa20007ed90 2024-05-03T14:35:56.850 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:56.072+0000 7fa1fe000700 1 -- 172.21.15.60:0/3223144314 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7fa1e403e510 msgr2=0x7fa1e40409d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6800/424391417 2024-05-03T14:35:56.850 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:56.072+0000 7fa1fe000700 1 --2- 172.21.15.60:0/3223144314 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7fa1e403e510 0x7fa1e40409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-03T14:35:56.850 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:56.174+0000 7fa205c00700 1 -- 172.21.15.60:0/3223144314 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fa2001a2d40 con 0x7fa20007ed90 2024-05-03T14:35:56.850 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:56.175+0000 7fa1f7400700 1 -- 172.21.15.60:0/3223144314 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fa1f001a7b0 con 0x7fa20007ed90 2024-05-03T14:35:56.850 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:56.178+0000 7fa205c00700 1 -- 172.21.15.60:0/3223144314 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7fa1e403e510 msgr2=0x7fa1e40409d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-03T14:35:56.850 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:56.178+0000 7fa205c00700 1 --2- 172.21.15.60:0/3223144314 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7fa1e403e510 0x7fa1e40409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:35:56.850 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:56.178+0000 7fa205c00700 1 -- 172.21.15.60:0/3223144314 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa20007ed90 msgr2=0x7fa20019f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:35:56.850 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:56.178+0000 7fa205c00700 1 --2- 172.21.15.60:0/3223144314 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa20007ed90 0x7fa20019f2d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0011bf0 tx=0x7fa1f0011cd0 comp rx=0 tx=0).stop 2024-05-03T14:35:56.850 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:56.178+0000 7fa205c00700 1 -- 172.21.15.60:0/3223144314 shutdown_connections 2024-05-03T14:35:56.850 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:56.178+0000 7fa205c00700 1 --2- 172.21.15.60:0/3223144314 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7fa1e403e510 0x7fa1e40409d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:35:56.850 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:56.178+0000 7fa205c00700 1 --2- 172.21.15.60:0/3223144314 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa20007ed90 0x7fa20019f2d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:35:56.850 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:56.178+0000 7fa205c00700 1 -- 172.21.15.60:0/3223144314 >> 172.21.15.60:0/3223144314 conn(0x7fa200104750 msgr2=0x7fa2001052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:35:56.851 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:56.178+0000 7fa205c00700 1 -- 172.21.15.60:0/3223144314 shutdown_connections 2024-05-03T14:35:56.851 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:56.178+0000 7fa205c00700 1 -- 172.21.15.60:0/3223144314 wait complete. 2024-05-03T14:35:56.851 INFO:teuthology.orchestra.run.smithi060.stdout:Waiting for the mgr to restart... 2024-05-03T14:35:56.851 INFO:teuthology.orchestra.run.smithi060.stdout:Waiting for mgr epoch 9... 2024-05-03T14:35:57.659 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:57 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:57.417+0000 7fd4c00ca200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-03T14:35:57.659 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:57 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:57.656+0000 7fd4c00ca200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-03T14:35:58.045 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:57 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:57.877+0000 7fd4c00ca200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-03T14:35:58.337 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:58 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:58.273+0000 7fd4c00ca200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-03T14:35:58.661 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:58 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:58.400+0000 7fd4c00ca200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-03T14:35:58.924 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:58 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:58.658+0000 7fd4c00ca200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-03T14:35:58.924 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:58 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:58.921+0000 7fd4c00ca200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-03T14:36:00.043 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:35:59 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:35:59.741+0000 7fd4c00ca200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-03T14:36:00.043 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:36:00 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:36:00.039+0000 7fd4c00ca200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-03T14:36:00.736 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:36:00 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:36:00.635+0000 7fd4c00ca200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-03T14:36:01.087 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:36:00 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:36:00.768+0000 7fd4c00ca200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-03T14:36:01.587 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:36:01 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:36:01.250+0000 7fd4c00ca200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-03T14:36:02.087 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:36:01 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:36:01.727+0000 7fd4c00ca200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-03T14:36:02.088 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:36:01 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:36:01.857+0000 7fd4c00ca200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-03T14:36:02.588 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:36:02 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:36:02.192+0000 7fd4c00ca200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-03T14:36:02.588 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:36:02 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:36:02.378+0000 7fd4c00ca200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-03T14:36:03.061 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:36:02 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:36:02.773+0000 7fd4c00ca200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-03T14:36:03.061 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:36:02 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:36:02.930+0000 7fd4c00ca200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-03T14:36:03.061 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:36:03 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:36:03.057+0000 7fd4c00ca200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-03T14:36:04.087 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:36:03 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:36:03.850+0000 7fd4c00ca200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-03T14:36:04.587 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:36:04 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:36:04.150+0000 7fd4c00ca200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-03T14:36:04.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:04 smithi060 ceph-mon[28873]: Active manager daemon x restarted 2024-05-03T14:36:04.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:04 smithi060 ceph-mon[28873]: Activating manager daemon x 2024-05-03T14:36:04.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:04 smithi060 ceph-mon[28873]: osdmap e3: 0 total, 0 up, 0 in 2024-05-03T14:36:04.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:04 smithi060 ceph-mon[28873]: mgrmap e10: x(active, starting, since 0.0105136s) 2024-05-03T14:36:04.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:04 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T14:36:04.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:04 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-03T14:36:04.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:04 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-03T14:36:04.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:04 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-03T14:36:04.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:04 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-03T14:36:04.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:04 smithi060 ceph-mon[28873]: Manager daemon x is now available 2024-05-03T14:36:05.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:05 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:36:05.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:05 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-03T14:36:05.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:05 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-03T14:36:05.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:05 smithi060 ceph-mon[28873]: [03/May/2024:14:36:04] ENGINE Bus STARTING 2024-05-03T14:36:05.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:05 smithi060 ceph-mon[28873]: [03/May/2024:14:36:04] ENGINE Serving on http://172.21.15.60:8765 2024-05-03T14:36:05.589 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:05 smithi060 ceph-mon[28873]: mgrmap e11: x(active, since 1.01821s) 2024-05-03T14:36:05.981 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout { 2024-05-03T14:36:05.982 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-03T14:36:05.982 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-03T14:36:05.982 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout } 2024-05-03T14:36:05.983 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:57.431+0000 7f21a3600700 1 Processor -- start 2024-05-03T14:36:05.983 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:57.431+0000 7f21a3600700 1 -- start start 2024-05-03T14:36:05.983 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:57.432+0000 7f21a3600700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f219c108010 0x7f219c108430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:36:05.984 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:57.432+0000 7f21a3600700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f219c108a00 con 0x7f219c108010 2024-05-03T14:36:05.984 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:57.432+0000 7f21a0c00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f219c108010 0x7f219c108430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:36:05.984 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:57.432+0000 7f21a0c00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f219c108010 0x7f219c108430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41062/0 (socket says 172.21.15.60:41062) 2024-05-03T14:36:05.984 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:57.432+0000 7f21a0c00700 1 -- 172.21.15.60:0/821749159 learned_addr learned my addr 172.21.15.60:0/821749159 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:36:05.985 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:57.433+0000 7f21a0c00700 1 -- 172.21.15.60:0/821749159 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f219c109210 con 0x7f219c108010 2024-05-03T14:36:05.985 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:57.433+0000 7f21a0c00700 1 --2- 172.21.15.60:0/821749159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f219c108010 0x7f219c108430 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f218c00bb40 tx=0x7f218c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce77122d9cbbedec server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:36:05.985 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:57.433+0000 7f219be00700 1 -- 172.21.15.60:0/821749159 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f218c00dbe0 con 0x7f219c108010 2024-05-03T14:36:05.985 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:57.433+0000 7f219be00700 1 -- 172.21.15.60:0/821749159 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f218c004530 con 0x7f219c108010 2024-05-03T14:36:05.987 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:57.433+0000 7f219be00700 1 -- 172.21.15.60:0/821749159 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f218c010490 con 0x7f219c108010 2024-05-03T14:36:05.987 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:57.434+0000 7f21a3600700 1 -- 172.21.15.60:0/821749159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f219c108010 msgr2=0x7f219c108430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:36:05.987 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:57.434+0000 7f21a3600700 1 --2- 172.21.15.60:0/821749159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f219c108010 0x7f219c108430 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f218c00bb40 tx=0x7f218c00be50 comp rx=0 tx=0).stop 2024-05-03T14:36:05.987 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:57.434+0000 7f21a3600700 1 -- 172.21.15.60:0/821749159 shutdown_connections 2024-05-03T14:36:05.988 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:57.434+0000 7f21a3600700 1 --2- 172.21.15.60:0/821749159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f219c108010 0x7f219c108430 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:36:05.988 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:57.434+0000 7f21a3600700 1 -- 172.21.15.60:0/821749159 >> 172.21.15.60:0/821749159 conn(0x7f219c1035f0 msgr2=0x7f219c105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:36:05.988 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:57.434+0000 7f21a3600700 1 -- 172.21.15.60:0/821749159 shutdown_connections 2024-05-03T14:36:05.988 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:57.434+0000 7f21a3600700 1 -- 172.21.15.60:0/821749159 wait complete. 2024-05-03T14:36:05.988 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:57.435+0000 7f21a3600700 1 Processor -- start 2024-05-03T14:36:05.988 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:57.436+0000 7f21a3600700 1 -- start start 2024-05-03T14:36:05.988 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:57.436+0000 7f21a3600700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f219c108010 0x7f219c19f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:36:05.988 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:57.436+0000 7f21a3600700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f219c07c170 con 0x7f219c108010 2024-05-03T14:36:05.988 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:57.436+0000 7f21a0c00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f219c108010 0x7f219c19f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:36:05.988 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:57.436+0000 7f21a0c00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f219c108010 0x7f219c19f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41072/0 (socket says 172.21.15.60:41072) 2024-05-03T14:36:05.988 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:57.436+0000 7f21a0c00700 1 -- 172.21.15.60:0/1060082482 learned_addr learned my addr 172.21.15.60:0/1060082482 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:36:05.989 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:57.437+0000 7f21a0c00700 1 -- 172.21.15.60:0/1060082482 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f219c19f8f0 con 0x7f219c108010 2024-05-03T14:36:05.989 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:57.437+0000 7f21a0c00700 1 --2- 172.21.15.60:0/1060082482 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f219c108010 0x7f219c19f3b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f218c000c00 tx=0x7f218c00de00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:36:05.993 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:57.437+0000 7f219a000700 1 -- 172.21.15.60:0/1060082482 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f218c010740 con 0x7f219c108010 2024-05-03T14:36:05.993 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:57.437+0000 7f219a000700 1 -- 172.21.15.60:0/1060082482 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f218c01d070 con 0x7f219c108010 2024-05-03T14:36:05.993 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:57.437+0000 7f21a3600700 1 -- 172.21.15.60:0/1060082482 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f219c19fb80 con 0x7f219c108010 2024-05-03T14:36:05.994 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:57.437+0000 7f219a000700 1 -- 172.21.15.60:0/1060082482 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f218c019c20 con 0x7f219c108010 2024-05-03T14:36:05.994 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:57.438+0000 7f21a3600700 1 -- 172.21.15.60:0/1060082482 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f219c1a2820 con 0x7f219c108010 2024-05-03T14:36:05.994 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:57.439+0000 7f219a000700 1 -- 172.21.15.60:0/1060082482 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f218c028060 con 0x7f219c108010 2024-05-03T14:36:05.994 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:57.439+0000 7f219a000700 1 --2- 172.21.15.60:0/1060082482 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f218003e520 0x7f21800409e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:36:05.994 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:57.439+0000 7f219a000700 1 -- 172.21.15.60:0/1060082482 --> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f21800410f0 con 0x7f218003e520 2024-05-03T14:36:05.994 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:57.439+0000 7f219a000700 1 -- 172.21.15.60:0/1060082482 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f218c0537d0 con 0x7f219c108010 2024-05-03T14:36:05.994 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:57.439+0000 7f2193e00700 1 -- 172.21.15.60:0/1060082482 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f218003e520 msgr2=0x7f21800409e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6800/424391417 2024-05-03T14:36:05.994 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:57.440+0000 7f2193e00700 1 --2- 172.21.15.60:0/1060082482 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f218003e520 0x7f21800409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-03T14:36:05.994 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:57.641+0000 7f2193e00700 1 -- 172.21.15.60:0/1060082482 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f218003e520 msgr2=0x7f21800409e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6800/424391417 2024-05-03T14:36:05.994 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:57.641+0000 7f2193e00700 1 --2- 172.21.15.60:0/1060082482 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f218003e520 0x7f21800409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-03T14:36:05.995 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:58.042+0000 7f2193e00700 1 -- 172.21.15.60:0/1060082482 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f218003e520 msgr2=0x7f21800409e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6800/424391417 2024-05-03T14:36:05.995 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:58.042+0000 7f2193e00700 1 --2- 172.21.15.60:0/1060082482 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f218003e520 0x7f21800409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-03T14:36:05.995 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:58.843+0000 7f2193e00700 1 -- 172.21.15.60:0/1060082482 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f218003e520 msgr2=0x7f21800409e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6800/424391417 2024-05-03T14:36:05.995 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:35:58.843+0000 7f2193e00700 1 --2- 172.21.15.60:0/1060082482 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f218003e520 0x7f21800409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-03T14:36:05.995 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:00.444+0000 7f2193e00700 1 -- 172.21.15.60:0/1060082482 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f218003e520 msgr2=0x7f21800409e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6800/424391417 2024-05-03T14:36:05.995 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:00.444+0000 7f2193e00700 1 --2- 172.21.15.60:0/1060082482 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f218003e520 0x7f21800409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-03T14:36:05.995 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:03.647+0000 7f2193e00700 1 -- 172.21.15.60:0/1060082482 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f218003e520 msgr2=0x7f21800409e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6800/424391417 2024-05-03T14:36:05.995 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:03.647+0000 7f2193e00700 1 --2- 172.21.15.60:0/1060082482 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f218003e520 0x7f21800409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-05-03T14:36:05.995 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:04.161+0000 7f219a000700 1 -- 172.21.15.60:0/1060082482 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mgrmap(e 10) ==== 50002+0+0 (secure 0 0 0) 0x7f218c01f030 con 0x7f219c108010 2024-05-03T14:36:05.995 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:04.161+0000 7f219a000700 1 -- 172.21.15.60:0/1060082482 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f218003e520 msgr2=0x7f21800409e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-03T14:36:05.996 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:04.161+0000 7f219a000700 1 --2- 172.21.15.60:0/1060082482 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f218003e520 0x7f21800409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:36:05.996 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:05.166+0000 7f219a000700 1 -- 172.21.15.60:0/1060082482 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mgrmap(e 11) ==== 50129+0+0 (secure 0 0 0) 0x7f218c055010 con 0x7f219c108010 2024-05-03T14:36:05.996 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:05.166+0000 7f219a000700 1 --2- 172.21.15.60:0/1060082482 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f2180041fa0 0x7f2180044390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:36:05.996 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:05.166+0000 7f219a000700 1 -- 172.21.15.60:0/1060082482 --> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f218c00d140 con 0x7f2180041fa0 2024-05-03T14:36:05.996 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:05.167+0000 7f2193e00700 1 --2- 172.21.15.60:0/1060082482 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f2180041fa0 0x7f2180044390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:36:05.996 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:05.169+0000 7f2193e00700 1 --2- 172.21.15.60:0/1060082482 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f2180041fa0 0x7f2180044390 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2184003a10 tx=0x7f2184009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:36:05.996 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:05.170+0000 7f219a000700 1 -- 172.21.15.60:0/1060082482 <== mgr.14152 v2:172.21.15.60:6800/3066142956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f218c00d140 con 0x7f2180041fa0 2024-05-03T14:36:05.996 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:05.180+0000 7f21a3600700 1 -- 172.21.15.60:0/1060082482 --> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f219c064b00 con 0x7f2180041fa0 2024-05-03T14:36:05.996 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:05.180+0000 7f219a000700 1 -- 172.21.15.60:0/1060082482 <== mgr.14152 v2:172.21.15.60:6800/3066142956 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f219c064b00 con 0x7f2180041fa0 2024-05-03T14:36:05.996 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:05.181+0000 7f21a3600700 1 -- 172.21.15.60:0/1060082482 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f2180041fa0 msgr2=0x7f2180044390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:36:05.996 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:05.181+0000 7f21a3600700 1 --2- 172.21.15.60:0/1060082482 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f2180041fa0 0x7f2180044390 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2184003a10 tx=0x7f2184009270 comp rx=0 tx=0).stop 2024-05-03T14:36:05.997 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:05.181+0000 7f21a3600700 1 -- 172.21.15.60:0/1060082482 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f219c108010 msgr2=0x7f219c19f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:36:05.997 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:05.181+0000 7f21a3600700 1 --2- 172.21.15.60:0/1060082482 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f219c108010 0x7f219c19f3b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f218c000c00 tx=0x7f218c00de00 comp rx=0 tx=0).stop 2024-05-03T14:36:05.997 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:05.181+0000 7f21a3600700 1 -- 172.21.15.60:0/1060082482 shutdown_connections 2024-05-03T14:36:05.997 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:05.181+0000 7f21a3600700 1 --2- 172.21.15.60:0/1060082482 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f2180041fa0 0x7f2180044390 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:36:05.997 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:05.181+0000 7f21a3600700 1 --2- 172.21.15.60:0/1060082482 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f219c108010 0x7f219c19f3b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:36:05.997 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:05.181+0000 7f21a3600700 1 --2- 172.21.15.60:0/1060082482 >> [v2:172.21.15.60:6800/424391417,v1:172.21.15.60:6801/424391417] conn(0x7f218003e520 0x7f21800409e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:36:05.997 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:05.181+0000 7f21a3600700 1 -- 172.21.15.60:0/1060082482 >> 172.21.15.60:0/1060082482 conn(0x7f219c1035f0 msgr2=0x7f219c1915b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:36:05.997 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:05.181+0000 7f21a3600700 1 -- 172.21.15.60:0/1060082482 shutdown_connections 2024-05-03T14:36:05.997 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:05.181+0000 7f21a3600700 1 -- 172.21.15.60:0/1060082482 wait complete. 2024-05-03T14:36:05.998 INFO:teuthology.orchestra.run.smithi060.stdout:mgr epoch 9 is available 2024-05-03T14:36:05.998 INFO:teuthology.orchestra.run.smithi060.stdout:Generating a dashboard self-signed certificate... 2024-05-03T14:36:06.231 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:06 smithi060 ceph-mon[28873]: [03/May/2024:14:36:04] ENGINE Serving on https://172.21.15.60:7150 2024-05-03T14:36:06.231 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:06 smithi060 ceph-mon[28873]: [03/May/2024:14:36:04] ENGINE Bus STARTED 2024-05-03T14:36:06.231 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:06 smithi060 ceph-mon[28873]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-03T14:36:06.232 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:06 smithi060 ceph-mon[28873]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-03T14:36:07.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:07 smithi060 ceph-mon[28873]: mgrmap e12: x(active, since 2s) 2024-05-03T14:36:07.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:07 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:36:08.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:08 smithi060 ceph-mon[28873]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:36:08.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:08 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:36:08.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:08 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:36:08.620 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-03T14:36:08.620 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:07.350+0000 7ff9cec00700 1 Processor -- start 2024-05-03T14:36:08.620 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:07.351+0000 7ff9cec00700 1 -- start start 2024-05-03T14:36:08.620 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:07.351+0000 7ff9cec00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9c807ee90 0x7ff9c807d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:36:08.620 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:07.351+0000 7ff9cec00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c807d910 con 0x7ff9c807ee90 2024-05-03T14:36:08.620 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:07.351+0000 7ff9cd800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9c807ee90 0x7ff9c807d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:36:08.621 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:07.351+0000 7ff9cd800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9c807ee90 0x7ff9c807d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33312/0 (socket says 172.21.15.60:33312) 2024-05-03T14:36:08.621 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:07.351+0000 7ff9cd800700 1 -- 172.21.15.60:0/4289635543 learned_addr learned my addr 172.21.15.60:0/4289635543 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:36:08.621 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:07.352+0000 7ff9cd800700 1 -- 172.21.15.60:0/4289635543 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9c807da90 con 0x7ff9c807ee90 2024-05-03T14:36:08.621 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:07.352+0000 7ff9cd800700 1 --2- 172.21.15.60:0/4289635543 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9c807ee90 0x7ff9c807d340 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc0098a0 tx=0x7ff9bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13c8a61964ff65d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:36:08.621 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:07.352+0000 7ff9c7e00700 1 -- 172.21.15.60:0/4289635543 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff9bc004160 con 0x7ff9c807ee90 2024-05-03T14:36:08.621 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:07.353+0000 7ff9c7e00700 1 -- 172.21.15.60:0/4289635543 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7ff9bc004300 con 0x7ff9c807ee90 2024-05-03T14:36:08.621 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:07.353+0000 7ff9c7e00700 1 -- 172.21.15.60:0/4289635543 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff9bc011c20 con 0x7ff9c807ee90 2024-05-03T14:36:08.621 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:07.353+0000 7ff9cec00700 1 -- 172.21.15.60:0/4289635543 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9c807ee90 msgr2=0x7ff9c807d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:36:08.621 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:07.353+0000 7ff9cec00700 1 --2- 172.21.15.60:0/4289635543 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9c807ee90 0x7ff9c807d340 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc0098a0 tx=0x7ff9bc009bb0 comp rx=0 tx=0).stop 2024-05-03T14:36:08.621 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:07.353+0000 7ff9cec00700 1 -- 172.21.15.60:0/4289635543 shutdown_connections 2024-05-03T14:36:08.621 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:07.353+0000 7ff9cec00700 1 --2- 172.21.15.60:0/4289635543 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9c807ee90 0x7ff9c807d340 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:36:08.621 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:07.353+0000 7ff9cec00700 1 -- 172.21.15.60:0/4289635543 >> 172.21.15.60:0/4289635543 conn(0x7ff9c8104500 msgr2=0x7ff9c8106920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:36:08.621 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:07.354+0000 7ff9cec00700 1 -- 172.21.15.60:0/4289635543 shutdown_connections 2024-05-03T14:36:08.621 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:07.354+0000 7ff9cec00700 1 -- 172.21.15.60:0/4289635543 wait complete. 2024-05-03T14:36:08.622 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:07.355+0000 7ff9cec00700 1 Processor -- start 2024-05-03T14:36:08.622 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:07.355+0000 7ff9cec00700 1 -- start start 2024-05-03T14:36:08.622 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:07.355+0000 7ff9cec00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9c807ee90 0x7ff9c819f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:36:08.622 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:07.356+0000 7ff9cec00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c810f580 con 0x7ff9c807ee90 2024-05-03T14:36:08.622 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:07.356+0000 7ff9cd800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9c807ee90 0x7ff9c819f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:36:08.622 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:07.356+0000 7ff9cd800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9c807ee90 0x7ff9c819f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33328/0 (socket says 172.21.15.60:33328) 2024-05-03T14:36:08.622 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:07.356+0000 7ff9cd800700 1 -- 172.21.15.60:0/482936076 learned_addr learned my addr 172.21.15.60:0/482936076 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:36:08.622 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:07.356+0000 7ff9cd800700 1 -- 172.21.15.60:0/482936076 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9c819f6d0 con 0x7ff9c807ee90 2024-05-03T14:36:08.622 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:07.357+0000 7ff9cd800700 1 --2- 172.21.15.60:0/482936076 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9c807ee90 0x7ff9c819f190 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc009fb0 tx=0x7ff9bc011a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:36:08.622 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:07.357+0000 7ff9c6000700 1 -- 172.21.15.60:0/482936076 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff9bc0036a0 con 0x7ff9c807ee90 2024-05-03T14:36:08.622 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:07.357+0000 7ff9c6000700 1 -- 172.21.15.60:0/482936076 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7ff9bc003d40 con 0x7ff9c807ee90 2024-05-03T14:36:08.622 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:07.357+0000 7ff9cec00700 1 -- 172.21.15.60:0/482936076 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9c819f960 con 0x7ff9c807ee90 2024-05-03T14:36:08.622 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:07.357+0000 7ff9c6000700 1 -- 172.21.15.60:0/482936076 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff9bc01a430 con 0x7ff9c807ee90 2024-05-03T14:36:08.622 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:07.357+0000 7ff9cec00700 1 -- 172.21.15.60:0/482936076 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9c81a2650 con 0x7ff9c807ee90 2024-05-03T14:36:08.623 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:07.358+0000 7ff9cec00700 1 -- 172.21.15.60:0/482936076 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9c8064b00 con 0x7ff9c807ee90 2024-05-03T14:36:08.623 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:07.359+0000 7ff9c6000700 1 -- 172.21.15.60:0/482936076 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7ff9bc024440 con 0x7ff9c807ee90 2024-05-03T14:36:08.623 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:07.359+0000 7ff9c6000700 1 --2- 172.21.15.60:0/482936076 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7ff9b003e470 0x7ff9b0040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:36:08.623 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:07.359+0000 7ff9c6000700 1 -- 172.21.15.60:0/482936076 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff9bc053850 con 0x7ff9c807ee90 2024-05-03T14:36:08.623 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:07.359+0000 7ff9cce00700 1 --2- 172.21.15.60:0/482936076 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7ff9b003e470 0x7ff9b0040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:36:08.623 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:07.360+0000 7ff9cce00700 1 --2- 172.21.15.60:0/482936076 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7ff9b003e470 0x7ff9b0040930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4006fd0 tx=0x7ff9b4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:36:08.623 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:07.374+0000 7ff9c6000700 1 -- 172.21.15.60:0/482936076 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7ff9bc0212a0 con 0x7ff9c807ee90 2024-05-03T14:36:08.623 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:07.617+0000 7ff9cec00700 1 -- 172.21.15.60:0/482936076 --> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7ff9c8105830 con 0x7ff9b003e470 2024-05-03T14:36:08.623 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:07.771+0000 7ff9c6000700 1 -- 172.21.15.60:0/482936076 <== mgr.14152 v2:172.21.15.60:6800/3066142956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7ff9c8105830 con 0x7ff9b003e470 2024-05-03T14:36:08.623 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:07.774+0000 7ff9cec00700 1 -- 172.21.15.60:0/482936076 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7ff9b003e470 msgr2=0x7ff9b0040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:36:08.624 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:07.774+0000 7ff9cec00700 1 --2- 172.21.15.60:0/482936076 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7ff9b003e470 0x7ff9b0040930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4006fd0 tx=0x7ff9b4006e40 comp rx=0 tx=0).stop 2024-05-03T14:36:08.624 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:07.774+0000 7ff9cec00700 1 -- 172.21.15.60:0/482936076 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9c807ee90 msgr2=0x7ff9c819f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:36:08.624 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:07.775+0000 7ff9cec00700 1 --2- 172.21.15.60:0/482936076 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9c807ee90 0x7ff9c819f190 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc009fb0 tx=0x7ff9bc011a70 comp rx=0 tx=0).stop 2024-05-03T14:36:08.624 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:07.775+0000 7ff9cec00700 1 -- 172.21.15.60:0/482936076 shutdown_connections 2024-05-03T14:36:08.624 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:07.775+0000 7ff9cec00700 1 --2- 172.21.15.60:0/482936076 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7ff9b003e470 0x7ff9b0040930 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:36:08.624 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:07.775+0000 7ff9cec00700 1 --2- 172.21.15.60:0/482936076 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9c807ee90 0x7ff9c819f190 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:36:08.624 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:07.775+0000 7ff9cec00700 1 -- 172.21.15.60:0/482936076 >> 172.21.15.60:0/482936076 conn(0x7ff9c8104500 msgr2=0x7ff9c8105120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:36:08.624 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:07.775+0000 7ff9cec00700 1 -- 172.21.15.60:0/482936076 shutdown_connections 2024-05-03T14:36:08.624 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:07.775+0000 7ff9cec00700 1 -- 172.21.15.60:0/482936076 wait complete. 2024-05-03T14:36:08.624 INFO:teuthology.orchestra.run.smithi060.stdout:Creating initial admin user... 2024-05-03T14:36:10.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:10 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:36:10.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:10 smithi060 ceph-mon[28873]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:36:10.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:10 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:36:10.491 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$dxd9SAt.kCS774WZ4GCU4ebaHwT3TA.kWMgj7K/KdxkO3aocyez6W", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1714746969, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-03T14:36:10.491 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:09.250+0000 7f2db5200700 1 Processor -- start 2024-05-03T14:36:10.491 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:09.250+0000 7f2db5200700 1 -- start start 2024-05-03T14:36:10.491 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:09.250+0000 7f2db5200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2db010b2d0 0x7f2db010b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:36:10.491 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:09.250+0000 7f2db5200700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f2db010bcc0 con 0x7f2db010b2d0 2024-05-03T14:36:10.491 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:09.251+0000 7f2dade00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2db010b2d0 0x7f2db010b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:36:10.491 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:09.251+0000 7f2dade00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2db010b2d0 0x7f2db010b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33340/0 (socket says 172.21.15.60:33340) 2024-05-03T14:36:10.491 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:09.251+0000 7f2dade00700 1 -- 172.21.15.60:0/1136178377 learned_addr learned my addr 172.21.15.60:0/1136178377 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:36:10.491 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:09.251+0000 7f2dade00700 1 -- 172.21.15.60:0/1136178377 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2db010c480 con 0x7f2db010b2d0 2024-05-03T14:36:10.491 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:09.252+0000 7f2dade00700 1 --2- 172.21.15.60:0/1136178377 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2db010b2d0 0x7f2db010b6f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2da00098a0 tx=0x7f2da0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a06add8f8e472bb8 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:36:10.492 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:09.252+0000 7f2daca00700 1 -- 172.21.15.60:0/1136178377 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2da0004160 con 0x7f2db010b2d0 2024-05-03T14:36:10.492 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:09.252+0000 7f2daca00700 1 -- 172.21.15.60:0/1136178377 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f2da0004300 con 0x7f2db010b2d0 2024-05-03T14:36:10.492 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:09.252+0000 7f2daca00700 1 -- 172.21.15.60:0/1136178377 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2da0011c20 con 0x7f2db010b2d0 2024-05-03T14:36:10.492 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:09.253+0000 7f2db5200700 1 -- 172.21.15.60:0/1136178377 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2db010b2d0 msgr2=0x7f2db010b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:36:10.492 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:09.253+0000 7f2db5200700 1 --2- 172.21.15.60:0/1136178377 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2db010b2d0 0x7f2db010b6f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2da00098a0 tx=0x7f2da0009bb0 comp rx=0 tx=0).stop 2024-05-03T14:36:10.492 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:09.253+0000 7f2db5200700 1 -- 172.21.15.60:0/1136178377 shutdown_connections 2024-05-03T14:36:10.492 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:09.253+0000 7f2db5200700 1 --2- 172.21.15.60:0/1136178377 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2db010b2d0 0x7f2db010b6f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:36:10.492 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:09.253+0000 7f2db5200700 1 -- 172.21.15.60:0/1136178377 >> 172.21.15.60:0/1136178377 conn(0x7f2db007e900 msgr2=0x7f2db01094f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:36:10.492 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:09.253+0000 7f2db5200700 1 -- 172.21.15.60:0/1136178377 shutdown_connections 2024-05-03T14:36:10.492 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:09.253+0000 7f2db5200700 1 -- 172.21.15.60:0/1136178377 wait complete. 2024-05-03T14:36:10.492 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:09.254+0000 7f2db5200700 1 Processor -- start 2024-05-03T14:36:10.492 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:09.255+0000 7f2db5200700 1 -- start start 2024-05-03T14:36:10.492 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:09.255+0000 7f2db5200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2db010b2d0 0x7f2db019f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:36:10.493 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:09.255+0000 7f2db5200700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f2db010f830 con 0x7f2db010b2d0 2024-05-03T14:36:10.493 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:09.255+0000 7f2dade00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2db010b2d0 0x7f2db019f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:36:10.493 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:09.255+0000 7f2dade00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2db010b2d0 0x7f2db019f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33356/0 (socket says 172.21.15.60:33356) 2024-05-03T14:36:10.493 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:09.255+0000 7f2dade00700 1 -- 172.21.15.60:0/2341794416 learned_addr learned my addr 172.21.15.60:0/2341794416 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:36:10.493 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:09.256+0000 7f2dade00700 1 -- 172.21.15.60:0/2341794416 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2db019f820 con 0x7f2db010b2d0 2024-05-03T14:36:10.493 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:09.256+0000 7f2dade00700 1 --2- 172.21.15.60:0/2341794416 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2db010b2d0 0x7f2db019f2e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2da00118e0 tx=0x7f2da0003980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:36:10.493 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:09.256+0000 7f2da6a00700 1 -- 172.21.15.60:0/2341794416 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2da0003db0 con 0x7f2db010b2d0 2024-05-03T14:36:10.493 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:09.257+0000 7f2da6a00700 1 -- 172.21.15.60:0/2341794416 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f2da0020070 con 0x7f2db010b2d0 2024-05-03T14:36:10.493 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:09.257+0000 7f2da6a00700 1 -- 172.21.15.60:0/2341794416 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2da00193c0 con 0x7f2db010b2d0 2024-05-03T14:36:10.493 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:09.257+0000 7f2db5200700 1 -- 172.21.15.60:0/2341794416 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2db019fab0 con 0x7f2db010b2d0 2024-05-03T14:36:10.493 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:09.257+0000 7f2db5200700 1 -- 172.21.15.60:0/2341794416 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2db01a27a0 con 0x7f2db010b2d0 2024-05-03T14:36:10.493 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:09.258+0000 7f2da6a00700 1 -- 172.21.15.60:0/2341794416 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f2da001a430 con 0x7f2db010b2d0 2024-05-03T14:36:10.494 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:09.259+0000 7f2db5200700 1 -- 172.21.15.60:0/2341794416 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2db010f9b0 con 0x7f2db010b2d0 2024-05-03T14:36:10.494 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:09.259+0000 7f2da6a00700 1 --2- 172.21.15.60:0/2341794416 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f2d9403e520 0x7f2d940409e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:36:10.494 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:09.259+0000 7f2da6a00700 1 -- 172.21.15.60:0/2341794416 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2da0053a40 con 0x7f2db010b2d0 2024-05-03T14:36:10.494 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:09.259+0000 7f2dad400700 1 --2- 172.21.15.60:0/2341794416 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f2d9403e520 0x7f2d940409e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:36:10.494 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:09.260+0000 7f2dad400700 1 --2- 172.21.15.60:0/2341794416 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f2d9403e520 0x7f2d940409e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c006fd0 tx=0x7f2d9c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:36:10.494 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:09.269+0000 7f2da6a00700 1 -- 172.21.15.60:0/2341794416 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f2da002b260 con 0x7f2db010b2d0 2024-05-03T14:36:10.494 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:09.540+0000 7f2db5200700 1 -- 172.21.15.60:0/2341794416 --> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f2db01a2a30 con 0x7f2d9403e520 2024-05-03T14:36:10.494 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:09.801+0000 7f2da6a00700 1 -- 172.21.15.60:0/2341794416 <== mgr.14152 v2:172.21.15.60:6800/3066142956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f2db01a2a30 con 0x7f2d9403e520 2024-05-03T14:36:10.494 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:09.804+0000 7f2db5200700 1 -- 172.21.15.60:0/2341794416 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f2d9403e520 msgr2=0x7f2d940409e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:36:10.494 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:09.804+0000 7f2db5200700 1 --2- 172.21.15.60:0/2341794416 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f2d9403e520 0x7f2d940409e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c006fd0 tx=0x7f2d9c006e40 comp rx=0 tx=0).stop 2024-05-03T14:36:10.494 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:09.804+0000 7f2db5200700 1 -- 172.21.15.60:0/2341794416 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2db010b2d0 msgr2=0x7f2db019f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:36:10.495 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:09.804+0000 7f2db5200700 1 --2- 172.21.15.60:0/2341794416 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2db010b2d0 0x7f2db019f2e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2da00118e0 tx=0x7f2da0003980 comp rx=0 tx=0).stop 2024-05-03T14:36:10.495 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:09.804+0000 7f2db5200700 1 -- 172.21.15.60:0/2341794416 shutdown_connections 2024-05-03T14:36:10.495 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:09.804+0000 7f2db5200700 1 --2- 172.21.15.60:0/2341794416 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f2d9403e520 0x7f2d940409e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:36:10.495 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:09.804+0000 7f2db5200700 1 --2- 172.21.15.60:0/2341794416 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2db010b2d0 0x7f2db019f2e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:36:10.495 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:09.804+0000 7f2db5200700 1 -- 172.21.15.60:0/2341794416 >> 172.21.15.60:0/2341794416 conn(0x7f2db007e900 msgr2=0x7f2db0108d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:36:10.495 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:09.804+0000 7f2db5200700 1 -- 172.21.15.60:0/2341794416 shutdown_connections 2024-05-03T14:36:10.495 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:09.804+0000 7f2db5200700 1 -- 172.21.15.60:0/2341794416 wait complete. 2024-05-03T14:36:10.495 INFO:teuthology.orchestra.run.smithi060.stdout:Fetching dashboard port number... 2024-05-03T14:36:12.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:12 smithi060 ceph-mon[28873]: mgrmap e13: x(active, since 6s) 2024-05-03T14:36:12.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:12 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/1139431210' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-03T14:36:12.682 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 8443 2024-05-03T14:36:12.682 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:11.332+0000 7faac2a00700 1 Processor -- start 2024-05-03T14:36:12.682 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:11.332+0000 7faac2a00700 1 -- start start 2024-05-03T14:36:12.682 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:11.333+0000 7faac2a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faabc10b2f0 0x7faabc10b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:36:12.682 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:11.333+0000 7faac2a00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7faabc10bce0 con 0x7faabc10b2f0 2024-05-03T14:36:12.684 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:11.333+0000 7faabbe00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faabc10b2f0 0x7faabc10b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:36:12.684 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:11.333+0000 7faabbe00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faabc10b2f0 0x7faabc10b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33362/0 (socket says 172.21.15.60:33362) 2024-05-03T14:36:12.684 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:11.333+0000 7faabbe00700 1 -- 172.21.15.60:0/537192367 learned_addr learned my addr 172.21.15.60:0/537192367 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:36:12.684 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:11.334+0000 7faabbe00700 1 -- 172.21.15.60:0/537192367 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faabc10c4a0 con 0x7faabc10b2f0 2024-05-03T14:36:12.684 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:11.334+0000 7faabbe00700 1 --2- 172.21.15.60:0/537192367 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faabc10b2f0 0x7faabc10b710 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7faaac0098a0 tx=0x7faaac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b74c891b2650e05 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:36:12.684 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:11.334+0000 7faabaa00700 1 -- 172.21.15.60:0/537192367 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faaac004160 con 0x7faabc10b2f0 2024-05-03T14:36:12.684 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:11.334+0000 7faabaa00700 1 -- 172.21.15.60:0/537192367 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7faaac004300 con 0x7faabc10b2f0 2024-05-03T14:36:12.684 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:11.334+0000 7faabaa00700 1 -- 172.21.15.60:0/537192367 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faaac011c20 con 0x7faabc10b2f0 2024-05-03T14:36:12.685 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:11.335+0000 7faac2a00700 1 -- 172.21.15.60:0/537192367 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faabc10b2f0 msgr2=0x7faabc10b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:36:12.685 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:11.335+0000 7faac2a00700 1 --2- 172.21.15.60:0/537192367 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faabc10b2f0 0x7faabc10b710 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7faaac0098a0 tx=0x7faaac009bb0 comp rx=0 tx=0).stop 2024-05-03T14:36:12.685 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:11.335+0000 7faac2a00700 1 -- 172.21.15.60:0/537192367 shutdown_connections 2024-05-03T14:36:12.685 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:11.335+0000 7faac2a00700 1 --2- 172.21.15.60:0/537192367 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faabc10b2f0 0x7faabc10b710 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:36:12.685 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:11.335+0000 7faac2a00700 1 -- 172.21.15.60:0/537192367 >> 172.21.15.60:0/537192367 conn(0x7faabc07e900 msgr2=0x7faabc1094f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:36:12.685 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:11.335+0000 7faac2a00700 1 -- 172.21.15.60:0/537192367 shutdown_connections 2024-05-03T14:36:12.689 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:11.335+0000 7faac2a00700 1 -- 172.21.15.60:0/537192367 wait complete. 2024-05-03T14:36:12.689 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:11.336+0000 7faac2a00700 1 Processor -- start 2024-05-03T14:36:12.689 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:11.337+0000 7faac2a00700 1 -- start start 2024-05-03T14:36:12.689 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:11.337+0000 7faac2a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faabc10b2f0 0x7faabc19f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:36:12.689 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:11.337+0000 7faac2a00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7faabc10f850 con 0x7faabc10b2f0 2024-05-03T14:36:12.689 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:11.337+0000 7faabbe00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faabc10b2f0 0x7faabc19f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:36:12.689 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:11.337+0000 7faabbe00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faabc10b2f0 0x7faabc19f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33372/0 (socket says 172.21.15.60:33372) 2024-05-03T14:36:12.689 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:11.337+0000 7faabbe00700 1 -- 172.21.15.60:0/1139431210 learned_addr learned my addr 172.21.15.60:0/1139431210 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:36:12.689 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:11.338+0000 7faabbe00700 1 -- 172.21.15.60:0/1139431210 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faabc19f8f0 con 0x7faabc10b2f0 2024-05-03T14:36:12.689 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:11.338+0000 7faabbe00700 1 --2- 172.21.15.60:0/1139431210 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faabc10b2f0 0x7faabc19f3b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7faaac0118e0 tx=0x7faaac003980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:36:12.690 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:11.338+0000 7faab8c00700 1 -- 172.21.15.60:0/1139431210 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faaac004520 con 0x7faabc10b2f0 2024-05-03T14:36:12.690 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:11.339+0000 7faab8c00700 1 -- 172.21.15.60:0/1139431210 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7faaac003e30 con 0x7faabc10b2f0 2024-05-03T14:36:12.690 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:11.339+0000 7faab8c00700 1 -- 172.21.15.60:0/1139431210 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faaac0194b0 con 0x7faabc10b2f0 2024-05-03T14:36:12.690 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:11.339+0000 7faac2a00700 1 -- 172.21.15.60:0/1139431210 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faabc19fb80 con 0x7faabc10b2f0 2024-05-03T14:36:12.690 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:11.339+0000 7faac2a00700 1 -- 172.21.15.60:0/1139431210 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faabc1a2870 con 0x7faabc10b2f0 2024-05-03T14:36:12.690 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:11.340+0000 7faac2a00700 1 -- 172.21.15.60:0/1139431210 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faabc10f9d0 con 0x7faabc10b2f0 2024-05-03T14:36:12.690 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:11.340+0000 7faab8c00700 1 -- 172.21.15.60:0/1139431210 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7faaac01a430 con 0x7faabc10b2f0 2024-05-03T14:36:12.690 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:11.341+0000 7faab8c00700 1 --2- 172.21.15.60:0/1139431210 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7faaa003e5c0 0x7faaa0040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:36:12.690 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:11.341+0000 7faab8c00700 1 -- 172.21.15.60:0/1139431210 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7faaac0539f0 con 0x7faabc10b2f0 2024-05-03T14:36:12.690 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:11.341+0000 7faabb400700 1 --2- 172.21.15.60:0/1139431210 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7faaa003e5c0 0x7faaa0040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:36:12.690 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:11.342+0000 7faabb400700 1 --2- 172.21.15.60:0/1139431210 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7faaa003e5c0 0x7faaa0040a80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7faaa8006fd0 tx=0x7faaa8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:36:12.690 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:11.347+0000 7faab8c00700 1 -- 172.21.15.60:0/1139431210 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7faaac00b3e0 con 0x7faabc10b2f0 2024-05-03T14:36:12.690 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:11.587+0000 7faac2a00700 1 -- 172.21.15.60:0/1139431210 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7faabc1a2a40 con 0x7faabc10b2f0 2024-05-03T14:36:12.691 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:11.588+0000 7faab8c00700 1 -- 172.21.15.60:0/1139431210 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7faaac02cdc0 con 0x7faabc10b2f0 2024-05-03T14:36:12.691 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:11.591+0000 7faac2a00700 1 -- 172.21.15.60:0/1139431210 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7faaa003e5c0 msgr2=0x7faaa0040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:36:12.691 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:11.591+0000 7faac2a00700 1 --2- 172.21.15.60:0/1139431210 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7faaa003e5c0 0x7faaa0040a80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7faaa8006fd0 tx=0x7faaa8006e40 comp rx=0 tx=0).stop 2024-05-03T14:36:12.691 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:11.591+0000 7faac2a00700 1 -- 172.21.15.60:0/1139431210 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faabc10b2f0 msgr2=0x7faabc19f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:36:12.691 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:11.591+0000 7faac2a00700 1 --2- 172.21.15.60:0/1139431210 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faabc10b2f0 0x7faabc19f3b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7faaac0118e0 tx=0x7faaac003980 comp rx=0 tx=0).stop 2024-05-03T14:36:12.691 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:11.591+0000 7faac2a00700 1 -- 172.21.15.60:0/1139431210 shutdown_connections 2024-05-03T14:36:12.691 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:11.591+0000 7faac2a00700 1 --2- 172.21.15.60:0/1139431210 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7faaa003e5c0 0x7faaa0040a80 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:36:12.691 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:11.591+0000 7faac2a00700 1 --2- 172.21.15.60:0/1139431210 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faabc10b2f0 0x7faabc19f3b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:36:12.691 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:11.591+0000 7faac2a00700 1 -- 172.21.15.60:0/1139431210 >> 172.21.15.60:0/1139431210 conn(0x7faabc07e900 msgr2=0x7faabc108d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:36:12.691 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:11.592+0000 7faac2a00700 1 -- 172.21.15.60:0/1139431210 shutdown_connections 2024-05-03T14:36:12.691 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:11.592+0000 7faac2a00700 1 -- 172.21.15.60:0/1139431210 wait complete. 2024-05-03T14:36:12.718 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-03T14:36:12.718 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout disabled 2024-05-03T14:36:12.740 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-03T14:36:12.740 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout inactive 2024-05-03T14:36:12.740 INFO:teuthology.orchestra.run.smithi060.stdout:firewalld.service is not enabled 2024-05-03T14:36:12.741 INFO:teuthology.orchestra.run.smithi060.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-03T14:36:12.748 INFO:teuthology.orchestra.run.smithi060.stdout:Ceph Dashboard is now available at: 2024-05-03T14:36:12.748 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:36:12.749 INFO:teuthology.orchestra.run.smithi060.stdout: URL: https://smithi060.front.sepia.ceph.com:8443/ 2024-05-03T14:36:12.749 INFO:teuthology.orchestra.run.smithi060.stdout: User: admin 2024-05-03T14:36:12.749 INFO:teuthology.orchestra.run.smithi060.stdout: Password: kdxsh5bgg6 2024-05-03T14:36:12.749 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:36:12.750 INFO:teuthology.orchestra.run.smithi060.stdout:Saving cluster configuration to /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/config directory 2024-05-03T14:36:12.752 INFO:teuthology.orchestra.run.smithi060.stdout:Enabling autotune for osd_memory_target 2024-05-03T14:36:14.333 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:13.362+0000 7fdcb1600700 1 Processor -- start 2024-05-03T14:36:14.333 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:13.363+0000 7fdcb1600700 1 -- start start 2024-05-03T14:36:14.333 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:13.363+0000 7fdcb1600700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdcac07ee70 0x7fdcac07d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:36:14.333 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:13.363+0000 7fdcb1600700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fdcac07d8f0 con 0x7fdcac07ee70 2024-05-03T14:36:14.333 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:13.363+0000 7fdcabe00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdcac07ee70 0x7fdcac07d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:36:14.333 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:13.364+0000 7fdcabe00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdcac07ee70 0x7fdcac07d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38360/0 (socket says 172.21.15.60:38360) 2024-05-03T14:36:14.334 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:13.364+0000 7fdcabe00700 1 -- 172.21.15.60:0/3102479263 learned_addr learned my addr 172.21.15.60:0/3102479263 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:36:14.334 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:13.364+0000 7fdcabe00700 1 -- 172.21.15.60:0/3102479263 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcac07da70 con 0x7fdcac07ee70 2024-05-03T14:36:14.334 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:13.364+0000 7fdcabe00700 1 --2- 172.21.15.60:0/3102479263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdcac07ee70 0x7fdcac07d320 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c0098a0 tx=0x7fdc9c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=844937ea34c93ae0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:36:14.334 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:13.365+0000 7fdcaaa00700 1 -- 172.21.15.60:0/3102479263 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc9c004160 con 0x7fdcac07ee70 2024-05-03T14:36:14.334 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:13.365+0000 7fdcaaa00700 1 -- 172.21.15.60:0/3102479263 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7fdc9c004300 con 0x7fdcac07ee70 2024-05-03T14:36:14.334 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:13.365+0000 7fdcaaa00700 1 -- 172.21.15.60:0/3102479263 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc9c011c20 con 0x7fdcac07ee70 2024-05-03T14:36:14.334 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:13.366+0000 7fdcb1600700 1 -- 172.21.15.60:0/3102479263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdcac07ee70 msgr2=0x7fdcac07d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:36:14.334 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:13.366+0000 7fdcb1600700 1 --2- 172.21.15.60:0/3102479263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdcac07ee70 0x7fdcac07d320 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c0098a0 tx=0x7fdc9c009bb0 comp rx=0 tx=0).stop 2024-05-03T14:36:14.334 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:13.366+0000 7fdcb1600700 1 -- 172.21.15.60:0/3102479263 shutdown_connections 2024-05-03T14:36:14.334 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:13.366+0000 7fdcb1600700 1 --2- 172.21.15.60:0/3102479263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdcac07ee70 0x7fdcac07d320 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:36:14.334 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:13.366+0000 7fdcb1600700 1 -- 172.21.15.60:0/3102479263 >> 172.21.15.60:0/3102479263 conn(0x7fdcac104500 msgr2=0x7fdcac106920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:36:14.334 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:13.366+0000 7fdcb1600700 1 -- 172.21.15.60:0/3102479263 shutdown_connections 2024-05-03T14:36:14.335 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:13.366+0000 7fdcb1600700 1 -- 172.21.15.60:0/3102479263 wait complete. 2024-05-03T14:36:14.335 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:13.367+0000 7fdcb1600700 1 Processor -- start 2024-05-03T14:36:14.335 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:13.367+0000 7fdcb1600700 1 -- start start 2024-05-03T14:36:14.335 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:13.368+0000 7fdcb1600700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdcac07ee70 0x7fdcac19f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:36:14.335 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:13.368+0000 7fdcb1600700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fdcac10f5a0 con 0x7fdcac07ee70 2024-05-03T14:36:14.335 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:13.368+0000 7fdcabe00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdcac07ee70 0x7fdcac19f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:36:14.335 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:13.368+0000 7fdcabe00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdcac07ee70 0x7fdcac19f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38370/0 (socket says 172.21.15.60:38370) 2024-05-03T14:36:14.335 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:13.368+0000 7fdcabe00700 1 -- 172.21.15.60:0/3888804459 learned_addr learned my addr 172.21.15.60:0/3888804459 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:36:14.335 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:13.368+0000 7fdcabe00700 1 -- 172.21.15.60:0/3888804459 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcac19f700 con 0x7fdcac07ee70 2024-05-03T14:36:14.335 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:13.369+0000 7fdcabe00700 1 --2- 172.21.15.60:0/3888804459 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdcac07ee70 0x7fdcac19f1c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c0046e0 tx=0x7fdc9c011a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:36:14.335 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:13.369+0000 7fdca8c00700 1 -- 172.21.15.60:0/3888804459 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc9c0038b0 con 0x7fdcac07ee70 2024-05-03T14:36:14.335 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:13.369+0000 7fdca8c00700 1 -- 172.21.15.60:0/3888804459 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7fdc9c003a50 con 0x7fdcac07ee70 2024-05-03T14:36:14.335 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:13.369+0000 7fdca8c00700 1 -- 172.21.15.60:0/3888804459 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc9c024630 con 0x7fdcac07ee70 2024-05-03T14:36:14.336 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:13.369+0000 7fdcb1600700 1 -- 172.21.15.60:0/3888804459 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcac19f990 con 0x7fdcac07ee70 2024-05-03T14:36:14.336 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:13.369+0000 7fdcb1600700 1 -- 172.21.15.60:0/3888804459 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcac1a2680 con 0x7fdcac07ee70 2024-05-03T14:36:14.336 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:13.370+0000 7fdcb1600700 1 -- 172.21.15.60:0/3888804459 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcac064b00 con 0x7fdcac07ee70 2024-05-03T14:36:14.336 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:13.370+0000 7fdca8c00700 1 -- 172.21.15.60:0/3888804459 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fdc9c022020 con 0x7fdcac07ee70 2024-05-03T14:36:14.336 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:13.371+0000 7fdca8c00700 1 --2- 172.21.15.60:0/3888804459 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fdc9003e5c0 0x7fdc90040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:36:14.336 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:13.371+0000 7fdca8c00700 1 -- 172.21.15.60:0/3888804459 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fdc9c0537d0 con 0x7fdcac07ee70 2024-05-03T14:36:14.336 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:13.371+0000 7fdcab400700 1 --2- 172.21.15.60:0/3888804459 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fdc9003e5c0 0x7fdc90040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:36:14.336 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:13.372+0000 7fdcab400700 1 --2- 172.21.15.60:0/3888804459 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fdc9003e5c0 0x7fdc90040a80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdc98006fd0 tx=0x7fdc98006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:36:14.336 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:13.376+0000 7fdca8c00700 1 -- 172.21.15.60:0/3888804459 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fdc9c00b3e0 con 0x7fdcac07ee70 2024-05-03T14:36:14.336 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:13.609+0000 7fdcb1600700 1 -- 172.21.15.60:0/3888804459 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7fdcac1a2ca0 con 0x7fdcac07ee70 2024-05-03T14:36:14.336 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:13.610+0000 7fdca8c00700 1 -- 172.21.15.60:0/3888804459 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7fdc9c003cb0 con 0x7fdcac07ee70 2024-05-03T14:36:14.336 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:13.613+0000 7fdcb1600700 1 -- 172.21.15.60:0/3888804459 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fdc9003e5c0 msgr2=0x7fdc90040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:36:14.336 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:13.613+0000 7fdcb1600700 1 --2- 172.21.15.60:0/3888804459 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fdc9003e5c0 0x7fdc90040a80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdc98006fd0 tx=0x7fdc98006e40 comp rx=0 tx=0).stop 2024-05-03T14:36:14.337 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:13.613+0000 7fdcb1600700 1 -- 172.21.15.60:0/3888804459 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdcac07ee70 msgr2=0x7fdcac19f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:36:14.337 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:13.613+0000 7fdcb1600700 1 --2- 172.21.15.60:0/3888804459 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdcac07ee70 0x7fdcac19f1c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c0046e0 tx=0x7fdc9c011a70 comp rx=0 tx=0).stop 2024-05-03T14:36:14.337 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:13.613+0000 7fdcb1600700 1 -- 172.21.15.60:0/3888804459 shutdown_connections 2024-05-03T14:36:14.337 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:13.613+0000 7fdcb1600700 1 --2- 172.21.15.60:0/3888804459 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fdc9003e5c0 0x7fdc90040a80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:36:14.337 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:13.613+0000 7fdcb1600700 1 --2- 172.21.15.60:0/3888804459 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdcac07ee70 0x7fdcac19f1c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:36:14.337 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:13.613+0000 7fdcb1600700 1 -- 172.21.15.60:0/3888804459 >> 172.21.15.60:0/3888804459 conn(0x7fdcac104500 msgr2=0x7fdcac105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:36:14.337 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:13.614+0000 7fdcb1600700 1 -- 172.21.15.60:0/3888804459 shutdown_connections 2024-05-03T14:36:14.337 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:13.614+0000 7fdcb1600700 1 -- 172.21.15.60:0/3888804459 wait complete. 2024-05-03T14:36:15.984 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:14.916+0000 7f2a9fc00700 1 Processor -- start 2024-05-03T14:36:15.984 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:14.916+0000 7f2a9fc00700 1 -- start start 2024-05-03T14:36:15.984 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:14.916+0000 7f2a9fc00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a98107fd0 0x7f2a981083f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:36:15.984 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:14.917+0000 7f2a9fc00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f2a981089c0 con 0x7f2a98107fd0 2024-05-03T14:36:15.984 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:14.917+0000 7f2a9d200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a98107fd0 0x7f2a981083f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:36:15.984 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:14.917+0000 7f2a9d200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a98107fd0 0x7f2a981083f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38372/0 (socket says 172.21.15.60:38372) 2024-05-03T14:36:15.984 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:14.917+0000 7f2a9d200700 1 -- 172.21.15.60:0/2068038659 learned_addr learned my addr 172.21.15.60:0/2068038659 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:36:15.984 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:14.917+0000 7f2a9d200700 1 -- 172.21.15.60:0/2068038659 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a981091d0 con 0x7f2a98107fd0 2024-05-03T14:36:15.984 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:14.918+0000 7f2a9d200700 1 --2- 172.21.15.60:0/2068038659 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a98107fd0 0x7f2a981083f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2a8800bb40 tx=0x7f2a8800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db846382b3343e87 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:36:15.985 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:14.918+0000 7f2a97e00700 1 -- 172.21.15.60:0/2068038659 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a8800dbe0 con 0x7f2a98107fd0 2024-05-03T14:36:15.985 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:14.918+0000 7f2a97e00700 1 -- 172.21.15.60:0/2068038659 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f2a88004530 con 0x7f2a98107fd0 2024-05-03T14:36:15.985 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:14.918+0000 7f2a97e00700 1 -- 172.21.15.60:0/2068038659 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a88010490 con 0x7f2a98107fd0 2024-05-03T14:36:15.985 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:14.919+0000 7f2a9fc00700 1 -- 172.21.15.60:0/2068038659 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a98107fd0 msgr2=0x7f2a981083f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:36:15.985 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:14.919+0000 7f2a9fc00700 1 --2- 172.21.15.60:0/2068038659 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a98107fd0 0x7f2a981083f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2a8800bb40 tx=0x7f2a8800be50 comp rx=0 tx=0).stop 2024-05-03T14:36:15.985 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:14.919+0000 7f2a9fc00700 1 -- 172.21.15.60:0/2068038659 shutdown_connections 2024-05-03T14:36:15.985 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:14.919+0000 7f2a9fc00700 1 --2- 172.21.15.60:0/2068038659 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a98107fd0 0x7f2a981083f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:36:15.985 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:14.919+0000 7f2a9fc00700 1 -- 172.21.15.60:0/2068038659 >> 172.21.15.60:0/2068038659 conn(0x7f2a981035b0 msgr2=0x7f2a981059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:36:15.985 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:14.919+0000 7f2a9fc00700 1 -- 172.21.15.60:0/2068038659 shutdown_connections 2024-05-03T14:36:15.985 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:14.919+0000 7f2a9fc00700 1 -- 172.21.15.60:0/2068038659 wait complete. 2024-05-03T14:36:15.985 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:14.920+0000 7f2a9fc00700 1 Processor -- start 2024-05-03T14:36:15.985 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:14.920+0000 7f2a9fc00700 1 -- start start 2024-05-03T14:36:15.985 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:14.921+0000 7f2a9fc00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a98107fd0 0x7f2a9819f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:36:15.985 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:14.921+0000 7f2a9fc00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f2a9807c170 con 0x7f2a98107fd0 2024-05-03T14:36:15.986 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:14.921+0000 7f2a9d200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a98107fd0 0x7f2a9819f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:36:15.986 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:14.921+0000 7f2a9d200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a98107fd0 0x7f2a9819f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38388/0 (socket says 172.21.15.60:38388) 2024-05-03T14:36:15.986 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:14.921+0000 7f2a9d200700 1 -- 172.21.15.60:0/3194653385 learned_addr learned my addr 172.21.15.60:0/3194653385 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:36:15.986 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:14.921+0000 7f2a9d200700 1 -- 172.21.15.60:0/3194653385 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a9819f950 con 0x7f2a98107fd0 2024-05-03T14:36:15.986 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:14.922+0000 7f2a9d200700 1 --2- 172.21.15.60:0/3194653385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a98107fd0 0x7f2a9819f410 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2a88000c00 tx=0x7f2a8800de00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:36:15.986 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:14.922+0000 7f2a96000700 1 -- 172.21.15.60:0/3194653385 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a88010740 con 0x7f2a98107fd0 2024-05-03T14:36:15.986 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:14.922+0000 7f2a96000700 1 -- 172.21.15.60:0/3194653385 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f2a8801d070 con 0x7f2a98107fd0 2024-05-03T14:36:15.986 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:14.922+0000 7f2a9fc00700 1 -- 172.21.15.60:0/3194653385 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a9819fbe0 con 0x7f2a98107fd0 2024-05-03T14:36:15.986 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:14.922+0000 7f2a96000700 1 -- 172.21.15.60:0/3194653385 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a88019c20 con 0x7f2a98107fd0 2024-05-03T14:36:15.986 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:14.922+0000 7f2a9fc00700 1 -- 172.21.15.60:0/3194653385 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a981a28d0 con 0x7f2a98107fd0 2024-05-03T14:36:15.986 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:14.923+0000 7f2a9fc00700 1 -- 172.21.15.60:0/3194653385 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a98064b00 con 0x7f2a98107fd0 2024-05-03T14:36:15.986 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:14.924+0000 7f2a96000700 1 -- 172.21.15.60:0/3194653385 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f2a88028060 con 0x7f2a98107fd0 2024-05-03T14:36:15.986 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:14.924+0000 7f2a96000700 1 --2- 172.21.15.60:0/3194653385 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f2a7c03e560 0x7f2a7c040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:36:15.987 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:14.924+0000 7f2a96000700 1 -- 172.21.15.60:0/3194653385 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2a88053a80 con 0x7f2a98107fd0 2024-05-03T14:36:15.987 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:14.925+0000 7f2a8fe00700 1 --2- 172.21.15.60:0/3194653385 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f2a7c03e560 0x7f2a7c040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:36:15.987 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:14.925+0000 7f2a8fe00700 1 --2- 172.21.15.60:0/3194653385 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f2a7c03e560 0x7f2a7c040a20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2a80006fd0 tx=0x7f2a80006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:36:15.987 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:14.930+0000 7f2a96000700 1 -- 172.21.15.60:0/3194653385 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f2a88055e20 con 0x7f2a98107fd0 2024-05-03T14:36:15.987 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:15.289+0000 7f2a9fc00700 1 -- 172.21.15.60:0/3194653385 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f2a980008d0 con 0x7f2a98107fd0 2024-05-03T14:36:15.987 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:15.292+0000 7f2a96000700 1 -- 172.21.15.60:0/3194653385 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7f2a88015070 con 0x7f2a98107fd0 2024-05-03T14:36:15.987 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:15.295+0000 7f2a9fc00700 1 -- 172.21.15.60:0/3194653385 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f2a7c03e560 msgr2=0x7f2a7c040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:36:15.987 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:15.295+0000 7f2a9fc00700 1 --2- 172.21.15.60:0/3194653385 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f2a7c03e560 0x7f2a7c040a20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2a80006fd0 tx=0x7f2a80006e40 comp rx=0 tx=0).stop 2024-05-03T14:36:15.987 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:15.295+0000 7f2a9fc00700 1 -- 172.21.15.60:0/3194653385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a98107fd0 msgr2=0x7f2a9819f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:36:15.987 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:15.295+0000 7f2a9fc00700 1 --2- 172.21.15.60:0/3194653385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a98107fd0 0x7f2a9819f410 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2a88000c00 tx=0x7f2a8800de00 comp rx=0 tx=0).stop 2024-05-03T14:36:15.987 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:15.296+0000 7f2a9fc00700 1 -- 172.21.15.60:0/3194653385 shutdown_connections 2024-05-03T14:36:15.987 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:15.296+0000 7f2a9fc00700 1 --2- 172.21.15.60:0/3194653385 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f2a7c03e560 0x7f2a7c040a20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:36:15.987 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:15.296+0000 7f2a9fc00700 1 --2- 172.21.15.60:0/3194653385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a98107fd0 0x7f2a9819f410 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:36:15.987 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:15.296+0000 7f2a9fc00700 1 -- 172.21.15.60:0/3194653385 >> 172.21.15.60:0/3194653385 conn(0x7f2a981035b0 msgr2=0x7f2a981939d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:36:15.988 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:15.296+0000 7f2a9fc00700 1 -- 172.21.15.60:0/3194653385 shutdown_connections 2024-05-03T14:36:15.988 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-05-03T14:36:15.296+0000 7f2a9fc00700 1 -- 172.21.15.60:0/3194653385 wait complete. 2024-05-03T14:36:15.988 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-03T14:36:15.988 INFO:teuthology.orchestra.run.smithi060.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-03T14:36:15.988 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:36:15.988 INFO:teuthology.orchestra.run.smithi060.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-03T14:36:15.988 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:36:15.988 INFO:teuthology.orchestra.run.smithi060.stdout:Or, if you are only running a single cluster on this host: 2024-05-03T14:36:15.988 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:36:15.988 INFO:teuthology.orchestra.run.smithi060.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-03T14:36:15.988 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:36:15.988 INFO:teuthology.orchestra.run.smithi060.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-03T14:36:15.989 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:36:15.989 INFO:teuthology.orchestra.run.smithi060.stdout: ceph telemetry on 2024-05-03T14:36:15.989 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:36:15.989 INFO:teuthology.orchestra.run.smithi060.stdout:For more information see: 2024-05-03T14:36:15.989 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:36:15.989 INFO:teuthology.orchestra.run.smithi060.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-03T14:36:15.989 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:36:15.989 INFO:teuthology.orchestra.run.smithi060.stdout:Bootstrap complete. 2024-05-03T14:36:16.080 INFO:tasks.cephadm:Fetching config... 2024-05-03T14:36:16.080 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-05-03T14:36:16.080 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-03T14:36:16.131 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-03T14:36:16.131 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-05-03T14:36:16.131 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-03T14:36:16.182 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-03T14:36:16.182 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-05-03T14:36:16.182 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/keyring of=/dev/stdout 2024-05-03T14:36:16.249 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-03T14:36:16.249 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-05-03T14:36:16.249 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-03T14:36:16.338 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-03T14:36:16.338 DEBUG:teuthology.orchestra.run.smithi060:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGj+Vyqd8aAQiJp4VPt9ZKpUtijfX5oIjqped5rn0oI9rdDS2uD8reuxqzIi5HZaW21BYBG25pAunMBdHpW2r5Iw6t17emAMGMh0AxPsQzJ8tG8J+mjWz2bX36Y6w+/8yyDvBziiWJYZXIRRB6bxxgHUQuO29wcg5bvG2M82wmguHSBQ+ZE5CQ7b7Tl8+nTbnif/4kD7lDxF74bsRQyMxbUf2gnC/XhKSEZpHXzWzD3/6+zaUZ7fm7gFxI4PgMo96XXQ7fYfCvpiUoqWPUnKLyh9/gQEZfT4Zmpagc+HHbjUoGiNWQlh1L/gsi8IpJOsb1T0tAUGd9jB+uxUMbcSTc2CfvrVf8S6qKB5rngS5D65Sy32no0JbBISShCsvoW6NZViImbtLc1mQZ6kcXmjXRbTxVuabIgz7rLT/GToKbnr7HmZAPiPCVcaC2Jn+FyFfMx6WMwPKwSkj5WqOKUDiE7gkzp9Zp/8YAbx9UwqMJgmSapet11CvOhKeXCVKqGlE= ceph-285d64b2-095a-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-03T14:36:16.496 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:16 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/3194653385' entity='client.admin' 2024-05-03T14:36:16.517 INFO:teuthology.orchestra.run.smithi060.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGj+Vyqd8aAQiJp4VPt9ZKpUtijfX5oIjqped5rn0oI9rdDS2uD8reuxqzIi5HZaW21BYBG25pAunMBdHpW2r5Iw6t17emAMGMh0AxPsQzJ8tG8J+mjWz2bX36Y6w+/8yyDvBziiWJYZXIRRB6bxxgHUQuO29wcg5bvG2M82wmguHSBQ+ZE5CQ7b7Tl8+nTbnif/4kD7lDxF74bsRQyMxbUf2gnC/XhKSEZpHXzWzD3/6+zaUZ7fm7gFxI4PgMo96XXQ7fYfCvpiUoqWPUnKLyh9/gQEZfT4Zmpagc+HHbjUoGiNWQlh1L/gsi8IpJOsb1T0tAUGd9jB+uxUMbcSTc2CfvrVf8S6qKB5rngS5D65Sy32no0JbBISShCsvoW6NZViImbtLc1mQZ6kcXmjXRbTxVuabIgz7rLT/GToKbnr7HmZAPiPCVcaC2Jn+FyFfMx6WMwPKwSkj5WqOKUDiE7gkzp9Zp/8YAbx9UwqMJgmSapet11CvOhKeXCVKqGlE= ceph-285d64b2-095a-11ef-bc97-c7b262605968 2024-05-03T14:36:16.560 DEBUG:teuthology.orchestra.run.smithi090:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGj+Vyqd8aAQiJp4VPt9ZKpUtijfX5oIjqped5rn0oI9rdDS2uD8reuxqzIi5HZaW21BYBG25pAunMBdHpW2r5Iw6t17emAMGMh0AxPsQzJ8tG8J+mjWz2bX36Y6w+/8yyDvBziiWJYZXIRRB6bxxgHUQuO29wcg5bvG2M82wmguHSBQ+ZE5CQ7b7Tl8+nTbnif/4kD7lDxF74bsRQyMxbUf2gnC/XhKSEZpHXzWzD3/6+zaUZ7fm7gFxI4PgMo96XXQ7fYfCvpiUoqWPUnKLyh9/gQEZfT4Zmpagc+HHbjUoGiNWQlh1L/gsi8IpJOsb1T0tAUGd9jB+uxUMbcSTc2CfvrVf8S6qKB5rngS5D65Sy32no0JbBISShCsvoW6NZViImbtLc1mQZ6kcXmjXRbTxVuabIgz7rLT/GToKbnr7HmZAPiPCVcaC2Jn+FyFfMx6WMwPKwSkj5WqOKUDiE7gkzp9Zp/8YAbx9UwqMJgmSapet11CvOhKeXCVKqGlE= ceph-285d64b2-095a-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-03T14:36:16.598 INFO:teuthology.orchestra.run.smithi090.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGj+Vyqd8aAQiJp4VPt9ZKpUtijfX5oIjqped5rn0oI9rdDS2uD8reuxqzIi5HZaW21BYBG25pAunMBdHpW2r5Iw6t17emAMGMh0AxPsQzJ8tG8J+mjWz2bX36Y6w+/8yyDvBziiWJYZXIRRB6bxxgHUQuO29wcg5bvG2M82wmguHSBQ+ZE5CQ7b7Tl8+nTbnif/4kD7lDxF74bsRQyMxbUf2gnC/XhKSEZpHXzWzD3/6+zaUZ7fm7gFxI4PgMo96XXQ7fYfCvpiUoqWPUnKLyh9/gQEZfT4Zmpagc+HHbjUoGiNWQlh1L/gsi8IpJOsb1T0tAUGd9jB+uxUMbcSTc2CfvrVf8S6qKB5rngS5D65Sy32no0JbBISShCsvoW6NZViImbtLc1mQZ6kcXmjXRbTxVuabIgz7rLT/GToKbnr7HmZAPiPCVcaC2Jn+FyFfMx6WMwPKwSkj5WqOKUDiE7gkzp9Zp/8YAbx9UwqMJgmSapet11CvOhKeXCVKqGlE= ceph-285d64b2-095a-11ef-bc97-c7b262605968 2024-05-03T14:36:16.610 DEBUG:teuthology.orchestra.run.smithi149:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGj+Vyqd8aAQiJp4VPt9ZKpUtijfX5oIjqped5rn0oI9rdDS2uD8reuxqzIi5HZaW21BYBG25pAunMBdHpW2r5Iw6t17emAMGMh0AxPsQzJ8tG8J+mjWz2bX36Y6w+/8yyDvBziiWJYZXIRRB6bxxgHUQuO29wcg5bvG2M82wmguHSBQ+ZE5CQ7b7Tl8+nTbnif/4kD7lDxF74bsRQyMxbUf2gnC/XhKSEZpHXzWzD3/6+zaUZ7fm7gFxI4PgMo96XXQ7fYfCvpiUoqWPUnKLyh9/gQEZfT4Zmpagc+HHbjUoGiNWQlh1L/gsi8IpJOsb1T0tAUGd9jB+uxUMbcSTc2CfvrVf8S6qKB5rngS5D65Sy32no0JbBISShCsvoW6NZViImbtLc1mQZ6kcXmjXRbTxVuabIgz7rLT/GToKbnr7HmZAPiPCVcaC2Jn+FyFfMx6WMwPKwSkj5WqOKUDiE7gkzp9Zp/8YAbx9UwqMJgmSapet11CvOhKeXCVKqGlE= ceph-285d64b2-095a-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-03T14:36:16.661 INFO:teuthology.orchestra.run.smithi149.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGj+Vyqd8aAQiJp4VPt9ZKpUtijfX5oIjqped5rn0oI9rdDS2uD8reuxqzIi5HZaW21BYBG25pAunMBdHpW2r5Iw6t17emAMGMh0AxPsQzJ8tG8J+mjWz2bX36Y6w+/8yyDvBziiWJYZXIRRB6bxxgHUQuO29wcg5bvG2M82wmguHSBQ+ZE5CQ7b7Tl8+nTbnif/4kD7lDxF74bsRQyMxbUf2gnC/XhKSEZpHXzWzD3/6+zaUZ7fm7gFxI4PgMo96XXQ7fYfCvpiUoqWPUnKLyh9/gQEZfT4Zmpagc+HHbjUoGiNWQlh1L/gsi8IpJOsb1T0tAUGd9jB+uxUMbcSTc2CfvrVf8S6qKB5rngS5D65Sy32no0JbBISShCsvoW6NZViImbtLc1mQZ6kcXmjXRbTxVuabIgz7rLT/GToKbnr7HmZAPiPCVcaC2Jn+FyFfMx6WMwPKwSkj5WqOKUDiE7gkzp9Zp/8YAbx9UwqMJgmSapet11CvOhKeXCVKqGlE= ceph-285d64b2-095a-11ef-bc97-c7b262605968 2024-05-03T14:36:16.674 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-03T14:36:17.122 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:36:19.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:19.722+0000 7f313b600700 1 -- 172.21.15.60:0/1565195514 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3134102f00 msgr2=0x7f3134103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:36:19.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:19.722+0000 7f313b600700 1 --2- 172.21.15.60:0/1565195514 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3134102f00 0x7f3134103320 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3124009910 tx=0x7f3124009c20 comp rx=0 tx=0).stop 2024-05-03T14:36:19.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:19.722+0000 7f313b600700 1 -- 172.21.15.60:0/1565195514 shutdown_connections 2024-05-03T14:36:19.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:19.722+0000 7f313b600700 1 --2- 172.21.15.60:0/1565195514 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3134102f00 0x7f3134103320 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:36:19.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:19.722+0000 7f313b600700 1 -- 172.21.15.60:0/1565195514 >> 172.21.15.60:0/1565195514 conn(0x7f31340fe4a0 msgr2=0x7f31341008e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:36:19.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:19.722+0000 7f313b600700 1 -- 172.21.15.60:0/1565195514 shutdown_connections 2024-05-03T14:36:19.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:19.722+0000 7f313b600700 1 -- 172.21.15.60:0/1565195514 wait complete. 2024-05-03T14:36:19.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:19.723+0000 7f313b600700 1 Processor -- start 2024-05-03T14:36:19.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:19.724+0000 7f313b600700 1 -- start start 2024-05-03T14:36:19.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:19.724+0000 7f313b600700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3134102f00 0x7f3134077160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:36:19.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:19.724+0000 7f313b600700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f3134109050 con 0x7f3134102f00 2024-05-03T14:36:19.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:19.724+0000 7f3138c00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3134102f00 0x7f3134077160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:36:19.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:19.724+0000 7f3138c00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3134102f00 0x7f3134077160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38402/0 (socket says 172.21.15.60:38402) 2024-05-03T14:36:19.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:19.724+0000 7f3138c00700 1 -- 172.21.15.60:0/162599163 learned_addr learned my addr 172.21.15.60:0/162599163 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:36:19.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:19.725+0000 7f3138c00700 1 -- 172.21.15.60:0/162599163 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31340776a0 con 0x7f3134102f00 2024-05-03T14:36:19.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:19.725+0000 7f3138c00700 1 --2- 172.21.15.60:0/162599163 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3134102f00 0x7f3134077160 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3124005950 tx=0x7f3124004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:36:19.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:19.726+0000 7f3132000700 1 -- 172.21.15.60:0/162599163 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3124027070 con 0x7f3134102f00 2024-05-03T14:36:19.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:19.726+0000 7f3132000700 1 -- 172.21.15.60:0/162599163 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f3124022470 con 0x7f3134102f00 2024-05-03T14:36:19.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:19.726+0000 7f3132000700 1 -- 172.21.15.60:0/162599163 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f312400f5f0 con 0x7f3134102f00 2024-05-03T14:36:19.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:19.726+0000 7f313b600700 1 -- 172.21.15.60:0/162599163 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3134075890 con 0x7f3134102f00 2024-05-03T14:36:19.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:19.726+0000 7f313b600700 1 -- 172.21.15.60:0/162599163 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3134075dd0 con 0x7f3134102f00 2024-05-03T14:36:19.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:19.727+0000 7f313b600700 1 -- 172.21.15.60:0/162599163 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31341097d0 con 0x7f3134102f00 2024-05-03T14:36:19.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:19.727+0000 7f3132000700 1 -- 172.21.15.60:0/162599163 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f312402a070 con 0x7f3134102f00 2024-05-03T14:36:19.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:19.728+0000 7f3132000700 1 --2- 172.21.15.60:0/162599163 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f311403e570 0x7f3114040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:36:19.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:19.728+0000 7f3132000700 1 -- 172.21.15.60:0/162599163 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f312401e030 con 0x7f3134102f00 2024-05-03T14:36:19.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:19.728+0000 7f312be00700 1 --2- 172.21.15.60:0/162599163 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f311403e570 0x7f3114040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:36:19.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:19.729+0000 7f312be00700 1 --2- 172.21.15.60:0/162599163 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f311403e570 0x7f3114040a30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f311c006fd0 tx=0x7f311c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:36:19.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:19.734+0000 7f3132000700 1 -- 172.21.15.60:0/162599163 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f3124013070 con 0x7f3134102f00 2024-05-03T14:36:19.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:19.967+0000 7f313b600700 1 -- 172.21.15.60:0/162599163 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f3134191ad0 con 0x7f3134102f00 2024-05-03T14:36:19.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:19.973+0000 7f3132000700 1 -- 172.21.15.60:0/162599163 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f3124020370 con 0x7f3134102f00 2024-05-03T14:36:19.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:19.977+0000 7f313b600700 1 -- 172.21.15.60:0/162599163 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f311403e570 msgr2=0x7f3114040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:36:19.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:19.977+0000 7f313b600700 1 --2- 172.21.15.60:0/162599163 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f311403e570 0x7f3114040a30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f311c006fd0 tx=0x7f311c006e40 comp rx=0 tx=0).stop 2024-05-03T14:36:19.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:19.977+0000 7f313b600700 1 -- 172.21.15.60:0/162599163 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3134102f00 msgr2=0x7f3134077160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:36:19.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:19.977+0000 7f313b600700 1 --2- 172.21.15.60:0/162599163 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3134102f00 0x7f3134077160 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3124005950 tx=0x7f3124004740 comp rx=0 tx=0).stop 2024-05-03T14:36:19.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:19.978+0000 7f313b600700 1 -- 172.21.15.60:0/162599163 shutdown_connections 2024-05-03T14:36:19.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:19.978+0000 7f313b600700 1 --2- 172.21.15.60:0/162599163 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f311403e570 0x7f3114040a30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:36:19.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:19.978+0000 7f313b600700 1 --2- 172.21.15.60:0/162599163 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3134102f00 0x7f3134077160 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:36:19.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:19.978+0000 7f313b600700 1 -- 172.21.15.60:0/162599163 >> 172.21.15.60:0/162599163 conn(0x7f31340fe4a0 msgr2=0x7f31340fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:36:19.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:19.978+0000 7f313b600700 1 -- 172.21.15.60:0/162599163 shutdown_connections 2024-05-03T14:36:19.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:19.978+0000 7f313b600700 1 -- 172.21.15.60:0/162599163 wait complete. 2024-05-03T14:36:20.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:20 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:36:20.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:20 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:36:20.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:20 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi060", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:36:20.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:20 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:36:20.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:20 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:36:20.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:20 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:36:20.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:20 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/162599163' entity='client.admin' 2024-05-03T14:36:20.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:20 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:36:20.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:20 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:36:20.731 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-03T14:36:20.732 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-03T14:36:21.165 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:36:23.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:23.191+0000 7fbd2f200700 1 -- 172.21.15.60:0/4211752573 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd28105130 msgr2=0x7fbd28105550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:36:23.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:23.191+0000 7fbd2f200700 1 --2- 172.21.15.60:0/4211752573 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd28105130 0x7fbd28105550 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbd1800b0a0 tx=0x7fbd1800b3b0 comp rx=0 tx=0).stop 2024-05-03T14:36:23.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:23.191+0000 7fbd2f200700 1 -- 172.21.15.60:0/4211752573 shutdown_connections 2024-05-03T14:36:23.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:23.191+0000 7fbd2f200700 1 --2- 172.21.15.60:0/4211752573 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd28105130 0x7fbd28105550 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:36:23.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:23.191+0000 7fbd2f200700 1 -- 172.21.15.60:0/4211752573 >> 172.21.15.60:0/4211752573 conn(0x7fbd281006f0 msgr2=0x7fbd28102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:36:23.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:23.191+0000 7fbd2f200700 1 -- 172.21.15.60:0/4211752573 shutdown_connections 2024-05-03T14:36:23.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:23.191+0000 7fbd2f200700 1 -- 172.21.15.60:0/4211752573 wait complete. 2024-05-03T14:36:23.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:23.193+0000 7fbd2f200700 1 Processor -- start 2024-05-03T14:36:23.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:23.193+0000 7fbd2f200700 1 -- start start 2024-05-03T14:36:23.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:23.193+0000 7fbd2f200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd28105130 0x7fbd2819a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:36:23.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:23.193+0000 7fbd2f200700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2810b300 con 0x7fbd28105130 2024-05-03T14:36:23.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:23.194+0000 7fbd27e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd28105130 0x7fbd2819a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:36:23.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:23.194+0000 7fbd27e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd28105130 0x7fbd2819a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36438/0 (socket says 172.21.15.60:36438) 2024-05-03T14:36:23.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:23.194+0000 7fbd27e00700 1 -- 172.21.15.60:0/3994666053 learned_addr learned my addr 172.21.15.60:0/3994666053 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:36:23.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:23.194+0000 7fbd27e00700 1 -- 172.21.15.60:0/3994666053 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd2819ae10 con 0x7fbd28105130 2024-05-03T14:36:23.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:23.194+0000 7fbd27e00700 1 --2- 172.21.15.60:0/3994666053 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd28105130 0x7fbd2819a8d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbd180077d0 tx=0x7fbd180076e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:36:23.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:23.195+0000 7fbd25600700 1 -- 172.21.15.60:0/3994666053 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd18017040 con 0x7fbd28105130 2024-05-03T14:36:23.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:23.195+0000 7fbd25600700 1 -- 172.21.15.60:0/3994666053 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7fbd18022070 con 0x7fbd28105130 2024-05-03T14:36:23.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:23.195+0000 7fbd25600700 1 -- 172.21.15.60:0/3994666053 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd1801d5e0 con 0x7fbd28105130 2024-05-03T14:36:23.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:23.195+0000 7fbd2f200700 1 -- 172.21.15.60:0/3994666053 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd2819b0a0 con 0x7fbd28105130 2024-05-03T14:36:23.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:23.195+0000 7fbd2f200700 1 -- 172.21.15.60:0/3994666053 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd2819dd90 con 0x7fbd28105130 2024-05-03T14:36:23.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:23.196+0000 7fbd2f200700 1 -- 172.21.15.60:0/3994666053 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd2810ba80 con 0x7fbd28105130 2024-05-03T14:36:23.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:23.196+0000 7fbd25600700 1 -- 172.21.15.60:0/3994666053 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fbd18004040 con 0x7fbd28105130 2024-05-03T14:36:23.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:23.197+0000 7fbd25600700 1 --2- 172.21.15.60:0/3994666053 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fbd0c03e570 0x7fbd0c040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:36:23.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:23.197+0000 7fbd25600700 1 -- 172.21.15.60:0/3994666053 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fbd1800f050 con 0x7fbd28105130 2024-05-03T14:36:23.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:23.197+0000 7fbd1f400700 1 --2- 172.21.15.60:0/3994666053 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fbd0c03e570 0x7fbd0c040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:36:23.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:23.198+0000 7fbd1f400700 1 --2- 172.21.15.60:0/3994666053 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fbd0c03e570 0x7fbd0c040a30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbd10006fd0 tx=0x7fbd10006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:36:23.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:23.203+0000 7fbd25600700 1 -- 172.21.15.60:0/3994666053 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fbd1801ca00 con 0x7fbd28105130 2024-05-03T14:36:23.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:23.445+0000 7fbd2f200700 1 -- 172.21.15.60:0/3994666053 --> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fbd2802e620 con 0x7fbd0c03e570 2024-05-03T14:36:23.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:23.450+0000 7fbd25600700 1 -- 172.21.15.60:0/3994666053 <== mgr.14152 v2:172.21.15.60:6800/3066142956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbd2802e620 con 0x7fbd0c03e570 2024-05-03T14:36:23.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:23.453+0000 7fbd2f200700 1 -- 172.21.15.60:0/3994666053 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fbd0c03e570 msgr2=0x7fbd0c040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:36:23.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:23.453+0000 7fbd2f200700 1 --2- 172.21.15.60:0/3994666053 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fbd0c03e570 0x7fbd0c040a30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbd10006fd0 tx=0x7fbd10006e40 comp rx=0 tx=0).stop 2024-05-03T14:36:23.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:23.453+0000 7fbd2f200700 1 -- 172.21.15.60:0/3994666053 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd28105130 msgr2=0x7fbd2819a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:36:23.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:23.453+0000 7fbd2f200700 1 --2- 172.21.15.60:0/3994666053 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd28105130 0x7fbd2819a8d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbd180077d0 tx=0x7fbd180076e0 comp rx=0 tx=0).stop 2024-05-03T14:36:23.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:23.454+0000 7fbd2f200700 1 -- 172.21.15.60:0/3994666053 shutdown_connections 2024-05-03T14:36:23.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:23.454+0000 7fbd2f200700 1 --2- 172.21.15.60:0/3994666053 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fbd0c03e570 0x7fbd0c040a30 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:36:23.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:23.454+0000 7fbd2f200700 1 --2- 172.21.15.60:0/3994666053 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd28105130 0x7fbd2819a8d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:36:23.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:23.454+0000 7fbd2f200700 1 -- 172.21.15.60:0/3994666053 >> 172.21.15.60:0/3994666053 conn(0x7fbd281006f0 msgr2=0x7fbd28101050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:36:23.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:23.454+0000 7fbd2f200700 1 -- 172.21.15.60:0/3994666053 shutdown_connections 2024-05-03T14:36:23.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:23.454+0000 7fbd2f200700 1 -- 172.21.15.60:0/3994666053 wait complete. 2024-05-03T14:36:24.236 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi090 2024-05-03T14:36:24.236 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-05-03T14:36:24.236 DEBUG:teuthology.orchestra.run.smithi090:> dd of=/etc/ceph/ceph.conf 2024-05-03T14:36:24.255 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-05-03T14:36:24.255 DEBUG:teuthology.orchestra.run.smithi090:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-03T14:36:24.312 INFO:tasks.cephadm:Adding host smithi090 to orchestrator... 2024-05-03T14:36:24.312 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph orch host add smithi090 2024-05-03T14:36:24.725 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:36:24.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:24 smithi060 ceph-mon[28873]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:36:24.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:24 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:36:24.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:24 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:36:24.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:24 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:36:24.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:24 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:36:24.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:24 smithi060 ceph-mon[28873]: Updating smithi060:/etc/ceph/ceph.conf 2024-05-03T14:36:25.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:25 smithi060 ceph-mon[28873]: Updating smithi060:/var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T14:36:25.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:25 smithi060 ceph-mon[28873]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:36:26.554 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:26 smithi060 ceph-mon[28873]: Updating smithi060:/etc/ceph/ceph.client.admin.keyring 2024-05-03T14:36:26.554 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:26 smithi060 ceph-mon[28873]: Updating smithi060:/var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-03T14:36:26.554 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:26 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:36:26.554 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:26 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:36:26.554 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:26 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:36:26.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:26.728+0000 7ff359e00700 1 -- 172.21.15.60:0/3079148044 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff354075840 msgr2=0x7ff354075c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:36:26.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:26.728+0000 7ff359e00700 1 --2- 172.21.15.60:0/3079148044 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff354075840 0x7ff354075c20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff348009910 tx=0x7ff348009c20 comp rx=0 tx=0).stop 2024-05-03T14:36:26.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:26.728+0000 7ff359e00700 1 -- 172.21.15.60:0/3079148044 shutdown_connections 2024-05-03T14:36:26.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:26.728+0000 7ff359e00700 1 --2- 172.21.15.60:0/3079148044 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff354075840 0x7ff354075c20 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:36:26.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:26.728+0000 7ff359e00700 1 -- 172.21.15.60:0/3079148044 >> 172.21.15.60:0/3079148044 conn(0x7ff3540fe800 msgr2=0x7ff354100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:36:26.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:26.728+0000 7ff359e00700 1 -- 172.21.15.60:0/3079148044 shutdown_connections 2024-05-03T14:36:26.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:26.728+0000 7ff359e00700 1 -- 172.21.15.60:0/3079148044 wait complete. 2024-05-03T14:36:26.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:26.729+0000 7ff359e00700 1 Processor -- start 2024-05-03T14:36:26.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:26.729+0000 7ff359e00700 1 -- start start 2024-05-03T14:36:26.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:26.730+0000 7ff359e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff354075840 0x7ff35419eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:36:26.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:26.730+0000 7ff359e00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff354113ad0 con 0x7ff354075840 2024-05-03T14:36:26.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:26.730+0000 7ff358a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff354075840 0x7ff35419eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:36:26.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:26.730+0000 7ff358a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff354075840 0x7ff35419eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36458/0 (socket says 172.21.15.60:36458) 2024-05-03T14:36:26.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:26.730+0000 7ff358a00700 1 -- 172.21.15.60:0/3400731690 learned_addr learned my addr 172.21.15.60:0/3400731690 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:36:26.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:26.731+0000 7ff358a00700 1 -- 172.21.15.60:0/3400731690 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff35419f400 con 0x7ff354075840 2024-05-03T14:36:26.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:26.731+0000 7ff358a00700 1 --2- 172.21.15.60:0/3400731690 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff354075840 0x7ff35419eec0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff348004c10 tx=0x7ff348004cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:36:26.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:26.731+0000 7ff351600700 1 -- 172.21.15.60:0/3400731690 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff348026070 con 0x7ff354075840 2024-05-03T14:36:26.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:26.731+0000 7ff351600700 1 -- 172.21.15.60:0/3400731690 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7ff348004370 con 0x7ff354075840 2024-05-03T14:36:26.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:26.731+0000 7ff359e00700 1 -- 172.21.15.60:0/3400731690 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff35419f690 con 0x7ff354075840 2024-05-03T14:36:26.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:26.731+0000 7ff359e00700 1 -- 172.21.15.60:0/3400731690 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff35419faf0 con 0x7ff354075840 2024-05-03T14:36:26.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:26.732+0000 7ff351600700 1 -- 172.21.15.60:0/3400731690 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff348003c00 con 0x7ff354075840 2024-05-03T14:36:26.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:26.733+0000 7ff359e00700 1 -- 172.21.15.60:0/3400731690 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff334005330 con 0x7ff354075840 2024-05-03T14:36:26.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:26.733+0000 7ff351600700 1 -- 172.21.15.60:0/3400731690 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7ff348029070 con 0x7ff354075840 2024-05-03T14:36:26.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:26.733+0000 7ff351600700 1 --2- 172.21.15.60:0/3400731690 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7ff33c03e570 0x7ff33c040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:36:26.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:26.733+0000 7ff351600700 1 -- 172.21.15.60:0/3400731690 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff34801e070 con 0x7ff354075840 2024-05-03T14:36:26.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:26.734+0000 7ff353e00700 1 --2- 172.21.15.60:0/3400731690 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7ff33c03e570 0x7ff33c040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:36:26.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:26.734+0000 7ff353e00700 1 --2- 172.21.15.60:0/3400731690 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7ff33c03e570 0x7ff33c040a30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff340006fd0 tx=0x7ff340006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:36:26.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:26.739+0000 7ff351600700 1 -- 172.21.15.60:0/3400731690 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7ff348013070 con 0x7ff354075840 2024-05-03T14:36:26.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:26.987+0000 7ff359e00700 1 -- 172.21.15.60:0/3400731690 --> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi090", "target": ["mon-mgr", ""]}) -- 0x7ff334000bc0 con 0x7ff33c03e570 2024-05-03T14:36:27.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:27 smithi060 ceph-mon[28873]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:36:28.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:28 smithi060 ceph-mon[28873]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi090", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:36:29.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:29 smithi060 ceph-mon[28873]: Deploying cephadm binary to smithi090 2024-05-03T14:36:29.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:29 smithi060 ceph-mon[28873]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:36:31.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:31 smithi060 ceph-mon[28873]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:36:31.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:31.846+0000 7ff351600700 1 -- 172.21.15.60:0/3400731690 <== mgr.14152 v2:172.21.15.60:6800/3066142956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7ff334000bc0 con 0x7ff33c03e570 2024-05-03T14:36:31.849 INFO:teuthology.orchestra.run.smithi060.stdout:Added host 'smithi090' with addr '172.21.15.90' 2024-05-03T14:36:31.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:31.850+0000 7ff359e00700 1 -- 172.21.15.60:0/3400731690 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7ff33c03e570 msgr2=0x7ff33c040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:36:31.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:31.850+0000 7ff359e00700 1 --2- 172.21.15.60:0/3400731690 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7ff33c03e570 0x7ff33c040a30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff340006fd0 tx=0x7ff340006e40 comp rx=0 tx=0).stop 2024-05-03T14:36:31.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:31.850+0000 7ff359e00700 1 -- 172.21.15.60:0/3400731690 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff354075840 msgr2=0x7ff35419eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:36:31.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:31.850+0000 7ff359e00700 1 --2- 172.21.15.60:0/3400731690 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff354075840 0x7ff35419eec0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff348004c10 tx=0x7ff348004cf0 comp rx=0 tx=0).stop 2024-05-03T14:36:31.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:31.850+0000 7ff359e00700 1 -- 172.21.15.60:0/3400731690 shutdown_connections 2024-05-03T14:36:31.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:31.850+0000 7ff359e00700 1 --2- 172.21.15.60:0/3400731690 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7ff33c03e570 0x7ff33c040a30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:36:31.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:31.850+0000 7ff359e00700 1 --2- 172.21.15.60:0/3400731690 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff354075840 0x7ff35419eec0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:36:31.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:31.850+0000 7ff359e00700 1 -- 172.21.15.60:0/3400731690 >> 172.21.15.60:0/3400731690 conn(0x7ff3540fe800 msgr2=0x7ff354100190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:36:31.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:31.851+0000 7ff359e00700 1 -- 172.21.15.60:0/3400731690 shutdown_connections 2024-05-03T14:36:31.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:31.851+0000 7ff359e00700 1 -- 172.21.15.60:0/3400731690 wait complete. 2024-05-03T14:36:32.507 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph orch host ls --format=json 2024-05-03T14:36:32.920 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:36:33.205 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:32 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:36:33.205 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:32 smithi060 ceph-mon[28873]: Added host smithi090 2024-05-03T14:36:33.205 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:32 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:36:33.205 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:32 smithi060 ceph-mon[28873]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:36:33.205 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:32 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:36:34.154 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:33 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:36:34.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:34.964+0000 7f0712c00700 1 -- 172.21.15.60:0/3376357809 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f070c0758d0 msgr2=0x7f070c075cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:36:34.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:34.964+0000 7f0712c00700 1 --2- 172.21.15.60:0/3376357809 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f070c0758d0 0x7f070c075cb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f06fc00b0a0 tx=0x7f06fc00b3b0 comp rx=0 tx=0).stop 2024-05-03T14:36:34.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:34.964+0000 7f0712c00700 1 -- 172.21.15.60:0/3376357809 shutdown_connections 2024-05-03T14:36:34.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:34.964+0000 7f0712c00700 1 --2- 172.21.15.60:0/3376357809 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f070c0758d0 0x7f070c075cb0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:36:34.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:34.964+0000 7f0712c00700 1 -- 172.21.15.60:0/3376357809 >> 172.21.15.60:0/3376357809 conn(0x7f070c0fe9e0 msgr2=0x7f070c100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:36:34.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:34.964+0000 7f0712c00700 1 -- 172.21.15.60:0/3376357809 shutdown_connections 2024-05-03T14:36:34.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:34.964+0000 7f0712c00700 1 -- 172.21.15.60:0/3376357809 wait complete. 2024-05-03T14:36:34.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:34.966+0000 7f0712c00700 1 Processor -- start 2024-05-03T14:36:34.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:34.966+0000 7f0712c00700 1 -- start start 2024-05-03T14:36:34.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:34.967+0000 7f0712c00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f070c0758d0 0x7f070c102e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:36:34.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:34.967+0000 7f0712c00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f070c10f8a0 con 0x7f070c0758d0 2024-05-03T14:36:34.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:34.967+0000 7f070be00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f070c0758d0 0x7f070c102e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:36:34.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:34.967+0000 7f070be00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f070c0758d0 0x7f070c102e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54546/0 (socket says 172.21.15.60:54546) 2024-05-03T14:36:34.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:34.967+0000 7f070be00700 1 -- 172.21.15.60:0/2754432553 learned_addr learned my addr 172.21.15.60:0/2754432553 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:36:34.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:34.968+0000 7f070be00700 1 -- 172.21.15.60:0/2754432553 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f070c106d70 con 0x7f070c0758d0 2024-05-03T14:36:34.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:34.968+0000 7f070be00700 1 --2- 172.21.15.60:0/2754432553 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f070c0758d0 0x7f070c102e10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f06fc000f80 tx=0x7f06fc003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:36:34.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:34.968+0000 7f0709600700 1 -- 172.21.15.60:0/2754432553 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06fc017040 con 0x7f070c0758d0 2024-05-03T14:36:34.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:34.968+0000 7f0709600700 1 -- 172.21.15.60:0/2754432553 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f06fc022070 con 0x7f070c0758d0 2024-05-03T14:36:34.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:34.968+0000 7f0709600700 1 -- 172.21.15.60:0/2754432553 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06fc01d450 con 0x7f070c0758d0 2024-05-03T14:36:34.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:34.968+0000 7f0712c00700 1 -- 172.21.15.60:0/2754432553 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f070c103350 con 0x7f070c0758d0 2024-05-03T14:36:34.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:34.968+0000 7f0712c00700 1 -- 172.21.15.60:0/2754432553 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f070c1037b0 con 0x7f070c0758d0 2024-05-03T14:36:34.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:34.970+0000 7f0709600700 1 -- 172.21.15.60:0/2754432553 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f06fc0078a0 con 0x7f070c0758d0 2024-05-03T14:36:34.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:34.970+0000 7f0712c00700 1 -- 172.21.15.60:0/2754432553 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f070c076f80 con 0x7f070c0758d0 2024-05-03T14:36:34.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:34.970+0000 7f0709600700 1 --2- 172.21.15.60:0/2754432553 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f06f003e560 0x7f06f0040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:36:34.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:34.970+0000 7f0709600700 1 -- 172.21.15.60:0/2754432553 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f06fc00f050 con 0x7f070c0758d0 2024-05-03T14:36:34.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:34.971+0000 7f0703400700 1 --2- 172.21.15.60:0/2754432553 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f06f003e560 0x7f06f0040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:36:34.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:34.971+0000 7f0703400700 1 --2- 172.21.15.60:0/2754432553 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f06f003e560 0x7f06f0040a20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f06f4006fd0 tx=0x7f06f4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:36:34.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:34.976+0000 7f0709600700 1 -- 172.21.15.60:0/2754432553 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f06fc01c850 con 0x7f070c0758d0 2024-05-03T14:36:35.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:35.226+0000 7f0712c00700 1 -- 172.21.15.60:0/2754432553 --> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f070c104340 con 0x7f06f003e560 2024-05-03T14:36:35.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:35.229+0000 7f0709600700 1 -- 172.21.15.60:0/2754432553 <== mgr.14152 v2:172.21.15.60:6800/3066142956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7f070c104340 con 0x7f06f003e560 2024-05-03T14:36:35.231 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:36:35.232 INFO:teuthology.orchestra.run.smithi060.stdout:[{"addr": "172.21.15.60", "hostname": "smithi060", "labels": [], "status": ""}, {"addr": "172.21.15.90", "hostname": "smithi090", "labels": [], "status": ""}] 2024-05-03T14:36:35.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:35.232+0000 7f0712c00700 1 -- 172.21.15.60:0/2754432553 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f06f003e560 msgr2=0x7f06f0040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:36:35.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:35.232+0000 7f0712c00700 1 --2- 172.21.15.60:0/2754432553 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f06f003e560 0x7f06f0040a20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f06f4006fd0 tx=0x7f06f4006e40 comp rx=0 tx=0).stop 2024-05-03T14:36:35.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:35.232+0000 7f0712c00700 1 -- 172.21.15.60:0/2754432553 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f070c0758d0 msgr2=0x7f070c102e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:36:35.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:35.232+0000 7f0712c00700 1 --2- 172.21.15.60:0/2754432553 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f070c0758d0 0x7f070c102e10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f06fc000f80 tx=0x7f06fc003ce0 comp rx=0 tx=0).stop 2024-05-03T14:36:35.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:35.232+0000 7f0712c00700 1 -- 172.21.15.60:0/2754432553 shutdown_connections 2024-05-03T14:36:35.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:35.232+0000 7f0712c00700 1 --2- 172.21.15.60:0/2754432553 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f06f003e560 0x7f06f0040a20 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:36:35.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:35.232+0000 7f0712c00700 1 --2- 172.21.15.60:0/2754432553 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f070c0758d0 0x7f070c102e10 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:36:35.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:35.232+0000 7f0712c00700 1 -- 172.21.15.60:0/2754432553 >> 172.21.15.60:0/2754432553 conn(0x7f070c0fe9e0 msgr2=0x7f070c107150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:36:35.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:35.233+0000 7f0712c00700 1 -- 172.21.15.60:0/2754432553 shutdown_connections 2024-05-03T14:36:35.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:35.233+0000 7f0712c00700 1 -- 172.21.15.60:0/2754432553 wait complete. 2024-05-03T14:36:35.935 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:35 smithi060 ceph-mon[28873]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:36:35.936 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:35 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:36:35.986 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi149 2024-05-03T14:36:35.986 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-03T14:36:35.986 DEBUG:teuthology.orchestra.run.smithi149:> dd of=/etc/ceph/ceph.conf 2024-05-03T14:36:36.006 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-03T14:36:36.006 DEBUG:teuthology.orchestra.run.smithi149:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-03T14:36:36.063 INFO:tasks.cephadm:Adding host smithi149 to orchestrator... 2024-05-03T14:36:36.063 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph orch host add smithi149 2024-05-03T14:36:36.519 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:36:36.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:36 smithi060 ceph-mon[28873]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T14:36:37.766 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:37 smithi060 ceph-mon[28873]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:36:38.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:38.603+0000 7fc77dc00700 1 -- 172.21.15.60:0/2523394201 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc778105320 msgr2=0x7fc778105700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:36:38.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:38.603+0000 7fc77dc00700 1 --2- 172.21.15.60:0/2523394201 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc778105320 0x7fc778105700 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc768009910 tx=0x7fc768009c20 comp rx=0 tx=0).stop 2024-05-03T14:36:38.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:38.603+0000 7fc77dc00700 1 -- 172.21.15.60:0/2523394201 shutdown_connections 2024-05-03T14:36:38.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:38.603+0000 7fc77dc00700 1 --2- 172.21.15.60:0/2523394201 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc778105320 0x7fc778105700 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:36:38.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:38.603+0000 7fc77dc00700 1 -- 172.21.15.60:0/2523394201 >> 172.21.15.60:0/2523394201 conn(0x7fc778100bb0 msgr2=0x7fc778102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:36:38.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:38.603+0000 7fc77dc00700 1 -- 172.21.15.60:0/2523394201 shutdown_connections 2024-05-03T14:36:38.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:38.603+0000 7fc77dc00700 1 -- 172.21.15.60:0/2523394201 wait complete. 2024-05-03T14:36:38.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:38.604+0000 7fc77dc00700 1 Processor -- start 2024-05-03T14:36:38.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:38.605+0000 7fc77dc00700 1 -- start start 2024-05-03T14:36:38.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:38.605+0000 7fc77dc00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc778105320 0x7fc7781a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:36:38.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:38.605+0000 7fc77dc00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc77810f830 con 0x7fc778105320 2024-05-03T14:36:38.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:38.605+0000 7fc776a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc778105320 0x7fc7781a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:36:38.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:38.605+0000 7fc776a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc778105320 0x7fc7781a08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54554/0 (socket says 172.21.15.60:54554) 2024-05-03T14:36:38.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:38.605+0000 7fc776a00700 1 -- 172.21.15.60:0/1584953536 learned_addr learned my addr 172.21.15.60:0/1584953536 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:36:38.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:38.606+0000 7fc776a00700 1 -- 172.21.15.60:0/1584953536 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7781a0e00 con 0x7fc778105320 2024-05-03T14:36:38.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:38.606+0000 7fc776a00700 1 --2- 172.21.15.60:0/1584953536 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc778105320 0x7fc7781a08c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc768006010 tx=0x7fc768004cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:36:38.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:38.607+0000 7fc76f400700 1 -- 172.21.15.60:0/1584953536 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc768026070 con 0x7fc778105320 2024-05-03T14:36:38.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:38.607+0000 7fc77dc00700 1 -- 172.21.15.60:0/1584953536 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7781a10f0 con 0x7fc778105320 2024-05-03T14:36:38.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:38.607+0000 7fc76f400700 1 -- 172.21.15.60:0/1584953536 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7fc7680044a0 con 0x7fc778105320 2024-05-03T14:36:38.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:38.607+0000 7fc76f400700 1 -- 172.21.15.60:0/1584953536 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc768003a70 con 0x7fc778105320 2024-05-03T14:36:38.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:38.607+0000 7fc77dc00700 1 -- 172.21.15.60:0/1584953536 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc77819aa80 con 0x7fc778105320 2024-05-03T14:36:38.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:38.608+0000 7fc77dc00700 1 -- 172.21.15.60:0/1584953536 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc778106800 con 0x7fc778105320 2024-05-03T14:36:38.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:38.608+0000 7fc76f400700 1 -- 172.21.15.60:0/1584953536 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fc768029070 con 0x7fc778105320 2024-05-03T14:36:38.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:38.609+0000 7fc76f400700 1 --2- 172.21.15.60:0/1584953536 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fc75c03e510 0x7fc75c0409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:36:38.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:38.609+0000 7fc76f400700 1 -- 172.21.15.60:0/1584953536 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc76801e070 con 0x7fc778105320 2024-05-03T14:36:38.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:38.609+0000 7fc776000700 1 --2- 172.21.15.60:0/1584953536 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fc75c03e510 0x7fc75c0409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:36:38.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:38.610+0000 7fc776000700 1 --2- 172.21.15.60:0/1584953536 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fc75c03e510 0x7fc75c0409d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc764006fd0 tx=0x7fc764006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:36:38.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:38.615+0000 7fc76f400700 1 -- 172.21.15.60:0/1584953536 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fc768013070 con 0x7fc778105320 2024-05-03T14:36:38.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:38 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:36:38.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:38 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:36:38.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:38 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:36:38.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:38 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:36:38.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:38 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi090", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:36:38.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:38 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:36:38.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:38 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:36:38.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:38 smithi060 ceph-mon[28873]: Updating smithi090:/etc/ceph/ceph.conf 2024-05-03T14:36:38.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:38.864+0000 7fc77dc00700 1 -- 172.21.15.60:0/1584953536 --> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi149", "target": ["mon-mgr", ""]}) -- 0x7fc77819bfb0 con 0x7fc75c03e510 2024-05-03T14:36:40.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:39 smithi060 ceph-mon[28873]: Updating smithi090:/var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T14:36:40.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:39 smithi060 ceph-mon[28873]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:36:40.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:39 smithi060 ceph-mon[28873]: Updating smithi090:/etc/ceph/ceph.client.admin.keyring 2024-05-03T14:36:40.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:39 smithi060 ceph-mon[28873]: Updating smithi090:/var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-03T14:36:40.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:39 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:36:40.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:39 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:36:40.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:39 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:36:41.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:40 smithi060 ceph-mon[28873]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi149", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:36:42.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:41 smithi060 ceph-mon[28873]: Deploying cephadm binary to smithi149 2024-05-03T14:36:42.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:41 smithi060 ceph-mon[28873]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:36:43.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:43.851+0000 7fc76f400700 1 -- 172.21.15.60:0/1584953536 <== mgr.14152 v2:172.21.15.60:6800/3066142956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fc77819bfb0 con 0x7fc75c03e510 2024-05-03T14:36:43.854 INFO:teuthology.orchestra.run.smithi060.stdout:Added host 'smithi149' with addr '172.21.15.149' 2024-05-03T14:36:43.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:43.855+0000 7fc77dc00700 1 -- 172.21.15.60:0/1584953536 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fc75c03e510 msgr2=0x7fc75c0409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:36:43.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:43.855+0000 7fc77dc00700 1 --2- 172.21.15.60:0/1584953536 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fc75c03e510 0x7fc75c0409d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc764006fd0 tx=0x7fc764006e40 comp rx=0 tx=0).stop 2024-05-03T14:36:43.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:43.855+0000 7fc77dc00700 1 -- 172.21.15.60:0/1584953536 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc778105320 msgr2=0x7fc7781a08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:36:43.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:43.855+0000 7fc77dc00700 1 --2- 172.21.15.60:0/1584953536 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc778105320 0x7fc7781a08c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc768006010 tx=0x7fc768004cc0 comp rx=0 tx=0).stop 2024-05-03T14:36:43.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:43.856+0000 7fc77dc00700 1 -- 172.21.15.60:0/1584953536 shutdown_connections 2024-05-03T14:36:43.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:43.856+0000 7fc77dc00700 1 --2- 172.21.15.60:0/1584953536 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fc75c03e510 0x7fc75c0409d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:36:43.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:43.856+0000 7fc77dc00700 1 --2- 172.21.15.60:0/1584953536 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc778105320 0x7fc7781a08c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:36:43.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:43.856+0000 7fc77dc00700 1 -- 172.21.15.60:0/1584953536 >> 172.21.15.60:0/1584953536 conn(0x7fc778100bb0 msgr2=0x7fc778107fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:36:43.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:43.856+0000 7fc77dc00700 1 -- 172.21.15.60:0/1584953536 shutdown_connections 2024-05-03T14:36:43.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:43.856+0000 7fc77dc00700 1 -- 172.21.15.60:0/1584953536 wait complete. 2024-05-03T14:36:44.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:43 smithi060 ceph-mon[28873]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:36:44.658 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph orch host ls --format=json 2024-05-03T14:36:45.131 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:36:45.279 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:44 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:36:45.279 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:44 smithi060 ceph-mon[28873]: Added host smithi149 2024-05-03T14:36:45.279 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:44 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:36:45.279 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:44 smithi060 ceph-mon[28873]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:36:45.279 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:44 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:36:46.301 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:45 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:36:47.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:47.182+0000 7f6df0400700 1 -- 172.21.15.60:0/2243259736 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6de8104050 msgr2=0x7f6de8104430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:36:47.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:47.182+0000 7f6df0400700 1 --2- 172.21.15.60:0/2243259736 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6de8104050 0x7f6de8104430 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc009910 tx=0x7f6ddc009c20 comp rx=0 tx=0).stop 2024-05-03T14:36:47.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:47.183+0000 7f6df0400700 1 -- 172.21.15.60:0/2243259736 shutdown_connections 2024-05-03T14:36:47.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:47.183+0000 7f6df0400700 1 --2- 172.21.15.60:0/2243259736 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6de8104050 0x7f6de8104430 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:36:47.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:47.183+0000 7f6df0400700 1 -- 172.21.15.60:0/2243259736 >> 172.21.15.60:0/2243259736 conn(0x7f6de80fd8a0 msgr2=0x7f6de80ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:36:47.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:47.183+0000 7f6df0400700 1 -- 172.21.15.60:0/2243259736 shutdown_connections 2024-05-03T14:36:47.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:47.183+0000 7f6df0400700 1 -- 172.21.15.60:0/2243259736 wait complete. 2024-05-03T14:36:47.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:47.184+0000 7f6df0400700 1 Processor -- start 2024-05-03T14:36:47.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:47.184+0000 7f6df0400700 1 -- start start 2024-05-03T14:36:47.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:47.185+0000 7f6df0400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6de8104050 0x7f6de819e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:36:47.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:47.185+0000 7f6df0400700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f6de810d680 con 0x7f6de8104050 2024-05-03T14:36:47.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:47.185+0000 7f6deda00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6de8104050 0x7f6de819e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:36:47.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:47.185+0000 7f6deda00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6de8104050 0x7f6de819e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43734/0 (socket says 172.21.15.60:43734) 2024-05-03T14:36:47.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:47.185+0000 7f6deda00700 1 -- 172.21.15.60:0/2166705073 learned_addr learned my addr 172.21.15.60:0/2166705073 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:36:47.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:47.186+0000 7f6deda00700 1 -- 172.21.15.60:0/2166705073 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6de819ec30 con 0x7f6de8104050 2024-05-03T14:36:47.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:47.186+0000 7f6deda00700 1 --2- 172.21.15.60:0/2166705073 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6de8104050 0x7f6de819e6f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc006010 tx=0x7f6ddc004cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:36:47.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:47.186+0000 7f6de6000700 1 -- 172.21.15.60:0/2166705073 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ddc026070 con 0x7f6de8104050 2024-05-03T14:36:47.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:47.186+0000 7f6de6000700 1 -- 172.21.15.60:0/2166705073 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f6ddc0044a0 con 0x7f6de8104050 2024-05-03T14:36:47.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:47.186+0000 7f6de6000700 1 -- 172.21.15.60:0/2166705073 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ddc003ba0 con 0x7f6de8104050 2024-05-03T14:36:47.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:47.186+0000 7f6df0400700 1 -- 172.21.15.60:0/2166705073 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6de819eec0 con 0x7f6de8104050 2024-05-03T14:36:47.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:47.186+0000 7f6df0400700 1 -- 172.21.15.60:0/2166705073 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6de81988b0 con 0x7f6de8104050 2024-05-03T14:36:47.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:47.188+0000 7f6de6000700 1 -- 172.21.15.60:0/2166705073 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f6ddc029070 con 0x7f6de8104050 2024-05-03T14:36:47.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:47.188+0000 7f6df0400700 1 -- 172.21.15.60:0/2166705073 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6de806ba30 con 0x7f6de8104050 2024-05-03T14:36:47.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:47.188+0000 7f6de6000700 1 --2- 172.21.15.60:0/2166705073 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f6dd003e570 0x7f6dd0040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:36:47.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:47.188+0000 7f6de6000700 1 -- 172.21.15.60:0/2166705073 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6ddc01e070 con 0x7f6de8104050 2024-05-03T14:36:47.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:47.189+0000 7f6ded000700 1 --2- 172.21.15.60:0/2166705073 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f6dd003e570 0x7f6dd0040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:36:47.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:47.189+0000 7f6ded000700 1 --2- 172.21.15.60:0/2166705073 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f6dd003e570 0x7f6dd0040a30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4006fd0 tx=0x7f6dd4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:36:47.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:47.194+0000 7f6de6000700 1 -- 172.21.15.60:0/2166705073 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f6ddc013070 con 0x7f6de8104050 2024-05-03T14:36:47.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:47.435+0000 7f6df0400700 1 -- 172.21.15.60:0/2166705073 --> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6de8199de0 con 0x7f6dd003e570 2024-05-03T14:36:47.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:47.437+0000 7f6de6000700 1 -- 172.21.15.60:0/2166705073 <== mgr.14152 v2:172.21.15.60:6800/3066142956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7f6de8199de0 con 0x7f6dd003e570 2024-05-03T14:36:47.440 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:36:47.440 INFO:teuthology.orchestra.run.smithi060.stdout:[{"addr": "172.21.15.60", "hostname": "smithi060", "labels": [], "status": ""}, {"addr": "172.21.15.90", "hostname": "smithi090", "labels": [], "status": ""}, {"addr": "172.21.15.149", "hostname": "smithi149", "labels": [], "status": ""}] 2024-05-03T14:36:47.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:47.440+0000 7f6df0400700 1 -- 172.21.15.60:0/2166705073 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f6dd003e570 msgr2=0x7f6dd0040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:36:47.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:47.440+0000 7f6df0400700 1 --2- 172.21.15.60:0/2166705073 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f6dd003e570 0x7f6dd0040a30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4006fd0 tx=0x7f6dd4006e40 comp rx=0 tx=0).stop 2024-05-03T14:36:47.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:47.440+0000 7f6df0400700 1 -- 172.21.15.60:0/2166705073 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6de8104050 msgr2=0x7f6de819e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:36:47.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:47.440+0000 7f6df0400700 1 --2- 172.21.15.60:0/2166705073 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6de8104050 0x7f6de819e6f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc006010 tx=0x7f6ddc004cc0 comp rx=0 tx=0).stop 2024-05-03T14:36:47.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:47.440+0000 7f6df0400700 1 -- 172.21.15.60:0/2166705073 shutdown_connections 2024-05-03T14:36:47.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:47.440+0000 7f6df0400700 1 --2- 172.21.15.60:0/2166705073 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f6dd003e570 0x7f6dd0040a30 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:36:47.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:47.440+0000 7f6df0400700 1 --2- 172.21.15.60:0/2166705073 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6de8104050 0x7f6de819e6f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:36:47.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:47.440+0000 7f6df0400700 1 -- 172.21.15.60:0/2166705073 >> 172.21.15.60:0/2166705073 conn(0x7f6de80fd8a0 msgr2=0x7f6de8104e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:36:47.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:47.440+0000 7f6df0400700 1 -- 172.21.15.60:0/2166705073 shutdown_connections 2024-05-03T14:36:47.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:47.441+0000 7f6df0400700 1 -- 172.21.15.60:0/2166705073 wait complete. 2024-05-03T14:36:47.814 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:47 smithi060 ceph-mon[28873]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:36:47.814 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:47 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:36:48.147 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-03T14:36:48.148 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph osd crush tunables default 2024-05-03T14:36:48.582 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:36:48.899 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:48 smithi060 ceph-mon[28873]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T14:36:49.833 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:49 smithi060 ceph-mon[28873]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:36:50.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:50.656+0000 7f9335c00700 1 -- 172.21.15.60:0/667982738 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9330103110 msgr2=0x7f93301034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:36:50.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:50.656+0000 7f9335c00700 1 --2- 172.21.15.60:0/667982738 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9330103110 0x7f93301034f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9320009910 tx=0x7f9320009c20 comp rx=0 tx=0).stop 2024-05-03T14:36:50.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:50.656+0000 7f9335c00700 1 -- 172.21.15.60:0/667982738 shutdown_connections 2024-05-03T14:36:50.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:50.656+0000 7f9335c00700 1 --2- 172.21.15.60:0/667982738 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9330103110 0x7f93301034f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:36:50.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:50.656+0000 7f9335c00700 1 -- 172.21.15.60:0/667982738 >> 172.21.15.60:0/667982738 conn(0x7f93300fe9e0 msgr2=0x7f9330100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:36:50.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:50.656+0000 7f9335c00700 1 -- 172.21.15.60:0/667982738 shutdown_connections 2024-05-03T14:36:50.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:50.656+0000 7f9335c00700 1 -- 172.21.15.60:0/667982738 wait complete. 2024-05-03T14:36:50.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:50.657+0000 7f9335c00700 1 Processor -- start 2024-05-03T14:36:50.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:50.657+0000 7f9335c00700 1 -- start start 2024-05-03T14:36:50.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:50.658+0000 7f9335c00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9330103110 0x7f93301a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:36:50.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:50.658+0000 7f9335c00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f933010d620 con 0x7f9330103110 2024-05-03T14:36:50.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:50.658+0000 7f932ea00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9330103110 0x7f93301a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:36:50.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:50.658+0000 7f932ea00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9330103110 0x7f93301a0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43748/0 (socket says 172.21.15.60:43748) 2024-05-03T14:36:50.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:50.658+0000 7f932ea00700 1 -- 172.21.15.60:0/931102818 learned_addr learned my addr 172.21.15.60:0/931102818 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:36:50.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:50.658+0000 7f932ea00700 1 -- 172.21.15.60:0/931102818 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93301a0dc0 con 0x7f9330103110 2024-05-03T14:36:50.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:50.659+0000 7f932ea00700 1 --2- 172.21.15.60:0/931102818 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9330103110 0x7f93301a0880 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9320006010 tx=0x7f9320004cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:36:50.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:50.659+0000 7f9327400700 1 -- 172.21.15.60:0/931102818 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9320026070 con 0x7f9330103110 2024-05-03T14:36:50.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:50.659+0000 7f9327400700 1 -- 172.21.15.60:0/931102818 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f93200044a0 con 0x7f9330103110 2024-05-03T14:36:50.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:50.659+0000 7f9327400700 1 -- 172.21.15.60:0/931102818 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9320003a70 con 0x7f9330103110 2024-05-03T14:36:50.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:50.659+0000 7f9335c00700 1 -- 172.21.15.60:0/931102818 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93301a10b0 con 0x7f9330103110 2024-05-03T14:36:50.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:50.659+0000 7f9335c00700 1 -- 172.21.15.60:0/931102818 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f933019aa40 con 0x7f9330103110 2024-05-03T14:36:50.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:50.661+0000 7f9335c00700 1 -- 172.21.15.60:0/931102818 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93301045f0 con 0x7f9330103110 2024-05-03T14:36:50.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:50.661+0000 7f9327400700 1 -- 172.21.15.60:0/931102818 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f9320029070 con 0x7f9330103110 2024-05-03T14:36:50.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:50.661+0000 7f9327400700 1 --2- 172.21.15.60:0/931102818 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f931403e570 0x7f9314040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:36:50.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:50.661+0000 7f9327400700 1 -- 172.21.15.60:0/931102818 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f932001e070 con 0x7f9330103110 2024-05-03T14:36:50.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:50.662+0000 7f9327e00700 1 --2- 172.21.15.60:0/931102818 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f931403e570 0x7f9314040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:36:50.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:50.662+0000 7f9327e00700 1 --2- 172.21.15.60:0/931102818 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f931403e570 0x7f9314040a30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9318006fd0 tx=0x7f9318006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:36:50.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:50.667+0000 7f9327400700 1 -- 172.21.15.60:0/931102818 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f9320013070 con 0x7f9330103110 2024-05-03T14:36:50.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:50.915+0000 7f9335c00700 1 -- 172.21.15.60:0/931102818 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f933019bf70 con 0x7f9330103110 2024-05-03T14:36:51.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:50 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:36:51.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:50 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:36:51.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:50 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:36:51.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:50 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:36:51.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:50 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:36:51.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:50 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:36:51.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:50 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:36:51.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:50 smithi060 ceph-mon[28873]: Updating smithi149:/etc/ceph/ceph.conf 2024-05-03T14:36:51.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:50 smithi060 ceph-mon[28873]: Updating smithi149:/var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T14:36:51.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:50 smithi060 ceph-mon[28873]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:36:51.340 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:50 smithi060 ceph-mon[28873]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2024-05-03T14:36:51.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:51.876+0000 7f9327400700 1 -- 172.21.15.60:0/931102818 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f9320003c90 con 0x7f9330103110 2024-05-03T14:36:51.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:51.880+0000 7f9335c00700 1 -- 172.21.15.60:0/931102818 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f931403e570 msgr2=0x7f9314040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:36:51.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:51.880+0000 7f9335c00700 1 --2- 172.21.15.60:0/931102818 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f931403e570 0x7f9314040a30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9318006fd0 tx=0x7f9318006e40 comp rx=0 tx=0).stop 2024-05-03T14:36:51.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:51.880+0000 7f9335c00700 1 -- 172.21.15.60:0/931102818 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9330103110 msgr2=0x7f93301a0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:36:51.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:51.881+0000 7f9335c00700 1 --2- 172.21.15.60:0/931102818 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9330103110 0x7f93301a0880 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9320006010 tx=0x7f9320004cc0 comp rx=0 tx=0).stop 2024-05-03T14:36:51.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:51.881+0000 7f9335c00700 1 -- 172.21.15.60:0/931102818 shutdown_connections 2024-05-03T14:36:51.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:51.881+0000 7f9335c00700 1 --2- 172.21.15.60:0/931102818 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f931403e570 0x7f9314040a30 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:36:51.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:51.881+0000 7f9335c00700 1 --2- 172.21.15.60:0/931102818 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9330103110 0x7f93301a0880 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:36:51.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:51.881+0000 7f9335c00700 1 -- 172.21.15.60:0/931102818 >> 172.21.15.60:0/931102818 conn(0x7f93300fe9e0 msgr2=0x7f9330077180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:36:51.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:51.881+0000 7f9335c00700 1 -- 172.21.15.60:0/931102818 shutdown_connections 2024-05-03T14:36:51.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:36:51.881+0000 7f9335c00700 1 -- 172.21.15.60:0/931102818 wait complete. 2024-05-03T14:36:51.885 INFO:teuthology.orchestra.run.smithi060.stderr:adjusted tunables profile to default 2024-05-03T14:36:52.279 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:51 smithi060 ceph-mon[28873]: Updating smithi149:/var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-03T14:36:52.279 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:51 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/931102818' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-03T14:36:52.279 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:51 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:36:52.279 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:51 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:36:52.279 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:51 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:36:52.639 INFO:tasks.cephadm:Adding mon.a on smithi060 2024-05-03T14:36:52.639 INFO:tasks.cephadm:Adding mon.b on smithi090 2024-05-03T14:36:52.640 INFO:tasks.cephadm:Adding mon.c on smithi149 2024-05-03T14:36:52.640 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph orch apply mon '3;smithi060:172.21.15.60=a;smithi090:172.21.15.90=b;smithi149:172.21.15.149=c' 2024-05-03T14:36:52.888 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T14:36:53.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:52 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/931102818' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-03T14:36:53.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:52 smithi060 ceph-mon[28873]: osdmap e4: 0 total, 0 up, 0 in 2024-05-03T14:36:53.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:52 smithi060 ceph-mon[28873]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:36:54.275 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:54.271+0000 7f63e2c57700 1 -- 172.21.15.149:0/3980503911 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63dc101ff0 msgr2=0x7f63dc1023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:36:54.275 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:54.271+0000 7f63e2c57700 1 --2- 172.21.15.149:0/3980503911 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63dc101ff0 0x7f63dc1023d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f63d0009910 tx=0x7f63d0009c20 comp rx=0 tx=0).stop 2024-05-03T14:36:54.275 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:54.271+0000 7f63e2c57700 1 -- 172.21.15.149:0/3980503911 shutdown_connections 2024-05-03T14:36:54.276 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:54.271+0000 7f63e2c57700 1 --2- 172.21.15.149:0/3980503911 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63dc101ff0 0x7f63dc1023d0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:36:54.276 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:54.271+0000 7f63e2c57700 1 -- 172.21.15.149:0/3980503911 >> 172.21.15.149:0/3980503911 conn(0x7f63dc0fd880 msgr2=0x7f63dc0ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:36:54.276 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:54.271+0000 7f63e2c57700 1 -- 172.21.15.149:0/3980503911 shutdown_connections 2024-05-03T14:36:54.276 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:54.271+0000 7f63e2c57700 1 -- 172.21.15.149:0/3980503911 wait complete. 2024-05-03T14:36:54.276 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:54.272+0000 7f63e2c57700 1 Processor -- start 2024-05-03T14:36:54.276 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:54.272+0000 7f63e2c57700 1 -- start start 2024-05-03T14:36:54.276 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:54.272+0000 7f63e2c57700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63dc101ff0 0x7f63dc199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:36:54.276 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:54.272+0000 7f63e2c57700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f63dc10c500 con 0x7f63dc101ff0 2024-05-03T14:36:54.277 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:54.272+0000 7f63e09f3700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63dc101ff0 0x7f63dc199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:36:54.277 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:54.273+0000 7f63e09f3700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63dc101ff0 0x7f63dc199890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.149:51858/0 (socket says 172.21.15.149:51858) 2024-05-03T14:36:54.277 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:54.273+0000 7f63e09f3700 1 -- 172.21.15.149:0/2415709280 learned_addr learned my addr 172.21.15.149:0/2415709280 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-05-03T14:36:54.277 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:54.273+0000 7f63e09f3700 1 -- 172.21.15.149:0/2415709280 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63dc199dd0 con 0x7f63dc101ff0 2024-05-03T14:36:54.277 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:54.274+0000 7f63e09f3700 1 --2- 172.21.15.149:0/2415709280 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63dc101ff0 0x7f63dc199890 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f63d00052a0 tx=0x7f63d0004cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:36:54.278 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:54.274+0000 7f63d9ffb700 1 -- 172.21.15.149:0/2415709280 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63d0026070 con 0x7f63dc101ff0 2024-05-03T14:36:54.278 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:54.274+0000 7f63e2c57700 1 -- 172.21.15.149:0/2415709280 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63dc19a0c0 con 0x7f63dc101ff0 2024-05-03T14:36:54.278 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:54.274+0000 7f63e2c57700 1 -- 172.21.15.149:0/2415709280 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63dc19a580 con 0x7f63dc101ff0 2024-05-03T14:36:54.278 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:54.274+0000 7f63d9ffb700 1 -- 172.21.15.149:0/2415709280 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f63d000bbf0 con 0x7f63dc101ff0 2024-05-03T14:36:54.278 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:54.274+0000 7f63d9ffb700 1 -- 172.21.15.149:0/2415709280 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63d00039b0 con 0x7f63dc101ff0 2024-05-03T14:36:54.279 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:54.275+0000 7f63e2c57700 1 -- 172.21.15.149:0/2415709280 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63dc1034d0 con 0x7f63dc101ff0 2024-05-03T14:36:54.279 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:54.276+0000 7f63d9ffb700 1 -- 172.21.15.149:0/2415709280 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f63d0029070 con 0x7f63dc101ff0 2024-05-03T14:36:54.279 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:54.276+0000 7f63d9ffb700 1 --2- 172.21.15.149:0/2415709280 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f63cc03e510 0x7f63cc0409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:36:54.279 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:54.276+0000 7f63d9ffb700 1 -- 172.21.15.149:0/2415709280 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f63d001e070 con 0x7f63dc101ff0 2024-05-03T14:36:54.280 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:54.277+0000 7f63dbfff700 1 --2- 172.21.15.149:0/2415709280 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f63cc03e510 0x7f63cc0409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:36:54.281 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:54.278+0000 7f63dbfff700 1 --2- 172.21.15.149:0/2415709280 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f63cc03e510 0x7f63cc0409d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f63c8006fd0 tx=0x7f63c8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:36:54.296 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:54.292+0000 7f63d9ffb700 1 -- 172.21.15.149:0/2415709280 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f63d0013070 con 0x7f63dc101ff0 2024-05-03T14:36:54.511 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:54.507+0000 7f63e2c57700 1 -- 172.21.15.149:0/2415709280 --> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi060:172.21.15.60=a;smithi090:172.21.15.90=b;smithi149:172.21.15.149=c", "target": ["mon-mgr", ""]}) -- 0x7f63dc06b940 con 0x7f63cc03e510 2024-05-03T14:36:54.526 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:54.523+0000 7f63d9ffb700 1 -- 172.21.15.149:0/2415709280 <== mgr.14152 v2:172.21.15.60:6800/3066142956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f63dc06b940 con 0x7f63cc03e510 2024-05-03T14:36:54.526 INFO:teuthology.orchestra.run.smithi149.stdout:Scheduled mon update... 2024-05-03T14:36:54.528 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:54.525+0000 7f63e2c57700 1 -- 172.21.15.149:0/2415709280 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f63cc03e510 msgr2=0x7f63cc0409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:36:54.528 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:54.525+0000 7f63e2c57700 1 --2- 172.21.15.149:0/2415709280 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f63cc03e510 0x7f63cc0409d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f63c8006fd0 tx=0x7f63c8006e40 comp rx=0 tx=0).stop 2024-05-03T14:36:54.529 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:54.525+0000 7f63e2c57700 1 -- 172.21.15.149:0/2415709280 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63dc101ff0 msgr2=0x7f63dc199890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:36:54.529 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:54.525+0000 7f63e2c57700 1 --2- 172.21.15.149:0/2415709280 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63dc101ff0 0x7f63dc199890 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f63d00052a0 tx=0x7f63d0004cc0 comp rx=0 tx=0).stop 2024-05-03T14:36:54.529 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:54.525+0000 7f63e2c57700 1 -- 172.21.15.149:0/2415709280 shutdown_connections 2024-05-03T14:36:54.529 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:54.525+0000 7f63e2c57700 1 --2- 172.21.15.149:0/2415709280 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f63cc03e510 0x7f63cc0409d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:36:54.529 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:54.525+0000 7f63e2c57700 1 --2- 172.21.15.149:0/2415709280 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63dc101ff0 0x7f63dc199890 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:36:54.529 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:54.525+0000 7f63e2c57700 1 -- 172.21.15.149:0/2415709280 >> 172.21.15.149:0/2415709280 conn(0x7f63dc0fd880 msgr2=0x7f63dc1039c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:36:54.529 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:54.525+0000 7f63e2c57700 1 -- 172.21.15.149:0/2415709280 shutdown_connections 2024-05-03T14:36:54.529 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:54.525+0000 7f63e2c57700 1 -- 172.21.15.149:0/2415709280 wait complete. 2024-05-03T14:36:55.148 DEBUG:teuthology.orchestra.run.smithi090:mon.b> sudo journalctl -f -n 0 -u ceph-285d64b2-095a-11ef-bc97-c7b262605968@mon.b.service 2024-05-03T14:36:55.151 DEBUG:teuthology.orchestra.run.smithi149:mon.c> sudo journalctl -f -n 0 -u ceph-285d64b2-095a-11ef-bc97-c7b262605968@mon.c.service 2024-05-03T14:36:55.153 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-03T14:36:55.153 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-03T14:36:55.809 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T14:36:55.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:55 smithi060 ceph-mon[28873]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:36:55.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:55 smithi060 ceph-mon[28873]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi060:172.21.15.60=a;smithi090:172.21.15.90=b;smithi149:172.21.15.149=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:36:55.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:55 smithi060 ceph-mon[28873]: Saving service mon spec with placement smithi060:172.21.15.60=a;smithi090:172.21.15.90=b;smithi149:172.21.15.149=c;count:3 2024-05-03T14:36:55.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:55 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:36:55.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:55 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:36:55.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:55 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:36:55.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:55 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:36:55.839 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:55 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:36:55.839 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:55 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T14:36:55.839 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:55 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:36:55.839 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:55 smithi060 ceph-mon[28873]: Deploying daemon mon.c on smithi149 2024-05-03T14:36:57.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:57 smithi060 ceph-mon[28873]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:36:58.278 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:58.274+0000 7f9441af2700 1 -- 172.21.15.149:0/4223984353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f943c102050 msgr2=0x7f943c102430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:36:58.278 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:58.274+0000 7f9441af2700 1 --2- 172.21.15.149:0/4223984353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f943c102050 0x7f943c102430 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f942c009910 tx=0x7f942c009c20 comp rx=0 tx=0).stop 2024-05-03T14:36:58.278 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:58.274+0000 7f9441af2700 1 -- 172.21.15.149:0/4223984353 shutdown_connections 2024-05-03T14:36:58.278 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:58.274+0000 7f9441af2700 1 --2- 172.21.15.149:0/4223984353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f943c102050 0x7f943c102430 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:36:58.278 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:58.274+0000 7f9441af2700 1 -- 172.21.15.149:0/4223984353 >> 172.21.15.149:0/4223984353 conn(0x7f943c077900 msgr2=0x7f943c077d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:36:58.278 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:58.274+0000 7f9441af2700 1 -- 172.21.15.149:0/4223984353 shutdown_connections 2024-05-03T14:36:58.278 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:58.274+0000 7f9441af2700 1 -- 172.21.15.149:0/4223984353 wait complete. 2024-05-03T14:36:58.279 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:58.275+0000 7f9441af2700 1 Processor -- start 2024-05-03T14:36:58.279 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:58.275+0000 7f9441af2700 1 -- start start 2024-05-03T14:36:58.279 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:58.275+0000 7f9441af2700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f943c102050 0x7f943c197690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:36:58.279 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:58.275+0000 7f9441af2700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f943c10c510 con 0x7f943c102050 2024-05-03T14:36:58.279 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:58.276+0000 7f943b7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f943c102050 0x7f943c197690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:36:58.279 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:58.276+0000 7f943b7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f943c102050 0x7f943c197690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.149:56730/0 (socket says 172.21.15.149:56730) 2024-05-03T14:36:58.279 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:58.276+0000 7f943b7fe700 1 -- 172.21.15.149:0/1993036852 learned_addr learned my addr 172.21.15.149:0/1993036852 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-05-03T14:36:58.280 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:58.277+0000 7f943b7fe700 1 -- 172.21.15.149:0/1993036852 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f943c06b6c0 con 0x7f943c102050 2024-05-03T14:36:58.280 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:58.277+0000 7f943b7fe700 1 --2- 172.21.15.149:0/1993036852 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f943c102050 0x7f943c197690 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f942c0052a0 tx=0x7f942c004cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:36:58.281 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:58.277+0000 7f94397fa700 1 -- 172.21.15.149:0/1993036852 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f942c026070 con 0x7f943c102050 2024-05-03T14:36:58.281 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:58.277+0000 7f9441af2700 1 -- 172.21.15.149:0/1993036852 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f943c197bd0 con 0x7f943c102050 2024-05-03T14:36:58.281 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:58.278+0000 7f9441af2700 1 -- 172.21.15.149:0/1993036852 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f943c198090 con 0x7f943c102050 2024-05-03T14:36:58.281 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:58.278+0000 7f94397fa700 1 -- 172.21.15.149:0/1993036852 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f942c00bbf0 con 0x7f943c102050 2024-05-03T14:36:58.281 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:58.278+0000 7f94397fa700 1 -- 172.21.15.149:0/1993036852 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f942c0039b0 con 0x7f943c102050 2024-05-03T14:36:58.281 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:58.278+0000 7f9441af2700 1 -- 172.21.15.149:0/1993036852 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f943c1034e0 con 0x7f943c102050 2024-05-03T14:36:58.282 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:58.279+0000 7f94397fa700 1 -- 172.21.15.149:0/1993036852 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f942c029070 con 0x7f943c102050 2024-05-03T14:36:58.282 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:58.279+0000 7f94397fa700 1 --2- 172.21.15.149:0/1993036852 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f942803e570 0x7f9428040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:36:58.282 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:58.279+0000 7f94397fa700 1 -- 172.21.15.149:0/1993036852 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f942c01e070 con 0x7f943c102050 2024-05-03T14:36:58.283 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:58.280+0000 7f9432ffd700 1 --2- 172.21.15.149:0/1993036852 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f942803e570 0x7f9428040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:36:58.284 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:58.281+0000 7f9432ffd700 1 --2- 172.21.15.149:0/1993036852 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f942803e570 0x7f9428040a30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9424006fd0 tx=0x7f9424006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:36:58.298 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:58.296+0000 7f94397fa700 1 -- 172.21.15.149:0/1993036852 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f942c013070 con 0x7f943c102050 2024-05-03T14:36:58.594 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:58.590+0000 7f9441af2700 1 -- 172.21.15.149:0/1993036852 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f943c06aeb0 con 0x7f943c102050 2024-05-03T14:36:58.595 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:58.591+0000 7f94397fa700 1 -- 172.21.15.149:0/1993036852 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f942c01f310 con 0x7f943c102050 2024-05-03T14:36:58.595 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-03T14:36:58.595 INFO:teuthology.orchestra.run.smithi149.stdout:{"epoch":1,"fsid":"285d64b2-095a-11ef-bc97-c7b262605968","modified":"2024-05-03T14:34:40.229121Z","created":"2024-05-03T14:34:40.229121Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:3300","nonce":0},{"type":"v1","addr":"172.21.15.60:6789","nonce":0}]},"addr":"172.21.15.60:6789/0","public_addr":"172.21.15.60:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-03T14:36:58.597 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:58.594+0000 7f9441af2700 1 -- 172.21.15.149:0/1993036852 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f942803e570 msgr2=0x7f9428040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:36:58.597 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:58.594+0000 7f9441af2700 1 --2- 172.21.15.149:0/1993036852 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f942803e570 0x7f9428040a30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9424006fd0 tx=0x7f9424006e40 comp rx=0 tx=0).stop 2024-05-03T14:36:58.598 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:58.594+0000 7f9441af2700 1 -- 172.21.15.149:0/1993036852 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f943c102050 msgr2=0x7f943c197690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:36:58.598 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:58.594+0000 7f9441af2700 1 --2- 172.21.15.149:0/1993036852 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f943c102050 0x7f943c197690 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f942c0052a0 tx=0x7f942c004cc0 comp rx=0 tx=0).stop 2024-05-03T14:36:58.598 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:58.594+0000 7f9441af2700 1 -- 172.21.15.149:0/1993036852 shutdown_connections 2024-05-03T14:36:58.598 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:58.594+0000 7f9441af2700 1 --2- 172.21.15.149:0/1993036852 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f942803e570 0x7f9428040a30 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:36:58.598 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:58.594+0000 7f9441af2700 1 --2- 172.21.15.149:0/1993036852 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f943c102050 0x7f943c197690 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:36:58.598 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:58.594+0000 7f9441af2700 1 -- 172.21.15.149:0/1993036852 >> 172.21.15.149:0/1993036852 conn(0x7f943c077900 msgr2=0x7f943c104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:36:58.598 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:58.594+0000 7f9441af2700 1 -- 172.21.15.149:0/1993036852 shutdown_connections 2024-05-03T14:36:58.598 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:36:58.594+0000 7f9441af2700 1 -- 172.21.15.149:0/1993036852 wait complete. 2024-05-03T14:36:58.599 INFO:teuthology.orchestra.run.smithi149.stderr:dumped monmap epoch 1 2024-05-03T14:36:59.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:59 smithi060 ceph-mon[28873]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:36:59.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:36:59 smithi060 ceph-mon[28873]: from='client.? 172.21.15.149:0/1993036852' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-03T14:37:00.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 podman[33313]: 2024-05-03 14:37:00.075859013 +0000 UTC m=+0.123038820 container create 2a9091c8d79b5595dca35e7ec7421628f58e612f25aaa6a5cff7876a0d652c0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mon-c, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5) 2024-05-03T14:37:00.565 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-03T14:37:00.565 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-03T14:37:00.588 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 podman[33313]: 2024-05-03 14:37:00.270022194 +0000 UTC m=+0.317202001 container init 2a9091c8d79b5595dca35e7ec7421628f58e612f25aaa6a5cff7876a0d652c0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mon-c, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430) 2024-05-03T14:37:00.588 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 podman[33313]: 2024-05-03 14:37:00.272889206 +0000 UTC m=+0.320069015 container start 2a9091c8d79b5595dca35e7ec7421628f58e612f25aaa6a5cff7876a0d652c0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0) 2024-05-03T14:37:00.588 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: set uid:gid to 167:167 (ceph:ceph) 2024-05-03T14:37:00.588 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: ceph version 19.0.0-3450-g9a047fe7 (9a047fe7599a549042d06a3fb5b6813a203eabea) squid (dev), process ceph-mon, pid 2 2024-05-03T14:37:00.588 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: pidfile_write: ignore empty --pid-file 2024-05-03T14:37:00.588 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: load: jerasure load: lrc 2024-05-03T14:37:00.588 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: RocksDB version: 7.9.2 2024-05-03T14:37:00.588 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Git sha 0 2024-05-03T14:37:00.588 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Compile date 2024-05-01 21:12:03 2024-05-03T14:37:00.589 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: DB SUMMARY 2024-05-03T14:37:00.589 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: DB Session ID: CJT2MOYLCJ2QU4BQQOA9 2024-05-03T14:37:00.589 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: CURRENT file: CURRENT 2024-05-03T14:37:00.589 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: IDENTITY file: IDENTITY 2024-05-03T14:37:00.589 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-05-03T14:37:00.589 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-05-03T14:37:00.589 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-05-03T14:37:00.589 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.error_if_exists: 0 2024-05-03T14:37:00.589 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.create_if_missing: 0 2024-05-03T14:37:00.589 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.paranoid_checks: 1 2024-05-03T14:37:00.589 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-03T14:37:00.589 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-03T14:37:00.590 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-03T14:37:00.590 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.env: 0x563bb48dbcc0 2024-05-03T14:37:00.590 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.fs: PosixFileSystem 2024-05-03T14:37:00.590 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.info_log: 0x563bb6b47da0 2024-05-03T14:37:00.590 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.max_file_opening_threads: 16 2024-05-03T14:37:00.595 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.statistics: (nil) 2024-05-03T14:37:00.595 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.use_fsync: 0 2024-05-03T14:37:00.595 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.max_log_file_size: 0 2024-05-03T14:37:00.595 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-03T14:37:00.595 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-03T14:37:00.595 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.keep_log_file_num: 1000 2024-05-03T14:37:00.595 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.recycle_log_file_num: 0 2024-05-03T14:37:00.596 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.allow_fallocate: 1 2024-05-03T14:37:00.596 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.allow_mmap_reads: 0 2024-05-03T14:37:00.596 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.allow_mmap_writes: 0 2024-05-03T14:37:00.596 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.use_direct_reads: 0 2024-05-03T14:37:00.596 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-03T14:37:00.596 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.create_missing_column_families: 0 2024-05-03T14:37:00.596 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.db_log_dir: 2024-05-03T14:37:00.596 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.wal_dir: 2024-05-03T14:37:00.596 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-03T14:37:00.596 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-03T14:37:00.596 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-03T14:37:00.596 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-03T14:37:00.597 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-03T14:37:00.597 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-03T14:37:00.597 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.advise_random_on_open: 1 2024-05-03T14:37:00.597 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.db_write_buffer_size: 0 2024-05-03T14:37:00.597 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.write_buffer_manager: 0x563bb6b190e0 2024-05-03T14:37:00.597 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-03T14:37:00.597 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-03T14:37:00.597 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-03T14:37:00.597 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.rate_limiter: (nil) 2024-05-03T14:37:00.597 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-03T14:37:00.597 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.wal_recovery_mode: 2 2024-05-03T14:37:00.597 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.enable_thread_tracking: 0 2024-05-03T14:37:00.597 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.enable_pipelined_write: 0 2024-05-03T14:37:00.597 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.unordered_write: 0 2024-05-03T14:37:00.598 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-03T14:37:00.598 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-03T14:37:00.598 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-03T14:37:00.598 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-03T14:37:00.598 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.row_cache: None 2024-05-03T14:37:00.598 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.wal_filter: None 2024-05-03T14:37:00.598 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-03T14:37:00.598 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.allow_ingest_behind: 0 2024-05-03T14:37:00.598 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.two_write_queues: 0 2024-05-03T14:37:00.598 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.manual_wal_flush: 0 2024-05-03T14:37:00.598 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.wal_compression: 0 2024-05-03T14:37:00.598 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.atomic_flush: 0 2024-05-03T14:37:00.599 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-03T14:37:00.599 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-03T14:37:00.599 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-03T14:37:00.599 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.log_readahead_size: 0 2024-05-03T14:37:00.599 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-03T14:37:00.599 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.best_efforts_recovery: 0 2024-05-03T14:37:00.599 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-03T14:37:00.599 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-03T14:37:00.599 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.allow_data_in_errors: 0 2024-05-03T14:37:00.599 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.db_host_id: __hostname__ 2024-05-03T14:37:00.599 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-03T14:37:00.599 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.max_background_jobs: 2 2024-05-03T14:37:00.599 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.max_background_compactions: -1 2024-05-03T14:37:00.599 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.max_subcompactions: 1 2024-05-03T14:37:00.599 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-03T14:37:00.600 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-03T14:37:00.600 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-03T14:37:00.600 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.max_total_wal_size: 0 2024-05-03T14:37:00.600 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-03T14:37:00.600 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-03T14:37:00.600 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-03T14:37:00.600 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-03T14:37:00.600 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.max_open_files: -1 2024-05-03T14:37:00.600 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.bytes_per_sync: 0 2024-05-03T14:37:00.600 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-03T14:37:00.600 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-03T14:37:00.600 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.compaction_readahead_size: 0 2024-05-03T14:37:00.600 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.max_background_flushes: -1 2024-05-03T14:37:00.601 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Compression algorithms supported: 2024-05-03T14:37:00.601 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-03T14:37:00.601 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: kZSTD supported: 0 2024-05-03T14:37:00.601 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: kXpressCompression supported: 0 2024-05-03T14:37:00.601 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: kLZ4HCCompression supported: 1 2024-05-03T14:37:00.601 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: kZlibCompression supported: 1 2024-05-03T14:37:00.601 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: kSnappyCompression supported: 1 2024-05-03T14:37:00.601 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: kLZ4Compression supported: 1 2024-05-03T14:37:00.601 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: kBZip2Compression supported: 0 2024-05-03T14:37:00.601 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-03T14:37:00.601 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-03T14:37:00.601 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-05-03T14:37:00.601 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-03T14:37:00.601 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-03T14:37:00.601 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.merge_operator: 2024-05-03T14:37:00.602 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.compaction_filter: None 2024-05-03T14:37:00.602 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.compaction_filter_factory: None 2024-05-03T14:37:00.602 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.sst_partitioner_factory: None 2024-05-03T14:37:00.602 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-03T14:37:00.602 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-03T14:37:00.602 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x563bb6b47f60) 2024-05-03T14:37:00.602 INFO:journalctl@ceph.mon.c.smithi149.stdout: cache_index_and_filter_blocks: 1 2024-05-03T14:37:00.602 INFO:journalctl@ceph.mon.c.smithi149.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-03T14:37:00.602 INFO:journalctl@ceph.mon.c.smithi149.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-03T14:37:00.602 INFO:journalctl@ceph.mon.c.smithi149.stdout: pin_top_level_index_and_filter: 1 2024-05-03T14:37:00.602 INFO:journalctl@ceph.mon.c.smithi149.stdout: index_type: 0 2024-05-03T14:37:00.602 INFO:journalctl@ceph.mon.c.smithi149.stdout: data_block_index_type: 0 2024-05-03T14:37:00.602 INFO:journalctl@ceph.mon.c.smithi149.stdout: index_shortening: 1 2024-05-03T14:37:00.602 INFO:journalctl@ceph.mon.c.smithi149.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-03T14:37:00.603 INFO:journalctl@ceph.mon.c.smithi149.stdout: checksum: 4 2024-05-03T14:37:00.603 INFO:journalctl@ceph.mon.c.smithi149.stdout: no_block_cache: 0 2024-05-03T14:37:00.603 INFO:journalctl@ceph.mon.c.smithi149.stdout: block_cache: 0x563bb6b4b1f0 2024-05-03T14:37:00.603 INFO:journalctl@ceph.mon.c.smithi149.stdout: block_cache_name: BinnedLRUCache 2024-05-03T14:37:00.603 INFO:journalctl@ceph.mon.c.smithi149.stdout: block_cache_options: 2024-05-03T14:37:00.603 INFO:journalctl@ceph.mon.c.smithi149.stdout: capacity : 536870912 2024-05-03T14:37:00.603 INFO:journalctl@ceph.mon.c.smithi149.stdout: num_shard_bits : 4 2024-05-03T14:37:00.603 INFO:journalctl@ceph.mon.c.smithi149.stdout: strict_capacity_limit : 0 2024-05-03T14:37:00.603 INFO:journalctl@ceph.mon.c.smithi149.stdout: high_pri_pool_ratio: 0.000 2024-05-03T14:37:00.603 INFO:journalctl@ceph.mon.c.smithi149.stdout: block_cache_compressed: (nil) 2024-05-03T14:37:00.603 INFO:journalctl@ceph.mon.c.smithi149.stdout: persistent_cache: (nil) 2024-05-03T14:37:00.603 INFO:journalctl@ceph.mon.c.smithi149.stdout: block_size: 4096 2024-05-03T14:37:00.603 INFO:journalctl@ceph.mon.c.smithi149.stdout: block_size_deviation: 10 2024-05-03T14:37:00.603 INFO:journalctl@ceph.mon.c.smithi149.stdout: block_restart_interval: 16 2024-05-03T14:37:00.603 INFO:journalctl@ceph.mon.c.smithi149.stdout: index_block_restart_interval: 1 2024-05-03T14:37:00.604 INFO:journalctl@ceph.mon.c.smithi149.stdout: metadata_block_size: 4096 2024-05-03T14:37:00.604 INFO:journalctl@ceph.mon.c.smithi149.stdout: partition_filters: 0 2024-05-03T14:37:00.604 INFO:journalctl@ceph.mon.c.smithi149.stdout: use_delta_encoding: 1 2024-05-03T14:37:00.604 INFO:journalctl@ceph.mon.c.smithi149.stdout: filter_policy: bloomfilter 2024-05-03T14:37:00.604 INFO:journalctl@ceph.mon.c.smithi149.stdout: whole_key_filtering: 1 2024-05-03T14:37:00.604 INFO:journalctl@ceph.mon.c.smithi149.stdout: verify_compression: 0 2024-05-03T14:37:00.604 INFO:journalctl@ceph.mon.c.smithi149.stdout: read_amp_bytes_per_bit: 0 2024-05-03T14:37:00.604 INFO:journalctl@ceph.mon.c.smithi149.stdout: format_version: 5 2024-05-03T14:37:00.604 INFO:journalctl@ceph.mon.c.smithi149.stdout: enable_index_compression: 1 2024-05-03T14:37:00.604 INFO:journalctl@ceph.mon.c.smithi149.stdout: block_align: 0 2024-05-03T14:37:00.604 INFO:journalctl@ceph.mon.c.smithi149.stdout: max_auto_readahead_size: 262144 2024-05-03T14:37:00.604 INFO:journalctl@ceph.mon.c.smithi149.stdout: prepopulate_block_cache: 0 2024-05-03T14:37:00.604 INFO:journalctl@ceph.mon.c.smithi149.stdout: initial_auto_readahead_size: 8192 2024-05-03T14:37:00.604 INFO:journalctl@ceph.mon.c.smithi149.stdout: num_file_reads_for_auto_readahead: 2 2024-05-03T14:37:00.604 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.write_buffer_size: 33554432 2024-05-03T14:37:00.605 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.max_write_buffer_number: 2 2024-05-03T14:37:00.605 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.compression: NoCompression 2024-05-03T14:37:00.605 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.bottommost_compression: Disabled 2024-05-03T14:37:00.605 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.prefix_extractor: nullptr 2024-05-03T14:37:00.605 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-03T14:37:00.605 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.num_levels: 7 2024-05-03T14:37:00.605 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-03T14:37:00.605 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-03T14:37:00.605 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-03T14:37:00.605 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-03T14:37:00.605 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-03T14:37:00.605 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-03T14:37:00.605 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-03T14:37:00.606 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-03T14:37:00.606 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-03T14:37:00.606 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-03T14:37:00.606 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-03T14:37:00.606 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-03T14:37:00.606 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-03T14:37:00.606 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.compression_opts.level: 32767 2024-05-03T14:37:00.606 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.compression_opts.strategy: 0 2024-05-03T14:37:00.606 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-03T14:37:00.606 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-03T14:37:00.606 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-03T14:37:00.606 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-03T14:37:00.606 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.compression_opts.enabled: false 2024-05-03T14:37:00.606 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-03T14:37:00.607 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-03T14:37:00.607 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-03T14:37:00.607 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-03T14:37:00.607 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.target_file_size_base: 67108864 2024-05-03T14:37:00.607 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-03T14:37:00.607 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-03T14:37:00.607 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-03T14:37:00.607 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-03T14:37:00.607 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-03T14:37:00.607 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-03T14:37:00.607 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-03T14:37:00.607 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-03T14:37:00.607 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-03T14:37:00.607 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-03T14:37:00.608 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-03T14:37:00.608 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-03T14:37:00.608 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-03T14:37:00.608 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-03T14:37:00.608 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.arena_block_size: 1048576 2024-05-03T14:37:00.608 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-03T14:37:00.608 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-03T14:37:00.608 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.disable_auto_compactions: 0 2024-05-03T14:37:00.608 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-03T14:37:00.608 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-03T14:37:00.608 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-03T14:37:00.608 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-03T14:37:00.609 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-03T14:37:00.609 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-03T14:37:00.609 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-03T14:37:00.609 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-03T14:37:00.609 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-03T14:37:00.609 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-03T14:37:00.609 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-03T14:37:00.609 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.inplace_update_support: 0 2024-05-03T14:37:00.609 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-03T14:37:00.609 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-03T14:37:00.609 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-03T14:37:00.609 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-03T14:37:00.609 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.bloom_locality: 0 2024-05-03T14:37:00.610 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.max_successive_merges: 0 2024-05-03T14:37:00.610 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-03T14:37:00.610 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.paranoid_file_checks: 0 2024-05-03T14:37:00.610 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.force_consistency_checks: 1 2024-05-03T14:37:00.610 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.report_bg_io_stats: 0 2024-05-03T14:37:00.610 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.ttl: 2592000 2024-05-03T14:37:00.610 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-03T14:37:00.610 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-03T14:37:00.610 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-03T14:37:00.610 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.enable_blob_files: false 2024-05-03T14:37:00.610 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.min_blob_size: 0 2024-05-03T14:37:00.610 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.blob_file_size: 268435456 2024-05-03T14:37:00.610 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-03T14:37:00.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-03T14:37:00.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-03T14:37:00.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-03T14:37:00.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-03T14:37:00.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.blob_file_starting_level: 0 2024-05-03T14:37:00.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-03T14:37:00.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-03T14:37:00.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-03T14:37:00.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 9388610f-34e8-4127-843c-71488bd1c301 2024-05-03T14:37:00.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714747020303602, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-03T14:37:00.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-03T14:37:00.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714747020304057, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714747020, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "9388610f-34e8-4127-843c-71488bd1c301", "db_session_id": "CJT2MOYLCJ2QU4BQQOA9", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-03T14:37:00.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714747020304138, "job": 1, "event": "recovery_finished"} 2024-05-03T14:37:00.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-03T14:37:00.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-03T14:37:00.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x563bb6c68000 2024-05-03T14:37:00.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: DB pointer 0x563bb6c54000 2024-05-03T14:37:00.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-03T14:37:00.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-03T14:37:00.612 INFO:journalctl@ceph.mon.c.smithi149.stdout: ** DB Stats ** 2024-05-03T14:37:00.612 INFO:journalctl@ceph.mon.c.smithi149.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-03T14:37:00.612 INFO:journalctl@ceph.mon.c.smithi149.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-03T14:37:00.612 INFO:journalctl@ceph.mon.c.smithi149.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-03T14:37:00.612 INFO:journalctl@ceph.mon.c.smithi149.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-03T14:37:00.612 INFO:journalctl@ceph.mon.c.smithi149.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-03T14:37:00.612 INFO:journalctl@ceph.mon.c.smithi149.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-03T14:37:00.612 INFO:journalctl@ceph.mon.c.smithi149.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-03T14:37:00.613 INFO:journalctl@ceph.mon.c.smithi149.stdout: 2024-05-03T14:37:00.613 INFO:journalctl@ceph.mon.c.smithi149.stdout: ** Compaction Stats [default] ** 2024-05-03T14:37:00.613 INFO:journalctl@ceph.mon.c.smithi149.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-03T14:37:00.613 INFO:journalctl@ceph.mon.c.smithi149.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-03T14:37:00.613 INFO:journalctl@ceph.mon.c.smithi149.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-03T14:37:00.613 INFO:journalctl@ceph.mon.c.smithi149.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-03T14:37:00.613 INFO:journalctl@ceph.mon.c.smithi149.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-03T14:37:00.613 INFO:journalctl@ceph.mon.c.smithi149.stdout: 2024-05-03T14:37:00.613 INFO:journalctl@ceph.mon.c.smithi149.stdout: ** Compaction Stats [default] ** 2024-05-03T14:37:00.613 INFO:journalctl@ceph.mon.c.smithi149.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-03T14:37:00.613 INFO:journalctl@ceph.mon.c.smithi149.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-03T14:37:00.613 INFO:journalctl@ceph.mon.c.smithi149.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-03T14:37:00.613 INFO:journalctl@ceph.mon.c.smithi149.stdout: 2024-05-03T14:37:00.613 INFO:journalctl@ceph.mon.c.smithi149.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-03T14:37:00.614 INFO:journalctl@ceph.mon.c.smithi149.stdout: 2024-05-03T14:37:00.614 INFO:journalctl@ceph.mon.c.smithi149.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-03T14:37:00.614 INFO:journalctl@ceph.mon.c.smithi149.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-03T14:37:00.614 INFO:journalctl@ceph.mon.c.smithi149.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-03T14:37:00.614 INFO:journalctl@ceph.mon.c.smithi149.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-03T14:37:00.614 INFO:journalctl@ceph.mon.c.smithi149.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-03T14:37:00.614 INFO:journalctl@ceph.mon.c.smithi149.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-03T14:37:00.614 INFO:journalctl@ceph.mon.c.smithi149.stdout: Cumulative compaction: 0.00 GB write, 0.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-03T14:37:00.614 INFO:journalctl@ceph.mon.c.smithi149.stdout: Interval compaction: 0.00 GB write, 0.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-03T14:37:00.614 INFO:journalctl@ceph.mon.c.smithi149.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-03T14:37:00.614 INFO:journalctl@ceph.mon.c.smithi149.stdout: Block cache BinnedLRUCache@0x563bb6b4b1f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-05-03T14:37:00.614 INFO:journalctl@ceph.mon.c.smithi149.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-03T14:37:00.614 INFO:journalctl@ceph.mon.c.smithi149.stdout: 2024-05-03T14:37:00.615 INFO:journalctl@ceph.mon.c.smithi149.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-03T14:37:00.615 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-05-03T14:37:00.615 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: using public_addr v2:172.21.15.149:0/0 -> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] 2024-05-03T14:37:00.615 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: starting mon.c rank -1 at public addrs [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] at bind addrs [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 285d64b2-095a-11ef-bc97-c7b262605968 2024-05-03T14:37:00.615 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: mon.c@-1(???) e0 preinit fsid 285d64b2-095a-11ef-bc97-c7b262605968 2024-05-03T14:37:00.615 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: mon.c@-1(synchronizing).mds e1 new map 2024-05-03T14:37:00.615 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: mon.c@-1(synchronizing).mds e1 print_map 2024-05-03T14:37:00.615 INFO:journalctl@ceph.mon.c.smithi149.stdout: e1 2024-05-03T14:37:00.615 INFO:journalctl@ceph.mon.c.smithi149.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-03T14:37:00.615 INFO:journalctl@ceph.mon.c.smithi149.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-03T14:37:00.615 INFO:journalctl@ceph.mon.c.smithi149.stdout: legacy client fscid: -1 2024-05-03T14:37:00.615 INFO:journalctl@ceph.mon.c.smithi149.stdout: 2024-05-03T14:37:00.615 INFO:journalctl@ceph.mon.c.smithi149.stdout: No filesystems configured 2024-05-03T14:37:00.615 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-03T14:37:00.616 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-03T14:37:00.616 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-03T14:37:00.616 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-03T14:37:00.616 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-03T14:37:00.616 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-03T14:37:00.616 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-03T14:37:00.616 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-03T14:37:00.616 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-03T14:37:00.616 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-03T14:37:00.616 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:00.616 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi090", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:37:00.616 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: Deploying cephadm binary to smithi090 2024-05-03T14:37:00.616 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:00.617 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:00.617 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:00.617 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: Added host smithi090 2024-05-03T14:37:00.617 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:37:00.617 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:00.617 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:00.617 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:00.617 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:00.617 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:00.617 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T14:37:00.617 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:00.617 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:00.618 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:00.618 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:00.618 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:00.618 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi090", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:37:00.618 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:37:00.618 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:37:00.618 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: Updating smithi090:/etc/ceph/ceph.conf 2024-05-03T14:37:00.618 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: Updating smithi090:/var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T14:37:00.618 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:00.618 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: Updating smithi090:/etc/ceph/ceph.client.admin.keyring 2024-05-03T14:37:00.618 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: Updating smithi090:/var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-03T14:37:00.618 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:00.618 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:00.619 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:00.619 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi149", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:37:00.619 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: Deploying cephadm binary to smithi149 2024-05-03T14:37:00.619 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:00.619 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:00.619 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:00.619 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: Added host smithi149 2024-05-03T14:37:00.619 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:37:00.619 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:00.619 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:00.619 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:00.619 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:00.619 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:00.620 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T14:37:00.620 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:00.620 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:00.620 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:00.620 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:00.620 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:00.620 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:37:00.620 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:37:00.620 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:37:00.620 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: Updating smithi149:/etc/ceph/ceph.conf 2024-05-03T14:37:00.620 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: Updating smithi149:/var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T14:37:00.620 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:00.620 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2024-05-03T14:37:00.621 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: Updating smithi149:/var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-03T14:37:00.621 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/931102818' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-03T14:37:00.621 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:00.621 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:00.621 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:00.621 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/931102818' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-03T14:37:00.621 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: osdmap e4: 0 total, 0 up, 0 in 2024-05-03T14:37:00.621 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:00.621 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:00.621 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi060:172.21.15.60=a;smithi090:172.21.15.90=b;smithi149:172.21.15.149=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:37:00.621 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: Saving service mon spec with placement smithi060:172.21.15.60=a;smithi090:172.21.15.90=b;smithi149:172.21.15.149=c;count:3 2024-05-03T14:37:00.621 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:00.621 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:37:00.621 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:37:00.622 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:37:00.622 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:00.622 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T14:37:00.622 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:37:00.622 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: Deploying daemon mon.c on smithi149 2024-05-03T14:37:00.622 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:00.622 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:00.622 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: from='client.? 172.21.15.149:0/1993036852' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-03T14:37:00.622 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 ceph-mon[33332]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-03T14:37:00.622 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 bash[33313]: 2a9091c8d79b5595dca35e7ec7421628f58e612f25aaa6a5cff7876a0d652c0f 2024-05-03T14:37:00.622 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:00 smithi149 systemd[1]: Started Ceph mon.c for 285d64b2-095a-11ef-bc97-c7b262605968. 2024-05-03T14:37:00.806 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T14:37:05.501 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:05.499+0000 7f379940d700 1 -- 172.21.15.149:0/2212078633 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f377c0035c0 msgr2=0x7f377c005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:37:05.501 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:05.499+0000 7f379940d700 1 --2- 172.21.15.149:0/2212078633 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f377c0035c0 0x7f377c005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3794102f00 tx=0x7f3788013e90 comp rx=0 tx=0).stop 2024-05-03T14:37:05.502 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:05.499+0000 7f379940d700 1 -- 172.21.15.149:0/2212078633 shutdown_connections 2024-05-03T14:37:05.502 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:05.499+0000 7f379940d700 1 --2- 172.21.15.149:0/2212078633 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f377c0035c0 0x7f377c005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:37:05.502 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:05.499+0000 7f379940d700 1 -- 172.21.15.149:0/2212078633 >> 172.21.15.149:0/2212078633 conn(0x7f37940feac0 msgr2=0x7f3794100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:37:05.502 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:05.499+0000 7f379940d700 1 -- 172.21.15.149:0/2212078633 shutdown_connections 2024-05-03T14:37:05.502 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:05.499+0000 7f379940d700 1 -- 172.21.15.149:0/2212078633 wait complete. 2024-05-03T14:37:05.502 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:05 smithi149 ceph-mon[33332]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:05.502 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:05 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T14:37:05.502 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:05 smithi149 ceph-mon[33332]: mon.a calling monitor election 2024-05-03T14:37:05.503 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:05 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T14:37:05.503 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:05 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T14:37:05.503 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:05 smithi149 ceph-mon[33332]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:05.503 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:05 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T14:37:05.503 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:05 smithi149 ceph-mon[33332]: mon.c calling monitor election 2024-05-03T14:37:05.503 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:05 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T14:37:05.503 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:05 smithi149 ceph-mon[33332]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:05.503 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:05 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T14:37:05.503 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:05 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T14:37:05.503 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:05 smithi149 ceph-mon[33332]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-03T14:37:05.503 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:05 smithi149 ceph-mon[33332]: monmap epoch 2 2024-05-03T14:37:05.503 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:05 smithi149 ceph-mon[33332]: fsid 285d64b2-095a-11ef-bc97-c7b262605968 2024-05-03T14:37:05.503 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:05 smithi149 ceph-mon[33332]: last_changed 2024-05-03T14:37:00.338377+0000 2024-05-03T14:37:05.504 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:05 smithi149 ceph-mon[33332]: created 2024-05-03T14:34:40.229121+0000 2024-05-03T14:37:05.504 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:05 smithi149 ceph-mon[33332]: min_mon_release 19 (squid) 2024-05-03T14:37:05.504 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:05 smithi149 ceph-mon[33332]: election_strategy: 1 2024-05-03T14:37:05.504 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:05 smithi149 ceph-mon[33332]: 0: [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] mon.a 2024-05-03T14:37:05.504 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:05 smithi149 ceph-mon[33332]: 1: [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] mon.c 2024-05-03T14:37:05.504 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:05 smithi149 ceph-mon[33332]: fsmap 2024-05-03T14:37:05.504 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:05 smithi149 ceph-mon[33332]: osdmap e4: 0 total, 0 up, 0 in 2024-05-03T14:37:05.504 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:05 smithi149 ceph-mon[33332]: mgrmap e13: x(active, since 61s) 2024-05-03T14:37:05.504 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:05 smithi149 ceph-mon[33332]: overall HEALTH_OK 2024-05-03T14:37:05.504 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:05 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:05.504 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:05 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:05.504 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:05 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:05.504 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:05 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T14:37:05.506 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:05.500+0000 7f379940d700 1 Processor -- start 2024-05-03T14:37:05.506 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:05.500+0000 7f379940d700 1 -- start start 2024-05-03T14:37:05.506 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:05.500+0000 7f379940d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f377c0035c0 0x7f379419e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:37:05.506 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:05.500+0000 7f379940d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f379419ed30 0x7f37941989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:37:05.506 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:05.500+0000 7f379940d700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f379410b1b0 con 0x7f379419ed30 2024-05-03T14:37:05.506 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:05.500+0000 7f379940d700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f379410b030 con 0x7f377c0035c0 2024-05-03T14:37:05.507 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:05.501+0000 7f3792ffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f377c0035c0 0x7f379419e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:37:05.507 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:05.501+0000 7f3792ffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f377c0035c0 0x7f379419e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:51690/0 (socket says 172.21.15.149:51690) 2024-05-03T14:37:05.507 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:05.501+0000 7f3792ffd700 1 -- 172.21.15.149:0/1401577634 learned_addr learned my addr 172.21.15.149:0/1401577634 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-05-03T14:37:05.507 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:05.501+0000 7f3792ffd700 1 -- 172.21.15.149:0/1401577634 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f377c0035c0 msgr2=0x7f379419e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-03T14:37:05.507 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:05.501+0000 7f3792ffd700 1 -- 172.21.15.149:0/1401577634 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f377c0035c0 msgr2=0x7f379419e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-03T14:37:05.507 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:05.501+0000 7f3792ffd700 1 --2- 172.21.15.149:0/1401577634 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f377c0035c0 0x7f379419e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-03T14:37:05.507 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:05.501+0000 7f3792ffd700 1 --2- 172.21.15.149:0/1401577634 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f377c0035c0 0x7f379419e7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-03T14:37:05.507 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:05.501+0000 7f37927fc700 1 --2- 172.21.15.149:0/1401577634 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f379419ed30 0x7f37941989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:37:05.507 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:05.502+0000 7f37927fc700 1 -- 172.21.15.149:0/1401577634 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f377c0035c0 msgr2=0x7f379419e7f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-03T14:37:05.507 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:05.502+0000 7f37927fc700 1 --2- 172.21.15.149:0/1401577634 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f377c0035c0 0x7f379419e7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:37:05.507 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:05.502+0000 7f37927fc700 1 -- 172.21.15.149:0/1401577634 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f379419f370 con 0x7f379419ed30 2024-05-03T14:37:05.507 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:05.502+0000 7f37927fc700 1 --2- 172.21.15.149:0/1401577634 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f379419ed30 0x7f37941989b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f378400b4d0 tx=0x7f378400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:37:05.507 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:05.503+0000 7f377bfff700 1 -- 172.21.15.149:0/1401577634 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f3784011630 con 0x7f379419ed30 2024-05-03T14:37:05.508 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:05.503+0000 7f379940d700 1 -- 172.21.15.149:0/1401577634 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3794198ef0 con 0x7f379419ed30 2024-05-03T14:37:05.508 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:05.503+0000 7f379940d700 1 -- 172.21.15.149:0/1401577634 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3794199480 con 0x7f379419ed30 2024-05-03T14:37:05.508 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:05.503+0000 7f377bfff700 1 -- 172.21.15.149:0/1401577634 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f3784011cd0 con 0x7f379419ed30 2024-05-03T14:37:05.508 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:05.503+0000 7f377bfff700 1 -- 172.21.15.149:0/1401577634 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f37840193b0 con 0x7f379419ed30 2024-05-03T14:37:05.508 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:05.504+0000 7f379940d700 1 -- 172.21.15.149:0/1401577634 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3794104710 con 0x7f379419ed30 2024-05-03T14:37:05.509 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:05.504+0000 7f377bfff700 1 -- 172.21.15.149:0/1401577634 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f3784018020 con 0x7f379419ed30 2024-05-03T14:37:05.509 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:05.505+0000 7f377bfff700 1 --2- 172.21.15.149:0/1401577634 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f378003e2b0 0x7f3780040770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:37:05.509 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:05.505+0000 7f377bfff700 1 -- 172.21.15.149:0/1401577634 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f37840524e0 con 0x7f379419ed30 2024-05-03T14:37:05.509 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:05.505+0000 7f3792ffd700 1 --2- 172.21.15.149:0/1401577634 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f378003e2b0 0x7f3780040770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:37:05.509 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:05.506+0000 7f3792ffd700 1 --2- 172.21.15.149:0/1401577634 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f378003e2b0 0x7f3780040770 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3788013a40 tx=0x7f378800f3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:37:05.522 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:05.520+0000 7f377bfff700 1 -- 172.21.15.149:0/1401577634 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f378401f020 con 0x7f379419ed30 2024-05-03T14:37:05.809 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:05.806+0000 7f379940d700 1 -- 172.21.15.149:0/1401577634 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f3794199ee0 con 0x7f379419ed30 2024-05-03T14:37:05.810 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:05.807+0000 7f377bfff700 1 -- 172.21.15.149:0/1401577634 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f3784022d60 con 0x7f379419ed30 2024-05-03T14:37:05.810 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-03T14:37:05.810 INFO:teuthology.orchestra.run.smithi149.stdout:{"epoch":2,"fsid":"285d64b2-095a-11ef-bc97-c7b262605968","modified":"2024-05-03T14:37:00.338377Z","created":"2024-05-03T14:34:40.229121Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:3300","nonce":0},{"type":"v1","addr":"172.21.15.60:6789","nonce":0}]},"addr":"172.21.15.60:6789/0","public_addr":"172.21.15.60:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:3300","nonce":0},{"type":"v1","addr":"172.21.15.149:6789","nonce":0}]},"addr":"172.21.15.149:6789/0","public_addr":"172.21.15.149:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-03T14:37:05.812 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:05.810+0000 7f379940d700 1 -- 172.21.15.149:0/1401577634 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f378003e2b0 msgr2=0x7f3780040770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:37:05.812 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:05.810+0000 7f379940d700 1 --2- 172.21.15.149:0/1401577634 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f378003e2b0 0x7f3780040770 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3788013a40 tx=0x7f378800f3c0 comp rx=0 tx=0).stop 2024-05-03T14:37:05.812 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:05.810+0000 7f379940d700 1 -- 172.21.15.149:0/1401577634 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f379419ed30 msgr2=0x7f37941989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:37:05.812 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:05.810+0000 7f379940d700 1 --2- 172.21.15.149:0/1401577634 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f379419ed30 0x7f37941989b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f378400b4d0 tx=0x7f378400b890 comp rx=0 tx=0).stop 2024-05-03T14:37:05.812 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:05.810+0000 7f379940d700 1 -- 172.21.15.149:0/1401577634 shutdown_connections 2024-05-03T14:37:05.812 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:05.810+0000 7f379940d700 1 --2- 172.21.15.149:0/1401577634 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f378003e2b0 0x7f3780040770 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:37:05.813 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:05.810+0000 7f379940d700 1 --2- 172.21.15.149:0/1401577634 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f377c0035c0 0x7f379419e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:37:05.813 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:05.810+0000 7f379940d700 1 --2- 172.21.15.149:0/1401577634 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f379419ed30 0x7f37941989b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:37:05.813 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:05.810+0000 7f379940d700 1 -- 172.21.15.149:0/1401577634 >> 172.21.15.149:0/1401577634 conn(0x7f37940feac0 msgr2=0x7f37940ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:37:05.813 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:05.810+0000 7f379940d700 1 -- 172.21.15.149:0/1401577634 shutdown_connections 2024-05-03T14:37:05.813 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:05.810+0000 7f379940d700 1 -- 172.21.15.149:0/1401577634 wait complete. 2024-05-03T14:37:05.813 INFO:teuthology.orchestra.run.smithi149.stderr:dumped monmap epoch 2 2024-05-03T14:37:05.839 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:05 smithi060 ceph-mon[28873]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:05.839 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:05 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T14:37:05.839 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:05 smithi060 ceph-mon[28873]: mon.a calling monitor election 2024-05-03T14:37:05.839 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:05 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T14:37:05.839 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:05 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T14:37:05.839 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:05 smithi060 ceph-mon[28873]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:05.840 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:05 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T14:37:05.840 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:05 smithi060 ceph-mon[28873]: mon.c calling monitor election 2024-05-03T14:37:05.840 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:05 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T14:37:05.840 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:05 smithi060 ceph-mon[28873]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:05.840 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:05 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T14:37:05.840 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:05 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T14:37:05.840 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:05 smithi060 ceph-mon[28873]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-03T14:37:05.840 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:05 smithi060 ceph-mon[28873]: monmap epoch 2 2024-05-03T14:37:05.840 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:05 smithi060 ceph-mon[28873]: fsid 285d64b2-095a-11ef-bc97-c7b262605968 2024-05-03T14:37:05.840 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:05 smithi060 ceph-mon[28873]: last_changed 2024-05-03T14:37:00.338377+0000 2024-05-03T14:37:05.840 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:05 smithi060 ceph-mon[28873]: created 2024-05-03T14:34:40.229121+0000 2024-05-03T14:37:05.840 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:05 smithi060 ceph-mon[28873]: min_mon_release 19 (squid) 2024-05-03T14:37:05.840 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:05 smithi060 ceph-mon[28873]: election_strategy: 1 2024-05-03T14:37:05.840 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:05 smithi060 ceph-mon[28873]: 0: [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] mon.a 2024-05-03T14:37:05.841 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:05 smithi060 ceph-mon[28873]: 1: [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] mon.c 2024-05-03T14:37:05.841 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:05 smithi060 ceph-mon[28873]: fsmap 2024-05-03T14:37:05.841 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:05 smithi060 ceph-mon[28873]: osdmap e4: 0 total, 0 up, 0 in 2024-05-03T14:37:05.841 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:05 smithi060 ceph-mon[28873]: mgrmap e13: x(active, since 61s) 2024-05-03T14:37:05.841 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:05 smithi060 ceph-mon[28873]: overall HEALTH_OK 2024-05-03T14:37:05.841 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:05 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:05.841 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:05 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:05.841 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:05 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:05.841 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:05 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T14:37:06.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:06 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:37:06.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:06 smithi149 ceph-mon[33332]: Deploying daemon mon.b on smithi090 2024-05-03T14:37:06.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:06 smithi149 ceph-mon[33332]: from='client.? 172.21.15.149:0/1401577634' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-03T14:37:06.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:06 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T14:37:06.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:06 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:37:06.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:06 smithi060 ceph-mon[28873]: Deploying daemon mon.b on smithi090 2024-05-03T14:37:06.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:06 smithi060 ceph-mon[28873]: from='client.? 172.21.15.149:0/1401577634' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-03T14:37:06.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:06 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T14:37:07.430 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-03T14:37:07.430 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-03T14:37:07.562 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:37:07 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:37:07.338+0000 7fd485000700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-05-03T14:37:07.562 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:07 smithi060 ceph-mon[28873]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:07.676 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T14:37:07.806 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:07 smithi149 ceph-mon[33332]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:09.306 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:09 smithi090 systemd[1]: Started Ceph mon.b for 285d64b2-095a-11ef-bc97-c7b262605968. 2024-05-03T14:37:14.589 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:14 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T14:37:14.589 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:14 smithi060 ceph-mon[28873]: mon.a calling monitor election 2024-05-03T14:37:14.589 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:14 smithi060 ceph-mon[28873]: mon.c calling monitor election 2024-05-03T14:37:14.589 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:14 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T14:37:14.589 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:14 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T14:37:14.589 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:14 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T14:37:14.589 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:14 smithi060 ceph-mon[28873]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:14.589 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:14 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T14:37:14.589 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:14 smithi060 ceph-mon[28873]: mon.b calling monitor election 2024-05-03T14:37:14.590 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:14 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T14:37:14.590 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:14 smithi060 ceph-mon[28873]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:14.590 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:14 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T14:37:14.590 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:14 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T14:37:14.590 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:14 smithi060 ceph-mon[28873]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-03T14:37:14.590 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:14 smithi060 ceph-mon[28873]: monmap epoch 3 2024-05-03T14:37:14.590 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:14 smithi060 ceph-mon[28873]: fsid 285d64b2-095a-11ef-bc97-c7b262605968 2024-05-03T14:37:14.590 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:14 smithi060 ceph-mon[28873]: last_changed 2024-05-03T14:37:09.170836+0000 2024-05-03T14:37:14.590 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:14 smithi060 ceph-mon[28873]: created 2024-05-03T14:34:40.229121+0000 2024-05-03T14:37:14.590 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:14 smithi060 ceph-mon[28873]: min_mon_release 19 (squid) 2024-05-03T14:37:14.590 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:14 smithi060 ceph-mon[28873]: election_strategy: 1 2024-05-03T14:37:14.590 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:14 smithi060 ceph-mon[28873]: 0: [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] mon.a 2024-05-03T14:37:14.590 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:14 smithi060 ceph-mon[28873]: 1: [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] mon.c 2024-05-03T14:37:14.591 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:14 smithi060 ceph-mon[28873]: 2: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.b 2024-05-03T14:37:14.591 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:14 smithi060 ceph-mon[28873]: fsmap 2024-05-03T14:37:14.591 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:14 smithi060 ceph-mon[28873]: osdmap e4: 0 total, 0 up, 0 in 2024-05-03T14:37:14.591 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:14 smithi060 ceph-mon[28873]: mgrmap e13: x(active, since 70s) 2024-05-03T14:37:14.591 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:14 smithi060 ceph-mon[28873]: overall HEALTH_OK 2024-05-03T14:37:14.591 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:14 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:14.591 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:14 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:14.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:14 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T14:37:14.626 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:14 smithi090 ceph-mon[33568]: mon.a calling monitor election 2024-05-03T14:37:14.626 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:14 smithi090 ceph-mon[33568]: mon.c calling monitor election 2024-05-03T14:37:14.626 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:14 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T14:37:14.626 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:14 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T14:37:14.626 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:14 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T14:37:14.626 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:14 smithi090 ceph-mon[33568]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:14.626 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:14 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T14:37:14.626 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:14 smithi090 ceph-mon[33568]: mon.b calling monitor election 2024-05-03T14:37:14.626 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:14 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T14:37:14.626 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:14 smithi090 ceph-mon[33568]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:14.626 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:14 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T14:37:14.626 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:14 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T14:37:14.626 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:14 smithi090 ceph-mon[33568]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-03T14:37:14.627 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:14 smithi090 ceph-mon[33568]: monmap epoch 3 2024-05-03T14:37:14.627 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:14 smithi090 ceph-mon[33568]: fsid 285d64b2-095a-11ef-bc97-c7b262605968 2024-05-03T14:37:14.627 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:14 smithi090 ceph-mon[33568]: last_changed 2024-05-03T14:37:09.170836+0000 2024-05-03T14:37:14.627 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:14 smithi090 ceph-mon[33568]: created 2024-05-03T14:34:40.229121+0000 2024-05-03T14:37:14.627 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:14 smithi090 ceph-mon[33568]: min_mon_release 19 (squid) 2024-05-03T14:37:14.627 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:14 smithi090 ceph-mon[33568]: election_strategy: 1 2024-05-03T14:37:14.627 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:14 smithi090 ceph-mon[33568]: 0: [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] mon.a 2024-05-03T14:37:14.627 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:14 smithi090 ceph-mon[33568]: 1: [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] mon.c 2024-05-03T14:37:14.627 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:14 smithi090 ceph-mon[33568]: 2: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.b 2024-05-03T14:37:14.627 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:14 smithi090 ceph-mon[33568]: fsmap 2024-05-03T14:37:14.627 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:14 smithi090 ceph-mon[33568]: osdmap e4: 0 total, 0 up, 0 in 2024-05-03T14:37:14.627 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:14 smithi090 ceph-mon[33568]: mgrmap e13: x(active, since 70s) 2024-05-03T14:37:14.627 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:14 smithi090 ceph-mon[33568]: overall HEALTH_OK 2024-05-03T14:37:14.627 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:14 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:14.627 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:14 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:14.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:14 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T14:37:14.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:14 smithi149 ceph-mon[33332]: mon.a calling monitor election 2024-05-03T14:37:14.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:14 smithi149 ceph-mon[33332]: mon.c calling monitor election 2024-05-03T14:37:14.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:14 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T14:37:14.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:14 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T14:37:14.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:14 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T14:37:14.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:14 smithi149 ceph-mon[33332]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:14.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:14 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T14:37:14.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:14 smithi149 ceph-mon[33332]: mon.b calling monitor election 2024-05-03T14:37:14.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:14 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T14:37:14.675 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:14 smithi149 ceph-mon[33332]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:14.675 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:14 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T14:37:14.675 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:14 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T14:37:14.675 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:14 smithi149 ceph-mon[33332]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-03T14:37:14.675 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:14 smithi149 ceph-mon[33332]: monmap epoch 3 2024-05-03T14:37:14.675 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:14 smithi149 ceph-mon[33332]: fsid 285d64b2-095a-11ef-bc97-c7b262605968 2024-05-03T14:37:14.675 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:14 smithi149 ceph-mon[33332]: last_changed 2024-05-03T14:37:09.170836+0000 2024-05-03T14:37:14.675 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:14 smithi149 ceph-mon[33332]: created 2024-05-03T14:34:40.229121+0000 2024-05-03T14:37:14.675 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:14 smithi149 ceph-mon[33332]: min_mon_release 19 (squid) 2024-05-03T14:37:14.675 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:14 smithi149 ceph-mon[33332]: election_strategy: 1 2024-05-03T14:37:14.675 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:14 smithi149 ceph-mon[33332]: 0: [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] mon.a 2024-05-03T14:37:14.675 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:14 smithi149 ceph-mon[33332]: 1: [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] mon.c 2024-05-03T14:37:14.675 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:14 smithi149 ceph-mon[33332]: 2: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.b 2024-05-03T14:37:14.676 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:14 smithi149 ceph-mon[33332]: fsmap 2024-05-03T14:37:14.676 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:14 smithi149 ceph-mon[33332]: osdmap e4: 0 total, 0 up, 0 in 2024-05-03T14:37:14.676 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:14 smithi149 ceph-mon[33332]: mgrmap e13: x(active, since 70s) 2024-05-03T14:37:14.676 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:14 smithi149 ceph-mon[33332]: overall HEALTH_OK 2024-05-03T14:37:14.676 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:14 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:14.676 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:14 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:15.205 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:15.201+0000 7f7b13f04700 1 -- 172.21.15.149:0/401841253 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7af80057d0 msgr2=0x7f7af8007c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:37:15.205 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:15.201+0000 7f7b13f04700 1 --2- 172.21.15.149:0/401841253 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7af80057d0 0x7f7af8007c60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c101de0 tx=0x7f7b0800ff50 comp rx=0 tx=0).stop 2024-05-03T14:37:15.205 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:15.201+0000 7f7b13f04700 1 -- 172.21.15.149:0/401841253 shutdown_connections 2024-05-03T14:37:15.205 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:15.201+0000 7f7b13f04700 1 --2- 172.21.15.149:0/401841253 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7af80057d0 0x7f7af8007c60 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:37:15.205 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:15.201+0000 7f7b13f04700 1 -- 172.21.15.149:0/401841253 >> 172.21.15.149:0/401841253 conn(0x7f7b0c0fd9a0 msgr2=0x7f7b0c0ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:37:15.205 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:15.201+0000 7f7b13f04700 1 -- 172.21.15.149:0/401841253 shutdown_connections 2024-05-03T14:37:15.205 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:15.201+0000 7f7b13f04700 1 -- 172.21.15.149:0/401841253 wait complete. 2024-05-03T14:37:15.205 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:15.202+0000 7f7b13f04700 1 Processor -- start 2024-05-03T14:37:15.205 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:15.202+0000 7f7b13f04700 1 -- start start 2024-05-03T14:37:15.210 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:15.202+0000 7f7b13f04700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7af80057d0 0x7f7b0c19af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:37:15.211 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:15.202+0000 7f7b13f04700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b0c19b4c0 0x7f7b0c19f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:37:15.211 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:15.202+0000 7f7b13f04700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b0c19fd90 0x7f7b0c1a0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:37:15.211 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:15.203+0000 7f7b13f04700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f7b0c108f10 con 0x7f7af80057d0 2024-05-03T14:37:15.211 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:15.203+0000 7f7b13f04700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7b0c108d90 con 0x7f7b0c19fd90 2024-05-03T14:37:15.211 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:15.203+0000 7f7b13f04700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7b0c109090 con 0x7f7b0c19b4c0 2024-05-03T14:37:15.211 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:15.203+0000 7f7b1149f700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b0c19b4c0 0x7f7b0c19f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:37:15.211 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:15.203+0000 7f7b1149f700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b0c19b4c0 0x7f7b0c19f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:39506/0 (socket says 172.21.15.149:39506) 2024-05-03T14:37:15.211 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:15.203+0000 7f7b1149f700 1 -- 172.21.15.149:0/4103173242 learned_addr learned my addr 172.21.15.149:0/4103173242 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-05-03T14:37:15.211 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:15.203+0000 7f7b124a1700 1 --2- 172.21.15.149:0/4103173242 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b0c19fd90 0x7f7b0c1a0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:37:15.211 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:15.203+0000 7f7b11ca0700 1 --2- 172.21.15.149:0/4103173242 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7af80057d0 0x7f7b0c19af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:37:15.211 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:15.203+0000 7f7b1149f700 1 -- 172.21.15.149:0/4103173242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b0c19b4c0 msgr2=0x7f7b0c19f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-03T14:37:15.212 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:15.203+0000 7f7b1149f700 1 -- 172.21.15.149:0/4103173242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b0c19b4c0 msgr2=0x7f7b0c19f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-03T14:37:15.212 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:15.203+0000 7f7b1149f700 1 --2- 172.21.15.149:0/4103173242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b0c19b4c0 0x7f7b0c19f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-03T14:37:15.212 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:15.203+0000 7f7b124a1700 1 -- 172.21.15.149:0/4103173242 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b0c19fd90 msgr2=0x7f7b0c1a0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-05-03T14:37:15.212 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:15.203+0000 7f7b124a1700 1 -- 172.21.15.149:0/4103173242 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b0c19fd90 msgr2=0x7f7b0c1a0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-03T14:37:15.212 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:15.203+0000 7f7b124a1700 1 --2- 172.21.15.149:0/4103173242 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b0c19fd90 0x7f7b0c1a0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-03T14:37:15.212 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:15.203+0000 7f7b1149f700 1 --2- 172.21.15.149:0/4103173242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b0c19b4c0 0x7f7b0c19f850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-03T14:37:15.212 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:15.203+0000 7f7b124a1700 1 --2- 172.21.15.149:0/4103173242 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b0c19fd90 0x7f7b0c1a0240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-03T14:37:15.212 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:15.203+0000 7f7b11ca0700 1 -- 172.21.15.149:0/4103173242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b0c19b4c0 msgr2=0x7f7b0c19f850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-03T14:37:15.212 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:15.203+0000 7f7b11ca0700 1 --2- 172.21.15.149:0/4103173242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b0c19b4c0 0x7f7b0c19f850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:37:15.212 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:15.203+0000 7f7b11ca0700 1 -- 172.21.15.149:0/4103173242 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b0c19fd90 msgr2=0x7f7b0c1a0240 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-03T14:37:15.212 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:15.203+0000 7f7b11ca0700 1 --2- 172.21.15.149:0/4103173242 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b0c19fd90 0x7f7b0c1a0240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:37:15.212 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:15.203+0000 7f7b11ca0700 1 -- 172.21.15.149:0/4103173242 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b0c1a7b00 con 0x7f7af80057d0 2024-05-03T14:37:15.212 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:15.204+0000 7f7b11ca0700 1 --2- 172.21.15.149:0/4103173242 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7af80057d0 0x7f7b0c19af80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7b08020000 tx=0x7f7b08004440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:37:15.213 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:15.204+0000 7f7b02ffd700 1 -- 172.21.15.149:0/4103173242 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0802a070 con 0x7f7af80057d0 2024-05-03T14:37:15.213 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:15.204+0000 7f7b13f04700 1 -- 172.21.15.149:0/4103173242 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b0c1a7cd0 con 0x7f7af80057d0 2024-05-03T14:37:15.214 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:15.204+0000 7f7b02ffd700 1 -- 172.21.15.149:0/4103173242 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f7b08003e20 con 0x7f7af80057d0 2024-05-03T14:37:15.216 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:15.204+0000 7f7b02ffd700 1 -- 172.21.15.149:0/4103173242 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b080267e0 con 0x7f7af80057d0 2024-05-03T14:37:15.216 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:15.204+0000 7f7b13f04700 1 -- 172.21.15.149:0/4103173242 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b0c1a81b0 con 0x7f7af80057d0 2024-05-03T14:37:15.216 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:15.205+0000 7f7b13f04700 1 -- 172.21.15.149:0/4103173242 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b0c064b00 con 0x7f7af80057d0 2024-05-03T14:37:15.217 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:15.206+0000 7f7b02ffd700 1 -- 172.21.15.149:0/4103173242 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f7b0802d070 con 0x7f7af80057d0 2024-05-03T14:37:15.217 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:15.206+0000 7f7b02ffd700 1 --2- 172.21.15.149:0/4103173242 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f7af003e9a0 0x7f7af0040e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:37:15.217 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:15.206+0000 7f7b02ffd700 1 -- 172.21.15.149:0/4103173242 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f7b080559e0 con 0x7f7af80057d0 2024-05-03T14:37:15.217 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:15.206+0000 7f7b1149f700 1 --2- 172.21.15.149:0/4103173242 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f7af003e9a0 0x7f7af0040e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:37:15.218 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:15.207+0000 7f7b1149f700 1 --2- 172.21.15.149:0/4103173242 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f7af003e9a0 0x7f7af0040e60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7afc009180 tx=0x7f7afc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:37:15.224 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:15.220+0000 7f7b02ffd700 1 -- 172.21.15.149:0/4103173242 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f7b080044b0 con 0x7f7af80057d0 2024-05-03T14:37:15.516 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:15 smithi149 ceph-mon[33332]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:15.516 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:15 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:15.516 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:15 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:15.516 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:15 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:37:15.516 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:15 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T14:37:15.521 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:15.519+0000 7f7b13f04700 1 -- 172.21.15.149:0/4103173242 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f7b0c02e510 con 0x7f7af80057d0 2024-05-03T14:37:15.523 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:15.520+0000 7f7b02ffd700 1 -- 172.21.15.149:0/4103173242 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1281 (secure 0 0 0) 0x7f7b0801dab0 con 0x7f7af80057d0 2024-05-03T14:37:15.523 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-03T14:37:15.523 INFO:teuthology.orchestra.run.smithi149.stdout:{"epoch":3,"fsid":"285d64b2-095a-11ef-bc97-c7b262605968","modified":"2024-05-03T14:37:09.170836Z","created":"2024-05-03T14:34:40.229121Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:3300","nonce":0},{"type":"v1","addr":"172.21.15.60:6789","nonce":0}]},"addr":"172.21.15.60:6789/0","public_addr":"172.21.15.60:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:3300","nonce":0},{"type":"v1","addr":"172.21.15.149:6789","nonce":0}]},"addr":"172.21.15.149:6789/0","public_addr":"172.21.15.149:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:3300","nonce":0},{"type":"v1","addr":"172.21.15.90:6789","nonce":0}]},"addr":"172.21.15.90:6789/0","public_addr":"172.21.15.90:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-05-03T14:37:15.525 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:15.523+0000 7f7b13f04700 1 -- 172.21.15.149:0/4103173242 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f7af003e9a0 msgr2=0x7f7af0040e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:37:15.525 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:15.523+0000 7f7b13f04700 1 --2- 172.21.15.149:0/4103173242 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f7af003e9a0 0x7f7af0040e60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7afc009180 tx=0x7f7afc008040 comp rx=0 tx=0).stop 2024-05-03T14:37:15.525 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:15.523+0000 7f7b13f04700 1 -- 172.21.15.149:0/4103173242 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7af80057d0 msgr2=0x7f7b0c19af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:37:15.526 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:15.523+0000 7f7b13f04700 1 --2- 172.21.15.149:0/4103173242 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7af80057d0 0x7f7b0c19af80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7b08020000 tx=0x7f7b08004440 comp rx=0 tx=0).stop 2024-05-03T14:37:15.526 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:15.523+0000 7f7b13f04700 1 -- 172.21.15.149:0/4103173242 shutdown_connections 2024-05-03T14:37:15.526 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:15.523+0000 7f7b13f04700 1 --2- 172.21.15.149:0/4103173242 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f7af003e9a0 0x7f7af0040e60 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:37:15.526 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:15.523+0000 7f7b13f04700 1 --2- 172.21.15.149:0/4103173242 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b0c19fd90 0x7f7b0c1a0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:37:15.526 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:15.523+0000 7f7b13f04700 1 --2- 172.21.15.149:0/4103173242 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7af80057d0 0x7f7b0c19af80 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:37:15.526 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:15.523+0000 7f7b13f04700 1 --2- 172.21.15.149:0/4103173242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b0c19b4c0 0x7f7b0c19f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:37:15.526 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:15.523+0000 7f7b13f04700 1 -- 172.21.15.149:0/4103173242 >> 172.21.15.149:0/4103173242 conn(0x7f7b0c0fd9a0 msgr2=0x7f7b0c0feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:37:15.526 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:15.523+0000 7f7b13f04700 1 -- 172.21.15.149:0/4103173242 shutdown_connections 2024-05-03T14:37:15.526 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:15.523+0000 7f7b13f04700 1 -- 172.21.15.149:0/4103173242 wait complete. 2024-05-03T14:37:15.526 INFO:teuthology.orchestra.run.smithi149.stderr:dumped monmap epoch 3 2024-05-03T14:37:15.554 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:15 smithi090 ceph-mon[33568]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:15.554 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:15 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:15.554 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:15 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:15.554 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:15 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:37:15.554 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:15 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T14:37:15.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:15 smithi060 ceph-mon[28873]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:15.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:15 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:15.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:15 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:15.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:15 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:37:15.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:15 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T14:37:16.525 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:16 smithi149 ceph-mon[33332]: from='client.? 172.21.15.149:0/4103173242' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-03T14:37:16.525 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:16 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:16.525 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:16 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:16.525 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:16 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:16.525 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:16 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:16.525 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:16 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:37:16.525 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:16 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:37:16.550 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-03T14:37:16.550 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph config generate-minimal-conf 2024-05-03T14:37:16.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:16 smithi060 ceph-mon[28873]: from='client.? 172.21.15.149:0/4103173242' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-03T14:37:16.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:16 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:16.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:16 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:16.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:16 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:16.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:16 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:16.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:16 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:37:16.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:16 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:37:16.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:16 smithi090 ceph-mon[33568]: from='client.? 172.21.15.149:0/4103173242' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-03T14:37:16.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:16 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:16.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:16 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:16.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:16 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:16.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:16 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:16.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:16 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:37:16.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:16 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:37:16.977 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:37:17.821 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:17 smithi149 ceph-mon[33332]: Updating smithi060:/etc/ceph/ceph.conf 2024-05-03T14:37:17.821 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:17 smithi149 ceph-mon[33332]: Updating smithi090:/etc/ceph/ceph.conf 2024-05-03T14:37:17.821 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:17 smithi149 ceph-mon[33332]: Updating smithi149:/etc/ceph/ceph.conf 2024-05-03T14:37:17.821 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:17 smithi149 ceph-mon[33332]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:17.822 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:17 smithi149 ceph-mon[33332]: Updating smithi090:/var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T14:37:17.822 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:17 smithi149 ceph-mon[33332]: Updating smithi149:/var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T14:37:17.822 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:17 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:17.822 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:17 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:17.822 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:17 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:17.822 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:17 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:17.822 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:17 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:17.822 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:17 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:17.822 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:17 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:17.833 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:17 smithi060 ceph-mon[28873]: Updating smithi060:/etc/ceph/ceph.conf 2024-05-03T14:37:17.833 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:17 smithi060 ceph-mon[28873]: Updating smithi090:/etc/ceph/ceph.conf 2024-05-03T14:37:17.833 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:17 smithi060 ceph-mon[28873]: Updating smithi149:/etc/ceph/ceph.conf 2024-05-03T14:37:17.834 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:17 smithi060 ceph-mon[28873]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:17.834 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:17 smithi060 ceph-mon[28873]: Updating smithi090:/var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T14:37:17.834 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:17 smithi060 ceph-mon[28873]: Updating smithi149:/var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T14:37:17.834 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:17 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:17.834 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:17 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:17.834 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:17 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:17.834 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:17 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:17.834 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:17 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:17.834 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:17 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:17.834 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:17 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:17.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:17 smithi090 ceph-mon[33568]: Updating smithi060:/etc/ceph/ceph.conf 2024-05-03T14:37:17.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:17 smithi090 ceph-mon[33568]: Updating smithi090:/etc/ceph/ceph.conf 2024-05-03T14:37:17.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:17 smithi090 ceph-mon[33568]: Updating smithi149:/etc/ceph/ceph.conf 2024-05-03T14:37:17.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:17 smithi090 ceph-mon[33568]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:17.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:17 smithi090 ceph-mon[33568]: Updating smithi090:/var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T14:37:17.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:17 smithi090 ceph-mon[33568]: Updating smithi149:/var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T14:37:17.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:17 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:17.876 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:17 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:17.876 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:17 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:17.876 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:17 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:17.876 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:17 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:17.876 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:17 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:17.876 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:17 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:18.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:18 smithi060 ceph-mon[28873]: Updating smithi060:/var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T14:37:18.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:18 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:18.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:18 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:18.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:18 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:18.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:18 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:18.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:18 smithi060 ceph-mon[28873]: Reconfiguring mon.a (unknown last config time)... 2024-05-03T14:37:18.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:18 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T14:37:18.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:18 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-03T14:37:18.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:18 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:37:18.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:18 smithi060 ceph-mon[28873]: Reconfiguring daemon mon.a on smithi060 2024-05-03T14:37:18.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:18 smithi090 ceph-mon[33568]: Updating smithi060:/var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T14:37:18.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:18 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:18.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:18 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:18.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:18 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:18.876 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:18 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:18.876 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:18 smithi090 ceph-mon[33568]: Reconfiguring mon.a (unknown last config time)... 2024-05-03T14:37:18.876 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:18 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T14:37:18.876 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:18 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-03T14:37:18.876 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:18 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:37:18.876 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:18 smithi090 ceph-mon[33568]: Reconfiguring daemon mon.a on smithi060 2024-05-03T14:37:18.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:18 smithi149 ceph-mon[33332]: Updating smithi060:/var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T14:37:18.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:18 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:18.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:18 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:18.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:18 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:18.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:18 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:18.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:18 smithi149 ceph-mon[33332]: Reconfiguring mon.a (unknown last config time)... 2024-05-03T14:37:18.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:18 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T14:37:18.925 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:18 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-03T14:37:18.925 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:18 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:37:18.925 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:18 smithi149 ceph-mon[33332]: Reconfiguring daemon mon.a on smithi060 2024-05-03T14:37:19.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:19.650+0000 7f0311200700 1 -- 172.21.15.60:0/1753861150 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f030c075840 msgr2=0x7f030c075c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:37:19.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:19.650+0000 7f0311200700 1 --2- 172.21.15.60:0/1753861150 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f030c075840 0x7f030c075c20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f02fc009910 tx=0x7f02fc009c20 comp rx=0 tx=0).stop 2024-05-03T14:37:19.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:19.650+0000 7f0311200700 1 -- 172.21.15.60:0/1753861150 shutdown_connections 2024-05-03T14:37:19.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:19.650+0000 7f0311200700 1 --2- 172.21.15.60:0/1753861150 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f030c075840 0x7f030c075c20 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:37:19.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:19.650+0000 7f0311200700 1 -- 172.21.15.60:0/1753861150 >> 172.21.15.60:0/1753861150 conn(0x7f030c0fe9e0 msgr2=0x7f030c100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:37:19.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:19.650+0000 7f0311200700 1 -- 172.21.15.60:0/1753861150 shutdown_connections 2024-05-03T14:37:19.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:19.650+0000 7f0311200700 1 -- 172.21.15.60:0/1753861150 wait complete. 2024-05-03T14:37:19.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:19.651+0000 7f0311200700 1 Processor -- start 2024-05-03T14:37:19.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:19.652+0000 7f0311200700 1 -- start start 2024-05-03T14:37:19.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:19.652+0000 7f0311200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f030c075840 0x7f030c19f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:37:19.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:19.652+0000 7f0311200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f030c19f890 0x7f030c1a3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:37:19.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:19.652+0000 7f0311200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f030c1a4160 0x7f030c1a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:37:19.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:19.652+0000 7f0311200700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f030c113c90 con 0x7f030c075840 2024-05-03T14:37:19.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:19.652+0000 7f0311200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f030c113b10 con 0x7f030c1a4160 2024-05-03T14:37:19.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:19.652+0000 7f0311200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f030c113e10 con 0x7f030c19f890 2024-05-03T14:37:19.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:19.652+0000 7f0309e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f030c075840 0x7f030c19f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:37:19.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:19.652+0000 7f0309e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f030c075840 0x7f030c19f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44070/0 (socket says 172.21.15.60:44070) 2024-05-03T14:37:19.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:19.652+0000 7f0309e00700 1 -- 172.21.15.60:0/1139569931 learned_addr learned my addr 172.21.15.60:0/1139569931 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:37:19.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:19.653+0000 7f030a800700 1 --2- 172.21.15.60:0/1139569931 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f030c1a4160 0x7f030c1a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:37:19.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:19.653+0000 7f0309400700 1 --2- 172.21.15.60:0/1139569931 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f030c19f890 0x7f030c1a3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:37:19.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:19.653+0000 7f0309e00700 1 -- 172.21.15.60:0/1139569931 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f030c19f890 msgr2=0x7f030c1a3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:37:19.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:19.653+0000 7f0309e00700 1 --2- 172.21.15.60:0/1139569931 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f030c19f890 0x7f030c1a3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:37:19.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:19.653+0000 7f0309e00700 1 -- 172.21.15.60:0/1139569931 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f030c1a4160 msgr2=0x7f030c1a4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:37:19.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:19.653+0000 7f0309e00700 1 --2- 172.21.15.60:0/1139569931 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f030c1a4160 0x7f030c1a4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:37:19.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:19.653+0000 7f0309e00700 1 -- 172.21.15.60:0/1139569931 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f030c1abed0 con 0x7f030c075840 2024-05-03T14:37:19.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:19.654+0000 7f0309e00700 1 --2- 172.21.15.60:0/1139569931 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f030c075840 0x7f030c19f350 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f02fc005320 tx=0x7f02fc0048c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:37:19.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:19.654+0000 7f0302a00700 1 -- 172.21.15.60:0/1139569931 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02fc027070 con 0x7f030c075840 2024-05-03T14:37:19.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:19.654+0000 7f0302a00700 1 -- 172.21.15.60:0/1139569931 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f02fc00bc50 con 0x7f030c075840 2024-05-03T14:37:19.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:19.654+0000 7f0311200700 1 -- 172.21.15.60:0/1139569931 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f030c1ac0a0 con 0x7f030c075840 2024-05-03T14:37:19.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:19.654+0000 7f0302a00700 1 -- 172.21.15.60:0/1139569931 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02fc022de0 con 0x7f030c075840 2024-05-03T14:37:19.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:19.654+0000 7f0311200700 1 -- 172.21.15.60:0/1139569931 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f030c1ac5d0 con 0x7f030c075840 2024-05-03T14:37:19.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:19.655+0000 7f0311200700 1 -- 172.21.15.60:0/1139569931 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f030c064b00 con 0x7f030c075840 2024-05-03T14:37:19.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:19.656+0000 7f0302a00700 1 -- 172.21.15.60:0/1139569931 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f02fc02a070 con 0x7f030c075840 2024-05-03T14:37:19.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:19.656+0000 7f0302a00700 1 --2- 172.21.15.60:0/1139569931 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f02f003e9a0 0x7f02f0040e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:37:19.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:19.656+0000 7f0302a00700 1 -- 172.21.15.60:0/1139569931 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f02fc0536c0 con 0x7f030c075840 2024-05-03T14:37:19.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:19.657+0000 7f0309400700 1 --2- 172.21.15.60:0/1139569931 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f02f003e9a0 0x7f02f0040e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:37:19.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:19.657+0000 7f0309400700 1 --2- 172.21.15.60:0/1139569931 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f02f003e9a0 0x7f02f0040e60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f02f8009ad0 tx=0x7f02f8009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:37:19.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:19.662+0000 7f0302a00700 1 -- 172.21.15.60:0/1139569931 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f02fc013070 con 0x7f030c075840 2024-05-03T14:37:19.758 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:19 smithi060 ceph-mon[28873]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:19.759 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:19 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:19.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:19.906+0000 7f0311200700 1 -- 172.21.15.60:0/1139569931 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f030c1ac8f0 con 0x7f030c075840 2024-05-03T14:37:19.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:19.907+0000 7f0302a00700 1 -- 172.21.15.60:0/1139569931 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7f02fc031090 con 0x7f030c075840 2024-05-03T14:37:19.911 INFO:teuthology.orchestra.run.smithi060.stdout:# minimal ceph.conf for 285d64b2-095a-11ef-bc97-c7b262605968 2024-05-03T14:37:19.911 INFO:teuthology.orchestra.run.smithi060.stdout:[global] 2024-05-03T14:37:19.911 INFO:teuthology.orchestra.run.smithi060.stdout: fsid = 285d64b2-095a-11ef-bc97-c7b262605968 2024-05-03T14:37:19.911 INFO:teuthology.orchestra.run.smithi060.stdout: mon_host = [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] 2024-05-03T14:37:19.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:19.910+0000 7f0311200700 1 -- 172.21.15.60:0/1139569931 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f02f003e9a0 msgr2=0x7f02f0040e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:37:19.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:19.910+0000 7f0311200700 1 --2- 172.21.15.60:0/1139569931 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f02f003e9a0 0x7f02f0040e60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f02f8009ad0 tx=0x7f02f8009210 comp rx=0 tx=0).stop 2024-05-03T14:37:19.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:19.910+0000 7f0311200700 1 -- 172.21.15.60:0/1139569931 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f030c075840 msgr2=0x7f030c19f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:37:19.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:19.910+0000 7f0311200700 1 --2- 172.21.15.60:0/1139569931 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f030c075840 0x7f030c19f350 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f02fc005320 tx=0x7f02fc0048c0 comp rx=0 tx=0).stop 2024-05-03T14:37:19.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:19.911+0000 7f0311200700 1 -- 172.21.15.60:0/1139569931 shutdown_connections 2024-05-03T14:37:19.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:19.911+0000 7f0311200700 1 --2- 172.21.15.60:0/1139569931 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f02f003e9a0 0x7f02f0040e60 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:37:19.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:19.911+0000 7f0311200700 1 --2- 172.21.15.60:0/1139569931 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f030c1a4160 0x7f030c1a4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:37:19.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:19.911+0000 7f0311200700 1 --2- 172.21.15.60:0/1139569931 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f030c075840 0x7f030c19f350 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:37:19.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:19.911+0000 7f0311200700 1 --2- 172.21.15.60:0/1139569931 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f030c19f890 0x7f030c1a3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:37:19.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:19.911+0000 7f0311200700 1 -- 172.21.15.60:0/1139569931 >> 172.21.15.60:0/1139569931 conn(0x7f030c0fe9e0 msgr2=0x7f030c100430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:37:19.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:19.911+0000 7f0311200700 1 -- 172.21.15.60:0/1139569931 shutdown_connections 2024-05-03T14:37:19.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:19.911+0000 7f0311200700 1 -- 172.21.15.60:0/1139569931 wait complete. 2024-05-03T14:37:19.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:19 smithi149 ceph-mon[33332]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:19.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:19 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:20.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:19 smithi090 ceph-mon[33568]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:20.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:19 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:20.768 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:20 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/1139569931' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:37:20.768 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:20 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:20.768 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:20 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:20.768 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:20 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T14:37:20.768 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:20 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-03T14:37:20.768 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:20 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:37:20.831 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-03T14:37:20.832 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-05-03T14:37:20.832 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/ceph/ceph.conf 2024-05-03T14:37:20.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:20 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/1139569931' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:37:20.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:20 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:20.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:20 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:20.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:20 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T14:37:20.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:20 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-03T14:37:20.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:20 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:37:20.965 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-05-03T14:37:20.965 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-03T14:37:21.023 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:20 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/1139569931' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:37:21.023 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:20 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:21.023 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:20 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:21.023 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:20 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T14:37:21.023 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:20 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-03T14:37:21.023 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:20 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:37:21.098 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-05-03T14:37:21.098 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/ceph/ceph.conf 2024-05-03T14:37:21.127 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-05-03T14:37:21.128 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-03T14:37:21.192 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-03T14:37:21.193 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/ceph/ceph.conf 2024-05-03T14:37:21.224 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-03T14:37:21.224 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-03T14:37:21.292 INFO:tasks.cephadm:Adding mgr.x on smithi060 2024-05-03T14:37:21.292 INFO:tasks.cephadm:Adding mgr.y on smithi090 2024-05-03T14:37:21.292 INFO:tasks.cephadm:Adding mgr.z on smithi149 2024-05-03T14:37:21.292 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph orch apply mgr '3;smithi060=x;smithi090=y;smithi149=z' 2024-05-03T14:37:21.568 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T14:37:21.902 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:21 smithi149 ceph-mon[33332]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:21.902 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:21 smithi149 ceph-mon[33332]: Reconfiguring mon.b (monmap changed)... 2024-05-03T14:37:21.903 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:21 smithi149 ceph-mon[33332]: Reconfiguring daemon mon.b on smithi090 2024-05-03T14:37:22.022 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:21 smithi090 ceph-mon[33568]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:22.023 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:21 smithi090 ceph-mon[33568]: Reconfiguring mon.b (monmap changed)... 2024-05-03T14:37:22.023 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:21 smithi090 ceph-mon[33568]: Reconfiguring daemon mon.b on smithi090 2024-05-03T14:37:22.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:21 smithi060 ceph-mon[28873]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:22.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:21 smithi060 ceph-mon[28873]: Reconfiguring mon.b (monmap changed)... 2024-05-03T14:37:22.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:21 smithi060 ceph-mon[28873]: Reconfiguring daemon mon.b on smithi090 2024-05-03T14:37:23.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:23 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:23.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:23 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:23.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:23 smithi090 ceph-mon[33568]: Reconfiguring mon.c (monmap changed)... 2024-05-03T14:37:23.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:23 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T14:37:23.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:23 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-03T14:37:23.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:23 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:37:23.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:23 smithi090 ceph-mon[33568]: Reconfiguring daemon mon.c on smithi149 2024-05-03T14:37:23.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:23 smithi090 ceph-mon[33568]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:23.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:23 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:23.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:23 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:23.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:23 smithi149 ceph-mon[33332]: Reconfiguring mon.c (monmap changed)... 2024-05-03T14:37:23.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:23 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T14:37:23.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:23 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-03T14:37:23.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:23 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:37:23.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:23 smithi149 ceph-mon[33332]: Reconfiguring daemon mon.c on smithi149 2024-05-03T14:37:23.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:23 smithi149 ceph-mon[33332]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:23.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:23 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:23.589 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:23 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:23.589 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:23 smithi060 ceph-mon[28873]: Reconfiguring mon.c (monmap changed)... 2024-05-03T14:37:23.589 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:23 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T14:37:23.589 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:23 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-03T14:37:23.589 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:23 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:37:23.589 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:23 smithi060 ceph-mon[28873]: Reconfiguring daemon mon.c on smithi149 2024-05-03T14:37:23.589 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:23 smithi060 ceph-mon[28873]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:23.810 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:23.805+0000 7f7c3b31b700 1 -- 172.21.15.149:0/1385788143 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c341012f0 msgr2=0x7f7c34101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:37:23.811 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:23.805+0000 7f7c3b31b700 1 --2- 172.21.15.149:0/1385788143 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c341012f0 0x7f7c34101710 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7c24009910 tx=0x7f7c24009c20 comp rx=0 tx=0).stop 2024-05-03T14:37:23.811 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:23.805+0000 7f7c3b31b700 1 -- 172.21.15.149:0/1385788143 shutdown_connections 2024-05-03T14:37:23.811 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:23.805+0000 7f7c3b31b700 1 --2- 172.21.15.149:0/1385788143 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c341012f0 0x7f7c34101710 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:37:23.811 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:23.805+0000 7f7c3b31b700 1 -- 172.21.15.149:0/1385788143 >> 172.21.15.149:0/1385788143 conn(0x7f7c340fcc50 msgr2=0x7f7c340ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:37:23.811 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:23.806+0000 7f7c3b31b700 1 -- 172.21.15.149:0/1385788143 shutdown_connections 2024-05-03T14:37:23.811 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:23.806+0000 7f7c3b31b700 1 -- 172.21.15.149:0/1385788143 wait complete. 2024-05-03T14:37:23.811 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:23.806+0000 7f7c3b31b700 1 Processor -- start 2024-05-03T14:37:23.811 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:23.806+0000 7f7c3b31b700 1 -- start start 2024-05-03T14:37:23.811 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:23.806+0000 7f7c3b31b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c341012f0 0x7f7c34103c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:37:23.811 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:23.806+0000 7f7c3b31b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7c34102280 0x7f7c34102700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:37:23.811 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:23.806+0000 7f7c3b31b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7c34102c40 0x7f7c341a2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:37:23.811 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:23.806+0000 7f7c3b31b700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f7c341091b0 con 0x7f7c341012f0 2024-05-03T14:37:23.812 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:23.806+0000 7f7c3b31b700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7c34109030 con 0x7f7c34102280 2024-05-03T14:37:23.812 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:23.806+0000 7f7c3b31b700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7c34109330 con 0x7f7c34102c40 2024-05-03T14:37:23.812 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:23.806+0000 7f7c398b8700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7c34102c40 0x7f7c341a2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:37:23.812 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:23.806+0000 7f7c398b8700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7c34102c40 0x7f7c341a2a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:39510/0 (socket says 172.21.15.149:39510) 2024-05-03T14:37:23.812 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:23.806+0000 7f7c398b8700 1 -- 172.21.15.149:0/3714112827 learned_addr learned my addr 172.21.15.149:0/3714112827 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-05-03T14:37:23.812 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:23.807+0000 7f7c388b6700 1 --2- 172.21.15.149:0/3714112827 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7c34102280 0x7f7c34102700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:37:23.812 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:23.807+0000 7f7c398b8700 1 -- 172.21.15.149:0/3714112827 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7c34102280 msgr2=0x7f7c34102700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:37:23.812 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:23.807+0000 7f7c398b8700 1 --2- 172.21.15.149:0/3714112827 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7c34102280 0x7f7c34102700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:37:23.812 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:23.807+0000 7f7c398b8700 1 -- 172.21.15.149:0/3714112827 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c341012f0 msgr2=0x7f7c34103c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:37:23.812 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:23.807+0000 7f7c398b8700 1 --2- 172.21.15.149:0/3714112827 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c341012f0 0x7f7c34103c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:37:23.812 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:23.807+0000 7f7c398b8700 1 -- 172.21.15.149:0/3714112827 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c341a2fe0 con 0x7f7c34102c40 2024-05-03T14:37:23.812 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:23.807+0000 7f7c398b8700 1 --2- 172.21.15.149:0/3714112827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7c34102c40 0x7f7c341a2a20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c00ba50 tx=0x7f7c2c00be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:37:23.812 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:23.807+0000 7f7c2a7fc700 1 -- 172.21.15.149:0/3714112827 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c019070 con 0x7f7c34102c40 2024-05-03T14:37:23.813 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:23.807+0000 7f7c3b31b700 1 -- 172.21.15.149:0/3714112827 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c341a32d0 con 0x7f7c34102c40 2024-05-03T14:37:23.813 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:23.807+0000 7f7c2a7fc700 1 -- 172.21.15.149:0/3714112827 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f7c2c014440 con 0x7f7c34102c40 2024-05-03T14:37:23.813 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:23.807+0000 7f7c2a7fc700 1 -- 172.21.15.149:0/3714112827 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c007590 con 0x7f7c34102c40 2024-05-03T14:37:23.813 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:23.807+0000 7f7c3b31b700 1 -- 172.21.15.149:0/3714112827 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c341a3890 con 0x7f7c34102c40 2024-05-03T14:37:23.813 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:23.808+0000 7f7c2a7fc700 1 -- 172.21.15.149:0/3714112827 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f7c2c01c070 con 0x7f7c34102c40 2024-05-03T14:37:23.813 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:23.808+0000 7f7c3b31b700 1 -- 172.21.15.149:0/3714112827 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c34064b00 con 0x7f7c34102c40 2024-05-03T14:37:23.813 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:23.808+0000 7f7c2a7fc700 1 --2- 172.21.15.149:0/3714112827 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f7c2003e9a0 0x7f7c20040e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:37:23.813 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:23.808+0000 7f7c2a7fc700 1 -- 172.21.15.149:0/3714112827 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f7c2c051ee0 con 0x7f7c34102c40 2024-05-03T14:37:23.813 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:23.809+0000 7f7c390b7700 1 --2- 172.21.15.149:0/3714112827 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f7c2003e9a0 0x7f7c20040e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:37:23.814 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:23.810+0000 7f7c390b7700 1 --2- 172.21.15.149:0/3714112827 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f7c2003e9a0 0x7f7c20040e60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7c2400b5c0 tx=0x7f7c240053a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:37:23.816 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:23.814+0000 7f7c2a7fc700 1 -- 172.21.15.149:0/3714112827 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f7c2c014a10 con 0x7f7c34102c40 2024-05-03T14:37:24.038 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:24.035+0000 7f7c3b31b700 1 -- 172.21.15.149:0/3714112827 --> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi060=x;smithi090=y;smithi149=z", "target": ["mon-mgr", ""]}) -- 0x7f7c34063970 con 0x7f7c2003e9a0 2024-05-03T14:37:24.051 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:24.049+0000 7f7c2a7fc700 1 -- 172.21.15.149:0/3714112827 <== mgr.14152 v2:172.21.15.60:6800/3066142956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f7c34063970 con 0x7f7c2003e9a0 2024-05-03T14:37:24.051 INFO:teuthology.orchestra.run.smithi149.stdout:Scheduled mgr update... 2024-05-03T14:37:24.054 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:24.051+0000 7f7c3b31b700 1 -- 172.21.15.149:0/3714112827 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f7c2003e9a0 msgr2=0x7f7c20040e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:37:24.054 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:24.051+0000 7f7c3b31b700 1 --2- 172.21.15.149:0/3714112827 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f7c2003e9a0 0x7f7c20040e60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7c2400b5c0 tx=0x7f7c240053a0 comp rx=0 tx=0).stop 2024-05-03T14:37:24.054 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:24.051+0000 7f7c3b31b700 1 -- 172.21.15.149:0/3714112827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7c34102c40 msgr2=0x7f7c341a2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:37:24.054 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:24.051+0000 7f7c3b31b700 1 --2- 172.21.15.149:0/3714112827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7c34102c40 0x7f7c341a2a20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c00ba50 tx=0x7f7c2c00be10 comp rx=0 tx=0).stop 2024-05-03T14:37:24.054 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:24.051+0000 7f7c3b31b700 1 -- 172.21.15.149:0/3714112827 shutdown_connections 2024-05-03T14:37:24.054 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:24.051+0000 7f7c3b31b700 1 --2- 172.21.15.149:0/3714112827 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f7c2003e9a0 0x7f7c20040e60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:37:24.054 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:24.051+0000 7f7c3b31b700 1 --2- 172.21.15.149:0/3714112827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7c34102c40 0x7f7c341a2a20 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:37:24.054 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:24.051+0000 7f7c3b31b700 1 --2- 172.21.15.149:0/3714112827 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c341012f0 0x7f7c34103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:37:24.055 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:24.051+0000 7f7c3b31b700 1 --2- 172.21.15.149:0/3714112827 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7c34102280 0x7f7c34102700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:37:24.055 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:24.051+0000 7f7c3b31b700 1 -- 172.21.15.149:0/3714112827 >> 172.21.15.149:0/3714112827 conn(0x7f7c340fcc50 msgr2=0x7f7c340fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:37:24.055 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:24.051+0000 7f7c3b31b700 1 -- 172.21.15.149:0/3714112827 shutdown_connections 2024-05-03T14:37:24.055 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:37:24.051+0000 7f7c3b31b700 1 -- 172.21.15.149:0/3714112827 wait complete. 2024-05-03T14:37:24.778 DEBUG:teuthology.orchestra.run.smithi090:mgr.y> sudo journalctl -f -n 0 -u ceph-285d64b2-095a-11ef-bc97-c7b262605968@mgr.y.service 2024-05-03T14:37:24.781 DEBUG:teuthology.orchestra.run.smithi149:mgr.z> sudo journalctl -f -n 0 -u ceph-285d64b2-095a-11ef-bc97-c7b262605968@mgr.z.service 2024-05-03T14:37:24.783 INFO:tasks.cephadm:Deploying OSDs... 2024-05-03T14:37:24.783 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-05-03T14:37:24.783 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/scratch_devs of=/dev/stdout 2024-05-03T14:37:24.842 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-03T14:37:24.842 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_1 2024-05-03T14:37:24.895 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-03T14:37:24.895 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-03T14:37:24.895 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 6h/6d Inode: 632 Links: 1 2024-05-03T14:37:24.895 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-03T14:37:24.895 INFO:teuthology.orchestra.run.smithi060.stdout:Context: system_u:object_r:device_t:s0 2024-05-03T14:37:24.895 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2024-05-03 14:36:17.808393120 +0000 2024-05-03T14:37:24.895 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2024-05-03 14:31:20.104713454 +0000 2024-05-03T14:37:24.895 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2024-05-03 14:31:20.104713454 +0000 2024-05-03T14:37:24.895 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: 2024-05-03 14:31:20.104713454 +0000 2024-05-03T14:37:24.896 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-03T14:37:24.977 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-05-03T14:37:24.977 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-05-03T14:37:24.977 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 0.000308203 s, 1.7 MB/s 2024-05-03T14:37:24.981 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-03T14:37:25.040 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_2 2024-05-03T14:37:25.053 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:25 smithi090 ceph-mon[33568]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi060=x;smithi090=y;smithi149=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:37:25.053 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:25 smithi090 ceph-mon[33568]: Saving service mgr spec with placement smithi060=x;smithi090=y;smithi149=z;count:3 2024-05-03T14:37:25.053 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:25 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:25.053 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:25 smithi090 ceph-mon[33568]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:25.053 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:25 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:25.053 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:25 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:25.053 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:25 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:37:25.092 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-03T14:37:25.092 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-03T14:37:25.092 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 6h/6d Inode: 650 Links: 1 2024-05-03T14:37:25.092 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-03T14:37:25.092 INFO:teuthology.orchestra.run.smithi060.stdout:Context: system_u:object_r:device_t:s0 2024-05-03T14:37:25.092 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2024-05-03 14:36:17.895392318 +0000 2024-05-03T14:37:25.092 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2024-05-03 14:31:20.699707526 +0000 2024-05-03T14:37:25.092 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2024-05-03 14:31:20.699707526 +0000 2024-05-03T14:37:25.093 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: 2024-05-03 14:31:20.699707526 +0000 2024-05-03T14:37:25.093 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-03T14:37:25.164 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-05-03T14:37:25.164 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-05-03T14:37:25.164 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 0.000205288 s, 2.5 MB/s 2024-05-03T14:37:25.167 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-03T14:37:25.225 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_3 2024-05-03T14:37:25.277 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-03T14:37:25.277 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-03T14:37:25.277 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 6h/6d Inode: 670 Links: 1 2024-05-03T14:37:25.277 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-03T14:37:25.277 INFO:teuthology.orchestra.run.smithi060.stdout:Context: system_u:object_r:device_t:s0 2024-05-03T14:37:25.277 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2024-05-03 14:36:17.991391433 +0000 2024-05-03T14:37:25.277 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2024-05-03 14:31:21.309701449 +0000 2024-05-03T14:37:25.278 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2024-05-03 14:31:21.309701449 +0000 2024-05-03T14:37:25.278 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: 2024-05-03 14:31:21.309701449 +0000 2024-05-03T14:37:25.278 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-03T14:37:25.327 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:25 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:37:25.327 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:25 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:37:25.327 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:25 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:25.328 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:25 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-03T14:37:25.328 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:25 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-03T14:37:25.328 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:25 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-03T14:37:25.328 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:25 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:37:25.328 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:25 smithi090 ceph-mon[33568]: Deploying daemon mgr.z on smithi149 2024-05-03T14:37:25.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:25 smithi060 ceph-mon[28873]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi060=x;smithi090=y;smithi149=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:37:25.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:25 smithi060 ceph-mon[28873]: Saving service mgr spec with placement smithi060=x;smithi090=y;smithi149=z;count:3 2024-05-03T14:37:25.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:25 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:25.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:25 smithi060 ceph-mon[28873]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:25.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:25 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:25.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:25 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:25.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:25 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:37:25.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:25 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:37:25.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:25 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:37:25.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:25 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:25.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:25 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-03T14:37:25.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:25 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-03T14:37:25.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:25 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-03T14:37:25.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:25 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:37:25.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:25 smithi060 ceph-mon[28873]: Deploying daemon mgr.z on smithi149 2024-05-03T14:37:25.361 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-05-03T14:37:25.361 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-05-03T14:37:25.361 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 0.000261956 s, 2.0 MB/s 2024-05-03T14:37:25.366 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-03T14:37:25.427 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_4 2024-05-03T14:37:25.450 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:25 smithi149 ceph-mon[33332]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi060=x;smithi090=y;smithi149=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:37:25.450 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:25 smithi149 ceph-mon[33332]: Saving service mgr spec with placement smithi060=x;smithi090=y;smithi149=z;count:3 2024-05-03T14:37:25.450 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:25 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:25.450 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:25 smithi149 ceph-mon[33332]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:25.450 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:25 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:25.450 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:25 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:25.450 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:25 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:37:25.451 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:25 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:37:25.451 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:25 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:37:25.451 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:25 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:25.451 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:25 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-03T14:37:25.451 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:25 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-03T14:37:25.451 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:25 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-03T14:37:25.451 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:25 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:37:25.451 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:25 smithi149 ceph-mon[33332]: Deploying daemon mgr.z on smithi149 2024-05-03T14:37:25.505 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-03T14:37:25.505 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-03T14:37:25.505 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 6h/6d Inode: 691 Links: 1 2024-05-03T14:37:25.505 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-03T14:37:25.505 INFO:teuthology.orchestra.run.smithi060.stdout:Context: system_u:object_r:device_t:s0 2024-05-03T14:37:25.505 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2024-05-03 14:36:18.062390778 +0000 2024-05-03T14:37:25.505 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2024-05-03 14:31:21.937695192 +0000 2024-05-03T14:37:25.505 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2024-05-03 14:31:21.937695192 +0000 2024-05-03T14:37:25.505 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: 2024-05-03 14:31:21.937695192 +0000 2024-05-03T14:37:25.506 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-03T14:37:25.589 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-05-03T14:37:25.589 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-05-03T14:37:25.589 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 0.000368577 s, 1.4 MB/s 2024-05-03T14:37:25.593 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-03T14:37:25.651 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-05-03T14:37:25.652 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/scratch_devs of=/dev/stdout 2024-05-03T14:37:25.667 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-03T14:37:25.667 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_1 2024-05-03T14:37:25.723 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-03T14:37:25.723 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-03T14:37:25.723 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-05-03T14:37:25.723 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-03T14:37:25.723 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:device_t:s0 2024-05-03T14:37:25.723 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-05-03 14:37:08.633836334 +0000 2024-05-03T14:37:25.723 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-05-03 14:29:57.783447839 +0000 2024-05-03T14:37:25.723 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-05-03 14:29:57.783447839 +0000 2024-05-03T14:37:25.723 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: 2024-05-03 14:29:57.783447839 +0000 2024-05-03T14:37:25.723 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-03T14:37:25.785 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2024-05-03T14:37:25.785 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2024-05-03T14:37:25.785 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.000269868 s, 1.9 MB/s 2024-05-03T14:37:25.786 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-03T14:37:25.842 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_2 2024-05-03T14:37:25.895 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-03T14:37:25.896 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-03T14:37:25.896 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-05-03T14:37:25.896 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-03T14:37:25.896 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:device_t:s0 2024-05-03T14:37:25.896 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-05-03 14:37:08.634836321 +0000 2024-05-03T14:37:25.896 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-05-03 14:29:58.300440592 +0000 2024-05-03T14:37:25.896 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-05-03 14:29:58.300440592 +0000 2024-05-03T14:37:25.896 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: 2024-05-03 14:29:58.300440592 +0000 2024-05-03T14:37:25.896 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-03T14:37:25.957 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2024-05-03T14:37:25.958 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2024-05-03T14:37:25.958 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.000184748 s, 2.8 MB/s 2024-05-03T14:37:25.959 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-03T14:37:26.013 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_3 2024-05-03T14:37:26.067 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-03T14:37:26.068 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-03T14:37:26.068 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-05-03T14:37:26.068 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-03T14:37:26.068 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:device_t:s0 2024-05-03T14:37:26.068 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-05-03 14:37:08.634836321 +0000 2024-05-03T14:37:26.068 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-05-03 14:29:58.618436134 +0000 2024-05-03T14:37:26.068 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-05-03 14:29:58.618436134 +0000 2024-05-03T14:37:26.068 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: 2024-05-03 14:29:58.618436134 +0000 2024-05-03T14:37:26.068 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-03T14:37:26.129 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2024-05-03T14:37:26.129 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2024-05-03T14:37:26.129 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.000177617 s, 2.9 MB/s 2024-05-03T14:37:26.131 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-03T14:37:26.186 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_4 2024-05-03T14:37:26.242 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-03T14:37:26.242 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-03T14:37:26.242 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-05-03T14:37:26.242 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-03T14:37:26.242 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:device_t:s0 2024-05-03T14:37:26.242 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-05-03 14:37:08.634836321 +0000 2024-05-03T14:37:26.242 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-05-03 14:29:58.960431340 +0000 2024-05-03T14:37:26.242 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-05-03 14:29:58.960431340 +0000 2024-05-03T14:37:26.242 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: 2024-05-03 14:29:58.960431340 +0000 2024-05-03T14:37:26.243 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-03T14:37:26.304 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2024-05-03T14:37:26.304 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2024-05-03T14:37:26.304 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.000234702 s, 2.2 MB/s 2024-05-03T14:37:26.305 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-03T14:37:26.361 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-03T14:37:26.361 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/scratch_devs of=/dev/stdout 2024-05-03T14:37:26.377 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-03T14:37:26.377 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_1 2024-05-03T14:37:26.433 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-03T14:37:26.433 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-03T14:37:26.433 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-05-03T14:37:26.433 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-03T14:37:26.433 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2024-05-03T14:37:26.433 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-05-03 14:37:26.291196571 +0000 2024-05-03T14:37:26.433 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-05-03 14:29:57.784815991 +0000 2024-05-03T14:37:26.433 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-05-03 14:29:57.784815991 +0000 2024-05-03T14:37:26.434 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-05-03 14:29:57.784815991 +0000 2024-05-03T14:37:26.434 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-03T14:37:26.495 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 03 14:37:26 smithi149 systemd[1]: Starting Ceph mgr.z for 285d64b2-095a-11ef-bc97-c7b262605968... 2024-05-03T14:37:26.498 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-05-03T14:37:26.498 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-05-03T14:37:26.498 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000246119 s, 2.1 MB/s 2024-05-03T14:37:26.499 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-03T14:37:26.555 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_2 2024-05-03T14:37:26.610 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-03T14:37:26.610 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-03T14:37:26.610 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-05-03T14:37:26.610 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-03T14:37:26.610 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2024-05-03T14:37:26.610 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-05-03 14:37:26.292196551 +0000 2024-05-03T14:37:26.611 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-05-03 14:29:58.154808260 +0000 2024-05-03T14:37:26.611 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-05-03 14:29:58.154808260 +0000 2024-05-03T14:37:26.611 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-05-03 14:29:58.154808260 +0000 2024-05-03T14:37:26.611 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-03T14:37:26.673 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-05-03T14:37:26.673 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-05-03T14:37:26.673 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000232066 s, 2.2 MB/s 2024-05-03T14:37:26.674 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-03T14:37:26.729 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_3 2024-05-03T14:37:26.783 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-03T14:37:26.783 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-03T14:37:26.783 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-05-03T14:37:26.783 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-03T14:37:26.783 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2024-05-03T14:37:26.783 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-05-03 14:37:26.292196551 +0000 2024-05-03T14:37:26.783 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-05-03 14:29:58.492801198 +0000 2024-05-03T14:37:26.784 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-05-03 14:29:58.492801198 +0000 2024-05-03T14:37:26.784 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-05-03 14:29:58.492801198 +0000 2024-05-03T14:37:26.784 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-03T14:37:26.812 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 03 14:37:26 smithi149 podman[34985]: 2024-05-03 14:37:26.480305662 +0000 UTC m=+0.018259123 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T14:37:26.812 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 03 14:37:26 smithi149 podman[34985]: 2024-05-03 14:37:26.59111502 +0000 UTC m=+0.129068504 container create 5a2811f816a5be827827d19c52011014f1a3e4b1098f80f1aa91b055798e90a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-z, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-03T14:37:26.813 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 03 14:37:26 smithi149 podman[34985]: 2024-05-03 14:37:26.810530043 +0000 UTC m=+0.348483494 container init 5a2811f816a5be827827d19c52011014f1a3e4b1098f80f1aa91b055798e90a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430) 2024-05-03T14:37:26.835 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-05-03T14:37:26.835 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-05-03T14:37:26.835 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000178807 s, 2.9 MB/s 2024-05-03T14:37:26.836 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-03T14:37:26.892 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_4 2024-05-03T14:37:26.949 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-03T14:37:26.949 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-03T14:37:26.949 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-05-03T14:37:26.950 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-03T14:37:26.950 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2024-05-03T14:37:26.950 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-05-03 14:37:26.292196551 +0000 2024-05-03T14:37:26.950 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-05-03 14:29:58.837793989 +0000 2024-05-03T14:37:26.950 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-05-03 14:29:58.837793989 +0000 2024-05-03T14:37:26.950 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-05-03 14:29:58.837793989 +0000 2024-05-03T14:37:26.950 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-03T14:37:27.013 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-05-03T14:37:27.014 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-05-03T14:37:27.014 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000205092 s, 2.5 MB/s 2024-05-03T14:37:27.015 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-03T14:37:27.070 INFO:tasks.cephadm:Deploying osd.0 on smithi060 with /dev/vg_nvme/lv_4... 2024-05-03T14:37:27.070 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-03T14:37:27.174 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 03 14:37:26 smithi149 podman[34985]: 2024-05-03 14:37:26.814706644 +0000 UTC m=+0.352660087 container start 5a2811f816a5be827827d19c52011014f1a3e4b1098f80f1aa91b055798e90a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-z, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T14:37:27.174 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 03 14:37:26 smithi149 ceph-mgr[35114]: -- 172.21.15.149:0/3061798653 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56491328c5a0 con 0x564913260c00 2024-05-03T14:37:27.174 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 03 14:37:26 smithi149 bash[34985]: 5a2811f816a5be827827d19c52011014f1a3e4b1098f80f1aa91b055798e90a3 2024-05-03T14:37:27.174 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 03 14:37:26 smithi149 systemd[1]: Started Ceph mgr.z for 285d64b2-095a-11ef-bc97-c7b262605968. 2024-05-03T14:37:27.174 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 03 14:37:26 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-z[35072]: 2024-05-03T14:37:26.980+0000 7f6cd03b9200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-03T14:37:27.529 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:37:27.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:27 smithi060 ceph-mon[28873]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:27.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:27 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:27.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:27 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:27.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:27 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:27.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:27 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-03T14:37:27.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:27 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-03T14:37:27.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:27 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-03T14:37:27.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:27 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:37:27.589 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 03 14:37:27 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-z[35072]: 2024-05-03T14:37:27.173+0000 7f6cd03b9200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-03T14:37:27.589 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 03 14:37:27 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-z[35072]: 2024-05-03T14:37:27.276+0000 7f6cd03b9200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-03T14:37:27.589 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 03 14:37:27 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-z[35072]: 2024-05-03T14:37:27.586+0000 7f6cd03b9200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-03T14:37:27.589 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:27 smithi149 ceph-mon[33332]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:27.589 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:27 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:27.589 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:27 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:27.589 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:27 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:27.589 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:27 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-03T14:37:27.589 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:27 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-03T14:37:27.589 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:27 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-03T14:37:27.590 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:27 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:37:27.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:27 smithi090 ceph-mon[33568]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:27.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:27 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:27.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:27 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:27.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:27 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:27.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:27 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-03T14:37:27.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:27 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-03T14:37:27.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:27 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-03T14:37:27.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:27 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:37:27.923 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 03 14:37:27 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-z[35072]: 2024-05-03T14:37:27.670+0000 7f6cd03b9200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-03T14:37:28.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:28 smithi149 ceph-mon[33332]: Deploying daemon mgr.y on smithi090 2024-05-03T14:37:28.522 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:28 smithi060 ceph-mon[28873]: Deploying daemon mgr.y on smithi090 2024-05-03T14:37:28.564 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:28 smithi090 ceph-mon[33568]: Deploying daemon mgr.y on smithi090 2024-05-03T14:37:29.173 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 03 14:37:28 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-z[35072]: 2024-05-03T14:37:28.878+0000 7f6cd03b9200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-03T14:37:29.174 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 03 14:37:29 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-z[35072]: 2024-05-03T14:37:29.047+0000 7f6cd03b9200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-03T14:37:29.394 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:29 smithi060 ceph-mon[28873]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:29.515 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:29 smithi149 ceph-mon[33332]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:29.516 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 03 14:37:29 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-z[35072]: 2024-05-03T14:37:29.222+0000 7f6cd03b9200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-03T14:37:29.516 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 03 14:37:29 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-z[35072]: 2024-05-03T14:37:29.513+0000 7f6cd03b9200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-03T14:37:29.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:29 smithi090 ceph-mon[33568]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:29.625 INFO:journalctl@ceph.mgr.y.smithi090.stdout:May 03 14:37:29 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-y[34968]: 2024-05-03T14:37:29.480+0000 7ff35736a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-03T14:37:29.819 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 03 14:37:29 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-z[35072]: 2024-05-03T14:37:29.606+0000 7f6cd03b9200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-03T14:37:29.819 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 03 14:37:29 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-z[35072]: 2024-05-03T14:37:29.816+0000 7f6cd03b9200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-03T14:37:30.095 INFO:journalctl@ceph.mgr.y.smithi090.stdout:May 03 14:37:29 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-y[34968]: 2024-05-03T14:37:29.663+0000 7ff35736a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-03T14:37:30.095 INFO:journalctl@ceph.mgr.y.smithi090.stdout:May 03 14:37:29 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-y[34968]: 2024-05-03T14:37:29.768+0000 7ff35736a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-03T14:37:30.095 INFO:journalctl@ceph.mgr.y.smithi090.stdout:May 03 14:37:30 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-y[34968]: 2024-05-03T14:37:30.091+0000 7ff35736a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-03T14:37:30.173 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 03 14:37:30 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-z[35072]: 2024-05-03T14:37:30.017+0000 7f6cd03b9200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-03T14:37:30.375 INFO:journalctl@ceph.mgr.y.smithi090.stdout:May 03 14:37:30 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-y[34968]: 2024-05-03T14:37:30.179+0000 7ff35736a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-03T14:37:30.476 INFO:teuthology.orchestra.run.smithi060.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-03T14:37:30.476 INFO:teuthology.orchestra.run.smithi060.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-03T14:37:30.476 INFO:teuthology.orchestra.run.smithi060.stderr: stderr: 10+0 records in 2024-05-03T14:37:30.476 INFO:teuthology.orchestra.run.smithi060.stderr:10+0 records out 2024-05-03T14:37:30.476 INFO:teuthology.orchestra.run.smithi060.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0368403 s, 285 MB/s 2024-05-03T14:37:30.476 INFO:teuthology.orchestra.run.smithi060.stderr:--> Zapping successful for: 2024-05-03T14:37:30.788 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:30 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:30.788 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:30 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:30.788 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:30 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:30.788 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:30 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:30.788 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:30 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:37:30.792 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 03 14:37:30 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-z[35072]: 2024-05-03T14:37:30.668+0000 7f6cd03b9200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-03T14:37:30.792 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:30 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:30.792 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:30 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:30.792 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:30 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:30.792 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:30 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:30.792 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:30 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:37:30.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:30 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:30.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:30 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:30.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:30 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:30.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:30 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:30.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:30 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:37:31.173 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 03 14:37:30 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-z[35072]: 2024-05-03T14:37:30.898+0000 7f6cd03b9200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-03T14:37:31.428 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi060:vg_nvme/lv_4 2024-05-03T14:37:31.674 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 03 14:37:31 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-z[35072]: 2024-05-03T14:37:31.329+0000 7f6cd03b9200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-03T14:37:31.674 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 03 14:37:31 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-z[35072]: 2024-05-03T14:37:31.425+0000 7f6cd03b9200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-03T14:37:31.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:31 smithi149 ceph-mon[33332]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:31.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:31 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:31.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:31 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:31.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:31 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:31.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:31 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:31.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:31 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:37:31.675 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:31 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:37:31.675 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:31 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:31.675 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:31 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-03T14:37:31.675 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:31 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-03T14:37:31.675 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:31 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:37:31.760 INFO:journalctl@ceph.mgr.y.smithi090.stdout:May 03 14:37:31 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-y[34968]: 2024-05-03T14:37:31.423+0000 7ff35736a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-03T14:37:31.760 INFO:journalctl@ceph.mgr.y.smithi090.stdout:May 03 14:37:31 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-y[34968]: 2024-05-03T14:37:31.590+0000 7ff35736a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-03T14:37:31.760 INFO:journalctl@ceph.mgr.y.smithi090.stdout:May 03 14:37:31 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-y[34968]: 2024-05-03T14:37:31.757+0000 7ff35736a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-03T14:37:31.760 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:31 smithi090 ceph-mon[33568]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:31.760 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:31 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:31.760 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:31 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:31.760 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:31 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:31.760 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:31 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:31.761 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:31 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:37:31.761 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:31 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:37:31.761 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:31 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:31.761 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:31 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-03T14:37:31.761 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:31 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-03T14:37:31.761 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:31 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:37:31.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:31 smithi060 ceph-mon[28873]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:31.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:31 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:31.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:31 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:31.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:31 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:31.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:31 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:31.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:31 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:37:31.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:31 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:37:31.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:31 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:31.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:31 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-03T14:37:31.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:31 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-03T14:37:31.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:31 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:37:31.898 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:37:32.157 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 03 14:37:31 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-z[35072]: 2024-05-03T14:37:31.804+0000 7f6cd03b9200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-03T14:37:32.158 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 03 14:37:32 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-z[35072]: 2024-05-03T14:37:32.155+0000 7f6cd03b9200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-03T14:37:32.329 INFO:journalctl@ceph.mgr.y.smithi090.stdout:May 03 14:37:32 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-y[34968]: 2024-05-03T14:37:32.038+0000 7ff35736a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-03T14:37:32.329 INFO:journalctl@ceph.mgr.y.smithi090.stdout:May 03 14:37:32 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-y[34968]: 2024-05-03T14:37:32.128+0000 7ff35736a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-03T14:37:32.329 INFO:journalctl@ceph.mgr.y.smithi090.stdout:May 03 14:37:32 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-y[34968]: 2024-05-03T14:37:32.326+0000 7ff35736a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-03T14:37:32.423 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 03 14:37:32 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-z[35072]: 2024-05-03T14:37:32.245+0000 7f6cd03b9200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-03T14:37:32.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:32 smithi090 ceph-mon[33568]: Reconfiguring mgr.x (unknown last config time)... 2024-05-03T14:37:32.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:32 smithi090 ceph-mon[33568]: Reconfiguring daemon mgr.x on smithi060 2024-05-03T14:37:32.625 INFO:journalctl@ceph.mgr.y.smithi090.stdout:May 03 14:37:32 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-y[34968]: 2024-05-03T14:37:32.515+0000 7ff35736a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-03T14:37:32.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:32 smithi060 ceph-mon[28873]: Reconfiguring mgr.x (unknown last config time)... 2024-05-03T14:37:32.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:32 smithi060 ceph-mon[28873]: Reconfiguring daemon mgr.x on smithi060 2024-05-03T14:37:32.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:32 smithi149 ceph-mon[33332]: Reconfiguring mgr.x (unknown last config time)... 2024-05-03T14:37:32.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:32 smithi149 ceph-mon[33332]: Reconfiguring daemon mgr.x on smithi060 2024-05-03T14:37:32.924 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 03 14:37:32 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-z[35072]: 2024-05-03T14:37:32.484+0000 7f6cd03b9200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-03T14:37:32.924 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 03 14:37:32 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-z[35072]: 2024-05-03T14:37:32.617+0000 7f6cd03b9200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-03T14:37:33.375 INFO:journalctl@ceph.mgr.y.smithi090.stdout:May 03 14:37:33 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-y[34968]: 2024-05-03T14:37:33.103+0000 7ff35736a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-03T14:37:33.375 INFO:journalctl@ceph.mgr.y.smithi090.stdout:May 03 14:37:33 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-y[34968]: 2024-05-03T14:37:33.319+0000 7ff35736a200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-03T14:37:33.423 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 03 14:37:32 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-z[35072]: 2024-05-03T14:37:32.928+0000 7f6cd03b9200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-03T14:37:33.424 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 03 14:37:33 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-z[35072]: 2024-05-03T14:37:33.042+0000 7f6cd03b9200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-03T14:37:33.424 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 03 14:37:33 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-z[35072]: 2024-05-03T14:37:33.132+0000 7f6cd03b9200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-03T14:37:33.741 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:33 smithi060 ceph-mon[28873]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:33.829 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:33 smithi090 ceph-mon[33568]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:33.830 INFO:journalctl@ceph.mgr.y.smithi090.stdout:May 03 14:37:33 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-y[34968]: 2024-05-03T14:37:33.733+0000 7ff35736a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-03T14:37:33.830 INFO:journalctl@ceph.mgr.y.smithi090.stdout:May 03 14:37:33 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-y[34968]: 2024-05-03T14:37:33.826+0000 7ff35736a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-03T14:37:33.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:33 smithi149 ceph-mon[33332]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:33.924 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 03 14:37:33 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-z[35072]: 2024-05-03T14:37:33.729+0000 7f6cd03b9200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-03T14:37:34.424 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 03 14:37:33 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-z[35072]: 2024-05-03T14:37:33.943+0000 7f6cd03b9200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-03T14:37:34.539 INFO:journalctl@ceph.mgr.y.smithi090.stdout:May 03 14:37:34 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-y[34968]: 2024-05-03T14:37:34.188+0000 7ff35736a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-03T14:37:34.539 INFO:journalctl@ceph.mgr.y.smithi090.stdout:May 03 14:37:34 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-y[34968]: 2024-05-03T14:37:34.535+0000 7ff35736a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-03T14:37:34.797 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:34 smithi060 ceph-mon[28873]: Standby manager daemon z started 2024-05-03T14:37:34.798 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:34 smithi060 ceph-mon[28873]: from='mgr.? 172.21.15.149:0/2925525373' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-03T14:37:34.798 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:34 smithi060 ceph-mon[28873]: from='mgr.? 172.21.15.149:0/2925525373' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-03T14:37:34.798 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:34 smithi060 ceph-mon[28873]: from='mgr.? 172.21.15.149:0/2925525373' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-03T14:37:34.798 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:34 smithi060 ceph-mon[28873]: from='mgr.? 172.21.15.149:0/2925525373' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-03T14:37:34.798 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:34 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:34.867 INFO:journalctl@ceph.mgr.y.smithi090.stdout:May 03 14:37:34 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-y[34968]: 2024-05-03T14:37:34.627+0000 7ff35736a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-03T14:37:34.867 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:34 smithi090 ceph-mon[33568]: Standby manager daemon z started 2024-05-03T14:37:34.867 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:34 smithi090 ceph-mon[33568]: from='mgr.? 172.21.15.149:0/2925525373' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-03T14:37:34.867 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:34 smithi090 ceph-mon[33568]: from='mgr.? 172.21.15.149:0/2925525373' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-03T14:37:34.867 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:34 smithi090 ceph-mon[33568]: from='mgr.? 172.21.15.149:0/2925525373' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-03T14:37:34.867 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:34 smithi090 ceph-mon[33568]: from='mgr.? 172.21.15.149:0/2925525373' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-03T14:37:34.867 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:34 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:34.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:34 smithi149 ceph-mon[33332]: Standby manager daemon z started 2024-05-03T14:37:34.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:34 smithi149 ceph-mon[33332]: from='mgr.? 172.21.15.149:0/2925525373' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-03T14:37:34.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:34 smithi149 ceph-mon[33332]: from='mgr.? 172.21.15.149:0/2925525373' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-03T14:37:34.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:34 smithi149 ceph-mon[33332]: from='mgr.? 172.21.15.149:0/2925525373' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-03T14:37:34.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:34 smithi149 ceph-mon[33332]: from='mgr.? 172.21.15.149:0/2925525373' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-03T14:37:34.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:34 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:34.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:34.971+0000 7f571d000700 1 -- 172.21.15.60:0/3521736949 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f571810b400 msgr2=0x7f571810d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:37:34.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:34.971+0000 7f571d000700 1 --2- 172.21.15.60:0/3521736949 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f571810b400 0x7f571810d870 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5710009f90 tx=0x7f571000b330 comp rx=0 tx=0).stop 2024-05-03T14:37:34.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:34.971+0000 7f571d000700 1 -- 172.21.15.60:0/3521736949 shutdown_connections 2024-05-03T14:37:34.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:34.971+0000 7f571d000700 1 --2- 172.21.15.60:0/3521736949 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f571810b400 0x7f571810d870 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:37:34.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:34.971+0000 7f571d000700 1 --2- 172.21.15.60:0/3521736949 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5718077450 0x7f57180758b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:37:34.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:34.971+0000 7f571d000700 1 --2- 172.21.15.60:0/3521736949 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5718075e80 0x7f5718076300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:37:34.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:34.971+0000 7f571d000700 1 -- 172.21.15.60:0/3521736949 >> 172.21.15.60:0/3521736949 conn(0x7f57180feaa0 msgr2=0x7f5718100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:37:34.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:34.971+0000 7f571d000700 1 -- 172.21.15.60:0/3521736949 shutdown_connections 2024-05-03T14:37:34.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:34.971+0000 7f571d000700 1 -- 172.21.15.60:0/3521736949 wait complete. 2024-05-03T14:37:34.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:34.972+0000 7f571d000700 1 Processor -- start 2024-05-03T14:37:34.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:34.972+0000 7f571d000700 1 -- start start 2024-05-03T14:37:34.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:34.973+0000 7f571d000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5718075e80 0x7f571819b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:37:34.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:34.973+0000 7f571d000700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5718077450 0x7f571819be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:37:34.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:34.973+0000 7f571d000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f571810b400 0x7f57181a2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:37:34.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:34.973+0000 7f571d000700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5718111400 con 0x7f5718077450 2024-05-03T14:37:34.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:34.973+0000 7f571d000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5718111280 con 0x7f571810b400 2024-05-03T14:37:34.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:34.973+0000 7f571d000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5718111580 con 0x7f5718075e80 2024-05-03T14:37:34.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:34.973+0000 7f5715e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5718075e80 0x7f571819b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:37:34.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:34.973+0000 7f5715e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5718075e80 0x7f571819b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.60:45732/0 (socket says 172.21.15.60:45732) 2024-05-03T14:37:34.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:34.973+0000 7f5715e00700 1 -- 172.21.15.60:0/2030688776 learned_addr learned my addr 172.21.15.60:0/2030688776 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:37:34.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:34.973+0000 7f5716800700 1 --2- 172.21.15.60:0/2030688776 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f571810b400 0x7f57181a2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:37:34.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:34.973+0000 7f5715400700 1 --2- 172.21.15.60:0/2030688776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5718077450 0x7f571819be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:37:34.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:34.974+0000 7f5716800700 1 -- 172.21.15.60:0/2030688776 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5718075e80 msgr2=0x7f571819b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:37:34.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:34.974+0000 7f5716800700 1 --2- 172.21.15.60:0/2030688776 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5718075e80 0x7f571819b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:37:34.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:34.974+0000 7f5716800700 1 -- 172.21.15.60:0/2030688776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5718077450 msgr2=0x7f571819be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:37:34.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:34.974+0000 7f5716800700 1 --2- 172.21.15.60:0/2030688776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5718077450 0x7f571819be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:37:34.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:34.974+0000 7f5716800700 1 -- 172.21.15.60:0/2030688776 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57181a3560 con 0x7f571810b400 2024-05-03T14:37:34.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:34.974+0000 7f5716800700 1 --2- 172.21.15.60:0/2030688776 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f571810b400 0x7f57181a2ec0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f571000d010 tx=0x7f5710009b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:37:34.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:34.974+0000 7f570ea00700 1 -- 172.21.15.60:0/2030688776 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5710021070 con 0x7f571810b400 2024-05-03T14:37:34.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:34.974+0000 7f570ea00700 1 -- 172.21.15.60:0/2030688776 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f5710007ac0 con 0x7f571810b400 2024-05-03T14:37:34.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:34.974+0000 7f570ea00700 1 -- 172.21.15.60:0/2030688776 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57100052a0 con 0x7f571810b400 2024-05-03T14:37:34.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:34.974+0000 7f571d000700 1 -- 172.21.15.60:0/2030688776 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57181a37f0 con 0x7f571810b400 2024-05-03T14:37:34.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:34.974+0000 7f571d000700 1 -- 172.21.15.60:0/2030688776 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57181a3c50 con 0x7f571810b400 2024-05-03T14:37:34.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:34.976+0000 7f571d000700 1 -- 172.21.15.60:0/2030688776 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f571806a190 con 0x7f571810b400 2024-05-03T14:37:34.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:34.977+0000 7f570ea00700 1 -- 172.21.15.60:0/2030688776 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 14) ==== 99937+0+0 (secure 0 0 0) 0x7f5710024070 con 0x7f571810b400 2024-05-03T14:37:34.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:34.978+0000 7f570ea00700 1 --2- 172.21.15.60:0/2030688776 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f56fc0787a0 0x7f56fc07ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:37:34.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:34.978+0000 7f570ea00700 1 -- 172.21.15.60:0/2030688776 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f571009ab50 con 0x7f571810b400 2024-05-03T14:37:34.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:34.978+0000 7f5715e00700 1 --2- 172.21.15.60:0/2030688776 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f56fc0787a0 0x7f56fc07ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:37:34.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:34.979+0000 7f5715e00700 1 --2- 172.21.15.60:0/2030688776 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f56fc0787a0 0x7f56fc07ac60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5708009d60 tx=0x7f5708009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:37:34.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:34.983+0000 7f570ea00700 1 -- 172.21.15.60:0/2030688776 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f5710064640 con 0x7f571810b400 2024-05-03T14:37:35.125 INFO:journalctl@ceph.mgr.y.smithi090.stdout:May 03 14:37:34 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-y[34968]: 2024-05-03T14:37:34.864+0000 7ff35736a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-03T14:37:35.125 INFO:journalctl@ceph.mgr.y.smithi090.stdout:May 03 14:37:35 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-y[34968]: 2024-05-03T14:37:35.000+0000 7ff35736a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-03T14:37:35.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:35.238+0000 7f571d000700 1 -- 172.21.15.60:0/2030688776 --> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f57180638e0 con 0x7f56fc0787a0 2024-05-03T14:37:35.573 INFO:journalctl@ceph.mgr.y.smithi090.stdout:May 03 14:37:35 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-y[34968]: 2024-05-03T14:37:35.309+0000 7ff35736a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-03T14:37:35.573 INFO:journalctl@ceph.mgr.y.smithi090.stdout:May 03 14:37:35 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-y[34968]: 2024-05-03T14:37:35.429+0000 7ff35736a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-03T14:37:35.573 INFO:journalctl@ceph.mgr.y.smithi090.stdout:May 03 14:37:35 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-y[34968]: 2024-05-03T14:37:35.524+0000 7ff35736a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-03T14:37:35.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:35 smithi060 ceph-mon[28873]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:35.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:35 smithi060 ceph-mon[28873]: mgrmap e14: x(active, since 90s), standbys: z 2024-05-03T14:37:35.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:35 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-03T14:37:35.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:35 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:35.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:35 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:35.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:35 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:37:35.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:35 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T14:37:35.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:35 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T14:37:35.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:35 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:37:35.839 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:35 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:37:35.839 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:35 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:37:35.839 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:35 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:35.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:35 smithi090 ceph-mon[33568]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:35.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:35 smithi090 ceph-mon[33568]: mgrmap e14: x(active, since 90s), standbys: z 2024-05-03T14:37:35.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:35 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-03T14:37:35.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:35 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:35.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:35 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:35.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:35 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:37:35.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:35 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T14:37:35.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:35 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T14:37:35.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:35 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:37:35.876 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:35 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:37:35.876 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:35 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:37:35.876 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:35 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:35.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:35 smithi149 ceph-mon[33332]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:35.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:35 smithi149 ceph-mon[33332]: mgrmap e14: x(active, since 90s), standbys: z 2024-05-03T14:37:35.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:35 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-03T14:37:35.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:35 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:35.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:35 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:35.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:35 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:37:35.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:35 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T14:37:35.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:35 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T14:37:35.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:35 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:37:35.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:35 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:37:35.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:35 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:37:35.925 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:35 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:36.590 INFO:journalctl@ceph.mgr.y.smithi090.stdout:May 03 14:37:36 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-y[34968]: 2024-05-03T14:37:36.123+0000 7ff35736a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-03T14:37:36.590 INFO:journalctl@ceph.mgr.y.smithi090.stdout:May 03 14:37:36 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-y[34968]: 2024-05-03T14:37:36.344+0000 7ff35736a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-03T14:37:36.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:37:36.590+0000 7f570ea00700 1 -- 172.21.15.60:0/2030688776 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7f5710007c70 con 0x7f571810b400 2024-05-03T14:37:36.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:36 smithi060 ceph-mon[28873]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:37:36.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:36 smithi060 ceph-mon[28873]: Standby manager daemon y started 2024-05-03T14:37:36.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:36 smithi060 ceph-mon[28873]: from='mgr.? 172.21.15.90:0/987456052' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-03T14:37:36.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:36 smithi060 ceph-mon[28873]: from='mgr.? 172.21.15.90:0/987456052' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-03T14:37:36.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:36 smithi060 ceph-mon[28873]: from='mgr.? 172.21.15.90:0/987456052' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-03T14:37:36.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:36 smithi060 ceph-mon[28873]: from='mgr.? 172.21.15.90:0/987456052' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-03T14:37:36.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:36 smithi090 ceph-mon[33568]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:37:36.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:36 smithi090 ceph-mon[33568]: Standby manager daemon y started 2024-05-03T14:37:36.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:36 smithi090 ceph-mon[33568]: from='mgr.? 172.21.15.90:0/987456052' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-03T14:37:36.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:36 smithi090 ceph-mon[33568]: from='mgr.? 172.21.15.90:0/987456052' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-03T14:37:36.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:36 smithi090 ceph-mon[33568]: from='mgr.? 172.21.15.90:0/987456052' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-03T14:37:36.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:36 smithi090 ceph-mon[33568]: from='mgr.? 172.21.15.90:0/987456052' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-03T14:37:36.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:36 smithi149 ceph-mon[33332]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:37:36.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:36 smithi149 ceph-mon[33332]: Standby manager daemon y started 2024-05-03T14:37:36.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:36 smithi149 ceph-mon[33332]: from='mgr.? 172.21.15.90:0/987456052' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-03T14:37:36.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:36 smithi149 ceph-mon[33332]: from='mgr.? 172.21.15.90:0/987456052' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-03T14:37:36.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:36 smithi149 ceph-mon[33332]: from='mgr.? 172.21.15.90:0/987456052' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-03T14:37:36.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:36 smithi149 ceph-mon[33332]: from='mgr.? 172.21.15.90:0/987456052' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-03T14:37:37.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:37 smithi090 ceph-mon[33568]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:37.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:37 smithi090 ceph-mon[33568]: mgrmap e15: x(active, since 92s), standbys: z, y 2024-05-03T14:37:37.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:37 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-03T14:37:37.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:37 smithi149 ceph-mon[33332]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:37.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:37 smithi149 ceph-mon[33332]: mgrmap e15: x(active, since 92s), standbys: z, y 2024-05-03T14:37:37.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:37 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-03T14:37:37.946 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:37 smithi060 ceph-mon[28873]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:37.946 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:37 smithi060 ceph-mon[28873]: mgrmap e15: x(active, since 92s), standbys: z, y 2024-05-03T14:37:37.946 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:37 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-03T14:37:39.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:39 smithi090 ceph-mon[33568]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:39.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:39 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/522925591' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f0797730-c4c1-47ea-9dce-9270d5ebb4ce"}]: dispatch 2024-05-03T14:37:39.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:39 smithi090 ceph-mon[33568]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f0797730-c4c1-47ea-9dce-9270d5ebb4ce"}]: dispatch 2024-05-03T14:37:39.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:39 smithi090 ceph-mon[33568]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f0797730-c4c1-47ea-9dce-9270d5ebb4ce"}]': finished 2024-05-03T14:37:39.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:39 smithi090 ceph-mon[33568]: osdmap e5: 1 total, 0 up, 1 in 2024-05-03T14:37:39.876 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:39 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T14:37:39.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:39 smithi149 ceph-mon[33332]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:39.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:39 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/522925591' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f0797730-c4c1-47ea-9dce-9270d5ebb4ce"}]: dispatch 2024-05-03T14:37:39.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:39 smithi149 ceph-mon[33332]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f0797730-c4c1-47ea-9dce-9270d5ebb4ce"}]: dispatch 2024-05-03T14:37:39.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:39 smithi149 ceph-mon[33332]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f0797730-c4c1-47ea-9dce-9270d5ebb4ce"}]': finished 2024-05-03T14:37:39.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:39 smithi149 ceph-mon[33332]: osdmap e5: 1 total, 0 up, 1 in 2024-05-03T14:37:39.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:39 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T14:37:40.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:39 smithi060 ceph-mon[28873]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:40.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:39 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/522925591' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f0797730-c4c1-47ea-9dce-9270d5ebb4ce"}]: dispatch 2024-05-03T14:37:40.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:39 smithi060 ceph-mon[28873]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f0797730-c4c1-47ea-9dce-9270d5ebb4ce"}]: dispatch 2024-05-03T14:37:40.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:39 smithi060 ceph-mon[28873]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f0797730-c4c1-47ea-9dce-9270d5ebb4ce"}]': finished 2024-05-03T14:37:40.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:39 smithi060 ceph-mon[28873]: osdmap e5: 1 total, 0 up, 1 in 2024-05-03T14:37:40.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:39 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T14:37:40.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:40 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/3507342028' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T14:37:40.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:40 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/3507342028' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T14:37:41.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:40 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/3507342028' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T14:37:41.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:41 smithi090 ceph-mon[33568]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:41.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:41 smithi149 ceph-mon[33332]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:42.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:41 smithi060 ceph-mon[28873]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:43.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:43 smithi090 ceph-mon[33568]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:43.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:43 smithi149 ceph-mon[33332]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:44.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:43 smithi060 ceph-mon[28873]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:45.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:45 smithi060 ceph-mon[28873]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:45.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:45 smithi090 ceph-mon[33568]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:45.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:45 smithi149 ceph-mon[33332]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:47.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:47 smithi090 ceph-mon[33568]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:47.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:47 smithi149 ceph-mon[33332]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:47.981 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:47 smithi060 ceph-mon[28873]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:49.897 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:49 smithi060 ceph-mon[28873]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:49.897 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:49 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-03T14:37:49.897 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:49 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:37:49.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:49 smithi149 ceph-mon[33332]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:49.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:49 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-03T14:37:49.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:49 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:37:50.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:49 smithi090 ceph-mon[33568]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:50.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:49 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-03T14:37:50.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:49 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:37:50.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:50 smithi149 ceph-mon[33332]: Deploying daemon osd.0 on smithi060 2024-05-03T14:37:50.925 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:50 smithi060 ceph-mon[28873]: Deploying daemon osd.0 on smithi060 2024-05-03T14:37:51.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:50 smithi090 ceph-mon[33568]: Deploying daemon osd.0 on smithi060 2024-05-03T14:37:51.748 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:51 smithi060 ceph-mon[28873]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:51.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:51 smithi149 ceph-mon[33332]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:52.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:51 smithi090 ceph-mon[33568]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:53.805 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:53 smithi060 ceph-mon[28873]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:53.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:53 smithi149 ceph-mon[33332]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:54.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:53 smithi090 ceph-mon[33568]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:55.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:55 smithi060 ceph-mon[28873]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:55.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:55 smithi149 ceph-mon[33332]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:55.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:55 smithi090 ceph-mon[33568]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:57.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:57 smithi060 ceph-mon[28873]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:57.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:57 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:57.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:57 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:57.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:57 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:37:57.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:57 smithi149 ceph-mon[33332]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:57.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:57 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:57.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:57 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:57.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:57 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:37:58.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:57 smithi090 ceph-mon[33568]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:58.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:57 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:58.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:57 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:37:58.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:57 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:37:59.912 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:37:59 smithi060 ceph-mon[28873]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:37:59.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:37:59 smithi149 ceph-mon[33332]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:38:00.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:37:59 smithi090 ceph-mon[33568]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:38:01.802 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:01 smithi060 ceph-mon[28873]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:38:01.802 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:01 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:01.802 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:01 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:01.802 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:01 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:38:01.802 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:01 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:38:01.803 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:01 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:01.803 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:01 smithi060 ceph-mon[28873]: from='osd.0 [v2:172.21.15.60:6802/2952930162,v1:172.21.15.60:6803/2952930162]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-03T14:38:01.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:01 smithi090 ceph-mon[33568]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:38:01.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:01 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:01.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:01 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:01.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:01 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:38:01.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:01 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:38:01.876 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:01 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:01.876 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:01 smithi090 ceph-mon[33568]: from='osd.0 [v2:172.21.15.60:6802/2952930162,v1:172.21.15.60:6803/2952930162]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-03T14:38:01.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:38:01.898+0000 7f570ea00700 1 -- 172.21.15.60:0/2030688776 <== mgr.14152 v2:172.21.15.60:6800/3066142956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f57180638e0 con 0x7f56fc0787a0 2024-05-03T14:38:01.901 INFO:teuthology.orchestra.run.smithi060.stdout:Created osd(s) 0 on host 'smithi060' 2024-05-03T14:38:01.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:38:01.902+0000 7f571d000700 1 -- 172.21.15.60:0/2030688776 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f56fc0787a0 msgr2=0x7f56fc07ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:38:01.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:38:01.902+0000 7f571d000700 1 --2- 172.21.15.60:0/2030688776 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f56fc0787a0 0x7f56fc07ac60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5708009d60 tx=0x7f5708009260 comp rx=0 tx=0).stop 2024-05-03T14:38:01.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:38:01.902+0000 7f571d000700 1 -- 172.21.15.60:0/2030688776 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f571810b400 msgr2=0x7f57181a2ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:38:01.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:38:01.902+0000 7f571d000700 1 --2- 172.21.15.60:0/2030688776 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f571810b400 0x7f57181a2ec0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f571000d010 tx=0x7f5710009b00 comp rx=0 tx=0).stop 2024-05-03T14:38:01.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:38:01.902+0000 7f571d000700 1 -- 172.21.15.60:0/2030688776 shutdown_connections 2024-05-03T14:38:01.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:38:01.902+0000 7f571d000700 1 --2- 172.21.15.60:0/2030688776 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f56fc0787a0 0x7f56fc07ac60 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:38:01.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:38:01.902+0000 7f571d000700 1 --2- 172.21.15.60:0/2030688776 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f571810b400 0x7f57181a2ec0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:38:01.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:38:01.902+0000 7f571d000700 1 --2- 172.21.15.60:0/2030688776 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5718075e80 0x7f571819b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:38:01.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:38:01.902+0000 7f571d000700 1 --2- 172.21.15.60:0/2030688776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5718077450 0x7f571819be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:38:01.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:38:01.902+0000 7f571d000700 1 -- 172.21.15.60:0/2030688776 >> 172.21.15.60:0/2030688776 conn(0x7f57180feaa0 msgr2=0x7f5718107350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:38:01.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:38:01.903+0000 7f571d000700 1 -- 172.21.15.60:0/2030688776 shutdown_connections 2024-05-03T14:38:01.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:38:01.903+0000 7f571d000700 1 -- 172.21.15.60:0/2030688776 wait complete. 2024-05-03T14:38:01.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:01 smithi149 ceph-mon[33332]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:38:01.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:01 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:01.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:01 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:01.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:01 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:38:01.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:01 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:38:01.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:01 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:01.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:01 smithi149 ceph-mon[33332]: from='osd.0 [v2:172.21.15.60:6802/2952930162,v1:172.21.15.60:6803/2952930162]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-03T14:38:02.753 DEBUG:teuthology.orchestra.run.smithi060:osd.0> sudo journalctl -f -n 0 -u ceph-285d64b2-095a-11ef-bc97-c7b262605968@osd.0.service 2024-05-03T14:38:02.758 INFO:tasks.cephadm:Deploying osd.1 on smithi090 with /dev/vg_nvme/lv_4... 2024-05-03T14:38:02.758 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-03T14:38:02.972 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:02 smithi060 ceph-mon[28873]: from='osd.0 [v2:172.21.15.60:6802/2952930162,v1:172.21.15.60:6803/2952930162]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-03T14:38:02.972 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:02 smithi060 ceph-mon[28873]: osdmap e6: 1 total, 0 up, 1 in 2024-05-03T14:38:02.972 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:02 smithi060 ceph-mon[28873]: from='osd.0 [v2:172.21.15.60:6802/2952930162,v1:172.21.15.60:6803/2952930162]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-05-03T14:38:02.972 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:02 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T14:38:02.972 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:02 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:02.972 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:02 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:02.973 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:02 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:38:03.001 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.b/config 2024-05-03T14:38:03.120 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:02 smithi090 ceph-mon[33568]: from='osd.0 [v2:172.21.15.60:6802/2952930162,v1:172.21.15.60:6803/2952930162]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-03T14:38:03.120 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:02 smithi090 ceph-mon[33568]: osdmap e6: 1 total, 0 up, 1 in 2024-05-03T14:38:03.120 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:02 smithi090 ceph-mon[33568]: from='osd.0 [v2:172.21.15.60:6802/2952930162,v1:172.21.15.60:6803/2952930162]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-05-03T14:38:03.121 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:02 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T14:38:03.121 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:02 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:03.121 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:02 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:03.121 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:02 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:38:03.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:02 smithi149 ceph-mon[33332]: from='osd.0 [v2:172.21.15.60:6802/2952930162,v1:172.21.15.60:6803/2952930162]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-03T14:38:03.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:02 smithi149 ceph-mon[33332]: osdmap e6: 1 total, 0 up, 1 in 2024-05-03T14:38:03.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:02 smithi149 ceph-mon[33332]: from='osd.0 [v2:172.21.15.60:6802/2952930162,v1:172.21.15.60:6803/2952930162]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-05-03T14:38:03.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:02 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T14:38:03.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:02 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:03.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:02 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:03.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:02 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:38:04.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:03 smithi060 ceph-mon[28873]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:38:04.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:03 smithi060 ceph-mon[28873]: from='osd.0 [v2:172.21.15.60:6802/2952930162,v1:172.21.15.60:6803/2952930162]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-05-03T14:38:04.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:03 smithi060 ceph-mon[28873]: osdmap e7: 1 total, 0 up, 1 in 2024-05-03T14:38:04.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:03 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T14:38:04.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:03 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T14:38:04.090 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:03 smithi090 ceph-mon[33568]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:38:04.090 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:03 smithi090 ceph-mon[33568]: from='osd.0 [v2:172.21.15.60:6802/2952930162,v1:172.21.15.60:6803/2952930162]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-05-03T14:38:04.090 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:03 smithi090 ceph-mon[33568]: osdmap e7: 1 total, 0 up, 1 in 2024-05-03T14:38:04.090 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:03 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T14:38:04.090 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:03 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T14:38:04.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:03 smithi149 ceph-mon[33332]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:38:04.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:03 smithi149 ceph-mon[33332]: from='osd.0 [v2:172.21.15.60:6802/2952930162,v1:172.21.15.60:6803/2952930162]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-05-03T14:38:04.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:03 smithi149 ceph-mon[33332]: osdmap e7: 1 total, 0 up, 1 in 2024-05-03T14:38:04.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:03 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T14:38:04.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:03 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T14:38:04.710 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:04 smithi060 ceph-mon[28873]: purged_snaps scrub starts 2024-05-03T14:38:04.710 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:04 smithi060 ceph-mon[28873]: purged_snaps scrub ok 2024-05-03T14:38:04.710 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:04 smithi060 ceph-mon[28873]: osd.0 [v2:172.21.15.60:6802/2952930162,v1:172.21.15.60:6803/2952930162] boot 2024-05-03T14:38:04.710 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:04 smithi060 ceph-mon[28873]: osdmap e8: 1 total, 1 up, 1 in 2024-05-03T14:38:04.710 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:04 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T14:38:04.710 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:04 smithi060 ceph-mon[28873]: pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:38:04.765 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-03T14:38:04.765 INFO:teuthology.orchestra.run.smithi090.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-03T14:38:04.765 INFO:teuthology.orchestra.run.smithi090.stderr: stderr: 10+0 records in 2024-05-03T14:38:04.766 INFO:teuthology.orchestra.run.smithi090.stderr:10+0 records out 2024-05-03T14:38:04.766 INFO:teuthology.orchestra.run.smithi090.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0131113 s, 800 MB/s 2024-05-03T14:38:04.766 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping successful for: 2024-05-03T14:38:05.123 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:04 smithi090 ceph-mon[33568]: purged_snaps scrub starts 2024-05-03T14:38:05.123 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:04 smithi090 ceph-mon[33568]: purged_snaps scrub ok 2024-05-03T14:38:05.123 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:04 smithi090 ceph-mon[33568]: osd.0 [v2:172.21.15.60:6802/2952930162,v1:172.21.15.60:6803/2952930162] boot 2024-05-03T14:38:05.123 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:04 smithi090 ceph-mon[33568]: osdmap e8: 1 total, 1 up, 1 in 2024-05-03T14:38:05.123 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:04 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T14:38:05.123 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:04 smithi090 ceph-mon[33568]: pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:38:05.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:04 smithi149 ceph-mon[33332]: purged_snaps scrub starts 2024-05-03T14:38:05.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:04 smithi149 ceph-mon[33332]: purged_snaps scrub ok 2024-05-03T14:38:05.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:04 smithi149 ceph-mon[33332]: osd.0 [v2:172.21.15.60:6802/2952930162,v1:172.21.15.60:6803/2952930162] boot 2024-05-03T14:38:05.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:04 smithi149 ceph-mon[33332]: osdmap e8: 1 total, 1 up, 1 in 2024-05-03T14:38:05.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:04 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T14:38:05.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:04 smithi149 ceph-mon[33332]: pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T14:38:05.329 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi090:vg_nvme/lv_4 2024-05-03T14:38:05.563 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.b/config 2024-05-03T14:38:05.983 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:05 smithi090 ceph-mon[33568]: osdmap e9: 1 total, 1 up, 1 in 2024-05-03T14:38:06.032 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:05 smithi060 ceph-mon[28873]: osdmap e9: 1 total, 1 up, 1 in 2024-05-03T14:38:06.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:05 smithi149 ceph-mon[33332]: osdmap e9: 1 total, 1 up, 1 in 2024-05-03T14:38:06.740 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:06 smithi090 ceph-mon[33568]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T14:38:06.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:38:06.947+0000 7ff6199c7700 1 -- 172.21.15.90:0/2420809667 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff614103fa0 msgr2=0x7ff61410a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:38:06.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:38:06.947+0000 7ff6199c7700 1 --2- 172.21.15.90:0/2420809667 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff614103fa0 0x7ff61410a850 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff60c0099c0 tx=0x7ff60c009cd0 comp rx=0 tx=0).stop 2024-05-03T14:38:06.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:38:06.947+0000 7ff6199c7700 1 -- 172.21.15.90:0/2420809667 shutdown_connections 2024-05-03T14:38:06.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:38:06.947+0000 7ff6199c7700 1 --2- 172.21.15.90:0/2420809667 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff614103fa0 0x7ff61410a850 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:38:06.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:38:06.947+0000 7ff6199c7700 1 --2- 172.21.15.90:0/2420809667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff614102400 0x7ff614102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:38:06.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:38:06.947+0000 7ff6199c7700 1 --2- 172.21.15.90:0/2420809667 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff614103600 0x7ff614103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:38:06.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:38:06.947+0000 7ff6199c7700 1 -- 172.21.15.90:0/2420809667 >> 172.21.15.90:0/2420809667 conn(0x7ff6140fd9a0 msgr2=0x7ff6140ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:38:06.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:38:06.947+0000 7ff6199c7700 1 -- 172.21.15.90:0/2420809667 shutdown_connections 2024-05-03T14:38:06.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:38:06.947+0000 7ff6199c7700 1 -- 172.21.15.90:0/2420809667 wait complete. 2024-05-03T14:38:06.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:38:06.948+0000 7ff6199c7700 1 Processor -- start 2024-05-03T14:38:06.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:38:06.948+0000 7ff6199c7700 1 -- start start 2024-05-03T14:38:06.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:38:06.948+0000 7ff6199c7700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff614102400 0x7ff614199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:38:06.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:38:06.948+0000 7ff6199c7700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff614103600 0x7ff614199d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:38:06.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:38:06.948+0000 7ff6199c7700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff614103fa0 0x7ff6141a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:38:06.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:38:06.948+0000 7ff6199c7700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff61410e1c0 con 0x7ff614103600 2024-05-03T14:38:06.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:38:06.948+0000 7ff6199c7700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff61410e040 con 0x7ff614103fa0 2024-05-03T14:38:06.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:38:06.948+0000 7ff6199c7700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff61410e340 con 0x7ff614102400 2024-05-03T14:38:06.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:38:06.948+0000 7ff6137fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff614103fa0 0x7ff6141a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:38:06.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:38:06.948+0000 7ff6137fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff614103fa0 0x7ff6141a0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49858/0 (socket says 172.21.15.90:49858) 2024-05-03T14:38:06.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:38:06.948+0000 7ff6137fe700 1 -- 172.21.15.90:0/299724575 learned_addr learned my addr 172.21.15.90:0/299724575 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-03T14:38:06.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:38:06.948+0000 7ff612ffd700 1 --2- 172.21.15.90:0/299724575 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff614102400 0x7ff614199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:38:06.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:38:06.948+0000 7ff6137fe700 1 -- 172.21.15.90:0/299724575 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff614102400 msgr2=0x7ff614199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:38:06.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:38:06.948+0000 7ff6137fe700 1 --2- 172.21.15.90:0/299724575 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff614102400 0x7ff614199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:38:06.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:38:06.948+0000 7ff6137fe700 1 -- 172.21.15.90:0/299724575 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff614103600 msgr2=0x7ff614199d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:38:06.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:38:06.948+0000 7ff6137fe700 1 --2- 172.21.15.90:0/299724575 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff614103600 0x7ff614199d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:38:06.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:38:06.948+0000 7ff6137fe700 1 -- 172.21.15.90:0/299724575 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6141a14b0 con 0x7ff614103fa0 2024-05-03T14:38:06.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:38:06.948+0000 7ff612ffd700 1 --2- 172.21.15.90:0/299724575 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff614102400 0x7ff614199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T14:38:06.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:38:06.949+0000 7ff6137fe700 1 --2- 172.21.15.90:0/299724575 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff614103fa0 0x7ff6141a0e10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff60c009990 tx=0x7ff60c0047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:38:06.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:38:06.949+0000 7ff6189c5700 1 -- 172.21.15.90:0/299724575 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60c027070 con 0x7ff614103fa0 2024-05-03T14:38:06.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:38:06.949+0000 7ff6189c5700 1 -- 172.21.15.90:0/299724575 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7ff60c003800 con 0x7ff614103fa0 2024-05-03T14:38:06.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:38:06.949+0000 7ff6199c7700 1 -- 172.21.15.90:0/299724575 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6141a1740 con 0x7ff614103fa0 2024-05-03T14:38:06.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:38:06.949+0000 7ff6189c5700 1 -- 172.21.15.90:0/299724575 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60c00f460 con 0x7ff614103fa0 2024-05-03T14:38:06.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:38:06.949+0000 7ff6199c7700 1 -- 172.21.15.90:0/299724575 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6141a1c10 con 0x7ff614103fa0 2024-05-03T14:38:06.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:38:06.950+0000 7ff6199c7700 1 -- 172.21.15.90:0/299724575 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6141a1e70 con 0x7ff614103fa0 2024-05-03T14:38:06.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:38:06.951+0000 7ff6189c5700 1 -- 172.21.15.90:0/299724575 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7ff60c02a070 con 0x7ff614103fa0 2024-05-03T14:38:06.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:38:06.951+0000 7ff6189c5700 1 --2- 172.21.15.90:0/299724575 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7ff6000b27f0 0x7ff6000b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:38:06.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:38:06.952+0000 7ff6189c5700 1 -- 172.21.15.90:0/299724575 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7ff60c0e1480 con 0x7ff614103fa0 2024-05-03T14:38:06.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:38:06.952+0000 7ff612ffd700 1 --2- 172.21.15.90:0/299724575 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7ff6000b27f0 0x7ff6000b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:38:06.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:38:06.953+0000 7ff612ffd700 1 --2- 172.21.15.90:0/299724575 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7ff6000b27f0 0x7ff6000b4cb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff614103460 tx=0x7ff6040073c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:38:06.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:38:06.956+0000 7ff6189c5700 1 -- 172.21.15.90:0/299724575 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7ff60c013070 con 0x7ff614103fa0 2024-05-03T14:38:07.000 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:06 smithi060 ceph-mon[28873]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T14:38:07.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:38:07.167+0000 7ff6199c7700 1 -- 172.21.15.90:0/299724575 --> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7ff6140ffae0 con 0x7ff6000b27f0 2024-05-03T14:38:07.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:06 smithi149 ceph-mon[33332]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T14:38:08.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:07 smithi090 ceph-mon[33568]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:38:08.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:07 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T14:38:08.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:07 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T14:38:08.021 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:07 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:38:08.021 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:07 smithi090 ceph-mon[33568]: Detected new or changed devices on smithi060 2024-05-03T14:38:08.021 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:07 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:08.021 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:07 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:08.021 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:07 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:38:08.021 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:07 smithi090 ceph-mon[33568]: Adjusting osd_memory_target on smithi060 to 17094M 2024-05-03T14:38:08.021 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:07 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:08.021 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:07 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:38:08.021 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:07 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:38:08.021 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:07 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:08.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:07 smithi060 ceph-mon[28873]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:38:08.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:07 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T14:38:08.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:07 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T14:38:08.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:07 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:38:08.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:07 smithi060 ceph-mon[28873]: Detected new or changed devices on smithi060 2024-05-03T14:38:08.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:07 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:08.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:07 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:08.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:07 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:38:08.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:07 smithi060 ceph-mon[28873]: Adjusting osd_memory_target on smithi060 to 17094M 2024-05-03T14:38:08.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:07 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:08.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:07 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:38:08.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:07 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:38:08.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:07 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:08.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:07 smithi149 ceph-mon[33332]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:38:08.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:07 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T14:38:08.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:07 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T14:38:08.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:07 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:38:08.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:07 smithi149 ceph-mon[33332]: Detected new or changed devices on smithi060 2024-05-03T14:38:08.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:07 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:08.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:07 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:08.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:07 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:38:08.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:07 smithi149 ceph-mon[33332]: Adjusting osd_memory_target on smithi060 to 17094M 2024-05-03T14:38:08.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:07 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:08.175 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:07 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:38:08.175 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:07 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:38:08.175 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:07 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:08.979 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:08 smithi090 ceph-mon[33568]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T14:38:09.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:08 smithi060 ceph-mon[28873]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T14:38:09.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:08 smithi149 ceph-mon[33332]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T14:38:10.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:09 smithi090 ceph-mon[33568]: from='client.? 172.21.15.90:0/2246155269' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "96d0ad08-cda4-4824-bad4-e64973508c0b"}]: dispatch 2024-05-03T14:38:10.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:09 smithi090 ceph-mon[33568]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "96d0ad08-cda4-4824-bad4-e64973508c0b"}]: dispatch 2024-05-03T14:38:10.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:09 smithi090 ceph-mon[33568]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "96d0ad08-cda4-4824-bad4-e64973508c0b"}]': finished 2024-05-03T14:38:10.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:09 smithi090 ceph-mon[33568]: osdmap e10: 2 total, 1 up, 2 in 2024-05-03T14:38:10.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:09 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T14:38:10.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:09 smithi149 ceph-mon[33332]: from='client.? 172.21.15.90:0/2246155269' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "96d0ad08-cda4-4824-bad4-e64973508c0b"}]: dispatch 2024-05-03T14:38:10.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:09 smithi149 ceph-mon[33332]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "96d0ad08-cda4-4824-bad4-e64973508c0b"}]: dispatch 2024-05-03T14:38:10.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:09 smithi149 ceph-mon[33332]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "96d0ad08-cda4-4824-bad4-e64973508c0b"}]': finished 2024-05-03T14:38:10.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:09 smithi149 ceph-mon[33332]: osdmap e10: 2 total, 1 up, 2 in 2024-05-03T14:38:10.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:09 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T14:38:10.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:09 smithi060 ceph-mon[28873]: from='client.? 172.21.15.90:0/2246155269' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "96d0ad08-cda4-4824-bad4-e64973508c0b"}]: dispatch 2024-05-03T14:38:10.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:09 smithi060 ceph-mon[28873]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "96d0ad08-cda4-4824-bad4-e64973508c0b"}]: dispatch 2024-05-03T14:38:10.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:09 smithi060 ceph-mon[28873]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "96d0ad08-cda4-4824-bad4-e64973508c0b"}]': finished 2024-05-03T14:38:10.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:09 smithi060 ceph-mon[28873]: osdmap e10: 2 total, 1 up, 2 in 2024-05-03T14:38:10.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:09 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T14:38:11.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:10 smithi090 ceph-mon[33568]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T14:38:11.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:10 smithi090 ceph-mon[33568]: from='client.? 172.21.15.90:0/790004868' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T14:38:11.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:10 smithi149 ceph-mon[33332]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T14:38:11.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:10 smithi149 ceph-mon[33332]: from='client.? 172.21.15.90:0/790004868' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T14:38:11.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:10 smithi060 ceph-mon[28873]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T14:38:11.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:10 smithi060 ceph-mon[28873]: from='client.? 172.21.15.90:0/790004868' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T14:38:13.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:13 smithi060 ceph-mon[28873]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T14:38:13.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:13 smithi090 ceph-mon[33568]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T14:38:13.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:13 smithi149 ceph-mon[33332]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T14:38:15.562 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:15 smithi090 ceph-mon[33568]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T14:38:15.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:15 smithi060 ceph-mon[28873]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T14:38:15.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:15 smithi149 ceph-mon[33332]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T14:38:17.522 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:17 smithi090 ceph-mon[33568]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T14:38:17.522 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:17 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-03T14:38:17.522 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:17 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:38:17.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:17 smithi060 ceph-mon[28873]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T14:38:17.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:17 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-03T14:38:17.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:17 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:38:17.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:17 smithi149 ceph-mon[33332]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T14:38:17.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:17 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-03T14:38:17.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:17 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:38:18.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:18 smithi060 ceph-mon[28873]: Deploying daemon osd.1 on smithi090 2024-05-03T14:38:18.608 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:18 smithi090 ceph-mon[33568]: Deploying daemon osd.1 on smithi090 2024-05-03T14:38:18.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:18 smithi149 ceph-mon[33332]: Deploying daemon osd.1 on smithi090 2024-05-03T14:38:19.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:19 smithi060 ceph-mon[28873]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T14:38:19.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:19 smithi090 ceph-mon[33568]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T14:38:19.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:19 smithi149 ceph-mon[33332]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T14:38:21.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:21 smithi060 ceph-mon[28873]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T14:38:21.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:21 smithi090 ceph-mon[33568]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T14:38:21.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:21 smithi149 ceph-mon[33332]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T14:38:23.570 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:23 smithi090 ceph-mon[33568]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T14:38:23.570 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:23 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:23.571 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:23 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:23.571 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:23 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:38:23.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:23 smithi060 ceph-mon[28873]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T14:38:23.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:23 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:23.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:23 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:23.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:23 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:38:23.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:23 smithi149 ceph-mon[33332]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T14:38:23.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:23 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:23.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:23 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:23.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:23 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:38:25.811 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:25 smithi090 ceph-mon[33568]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T14:38:25.811 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:25 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:25.811 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:25 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:25.811 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:25 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:38:25.811 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:25 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:38:25.811 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:25 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:25.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:25 smithi060 ceph-mon[28873]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T14:38:25.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:25 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:25.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:25 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:25.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:25 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:38:25.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:25 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:38:25.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:25 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:25.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:38:25.864+0000 7ff6189c5700 1 -- 172.21.15.90:0/299724575 <== mgr.14152 v2:172.21.15.60:6800/3066142956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff6140ffae0 con 0x7ff6000b27f0 2024-05-03T14:38:25.868 INFO:teuthology.orchestra.run.smithi090.stdout:Created osd(s) 1 on host 'smithi090' 2024-05-03T14:38:25.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:38:25.866+0000 7ff6199c7700 1 -- 172.21.15.90:0/299724575 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7ff6000b27f0 msgr2=0x7ff6000b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:38:25.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:38:25.866+0000 7ff6199c7700 1 --2- 172.21.15.90:0/299724575 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7ff6000b27f0 0x7ff6000b4cb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff614103460 tx=0x7ff6040073c0 comp rx=0 tx=0).stop 2024-05-03T14:38:25.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:38:25.866+0000 7ff6199c7700 1 -- 172.21.15.90:0/299724575 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff614103fa0 msgr2=0x7ff6141a0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:38:25.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:38:25.866+0000 7ff6199c7700 1 --2- 172.21.15.90:0/299724575 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff614103fa0 0x7ff6141a0e10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff60c009990 tx=0x7ff60c0047b0 comp rx=0 tx=0).stop 2024-05-03T14:38:25.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:38:25.866+0000 7ff6199c7700 1 -- 172.21.15.90:0/299724575 shutdown_connections 2024-05-03T14:38:25.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:38:25.867+0000 7ff6199c7700 1 --2- 172.21.15.90:0/299724575 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7ff6000b27f0 0x7ff6000b4cb0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:38:25.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:38:25.867+0000 7ff6199c7700 1 --2- 172.21.15.90:0/299724575 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff614103fa0 0x7ff6141a0e10 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:38:25.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:38:25.867+0000 7ff6199c7700 1 --2- 172.21.15.90:0/299724575 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff614102400 0x7ff614199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:38:25.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:38:25.867+0000 7ff6199c7700 1 --2- 172.21.15.90:0/299724575 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff614103600 0x7ff614199d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:38:25.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:38:25.867+0000 7ff6199c7700 1 -- 172.21.15.90:0/299724575 >> 172.21.15.90:0/299724575 conn(0x7ff6140fd9a0 msgr2=0x7ff6140ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:38:25.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:38:25.867+0000 7ff6199c7700 1 -- 172.21.15.90:0/299724575 shutdown_connections 2024-05-03T14:38:25.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:38:25.867+0000 7ff6199c7700 1 -- 172.21.15.90:0/299724575 wait complete. 2024-05-03T14:38:25.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:25 smithi149 ceph-mon[33332]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T14:38:25.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:25 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:25.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:25 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:25.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:25 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:38:25.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:25 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:38:25.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:25 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:26.525 DEBUG:teuthology.orchestra.run.smithi090:osd.1> sudo journalctl -f -n 0 -u ceph-285d64b2-095a-11ef-bc97-c7b262605968@osd.1.service 2024-05-03T14:38:26.527 INFO:tasks.cephadm:Deploying osd.2 on smithi149 with /dev/vg_nvme/lv_4... 2024-05-03T14:38:26.527 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-03T14:38:26.771 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T14:38:27.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:26 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:27.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:26 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:27.126 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:26 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:38:27.126 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:26 smithi090 ceph-mon[33568]: from='osd.1 [v2:172.21.15.90:6800/720063242,v1:172.21.15.90:6801/720063242]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-03T14:38:27.126 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:26 smithi090 ceph-mon[33568]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-03T14:38:27.126 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:26 smithi090 ceph-mon[33568]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T14:38:27.139 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:26 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:27.139 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:26 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:27.139 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:26 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:38:27.139 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:26 smithi149 ceph-mon[33332]: from='osd.1 [v2:172.21.15.90:6800/720063242,v1:172.21.15.90:6801/720063242]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-03T14:38:27.139 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:26 smithi149 ceph-mon[33332]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-03T14:38:27.139 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:26 smithi149 ceph-mon[33332]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T14:38:27.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:26 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:27.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:26 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:27.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:26 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:38:27.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:26 smithi060 ceph-mon[28873]: from='osd.1 [v2:172.21.15.90:6800/720063242,v1:172.21.15.90:6801/720063242]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-03T14:38:27.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:26 smithi060 ceph-mon[28873]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-03T14:38:27.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:26 smithi060 ceph-mon[28873]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T14:38:28.096 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:27 smithi149 ceph-mon[33332]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-03T14:38:28.097 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:27 smithi149 ceph-mon[33332]: osdmap e11: 2 total, 1 up, 2 in 2024-05-03T14:38:28.097 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:27 smithi149 ceph-mon[33332]: from='osd.1 [v2:172.21.15.90:6800/720063242,v1:172.21.15.90:6801/720063242]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-05-03T14:38:28.097 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:27 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T14:38:28.097 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:27 smithi149 ceph-mon[33332]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-05-03T14:38:28.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:27 smithi090 ceph-mon[33568]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-03T14:38:28.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:27 smithi090 ceph-mon[33568]: osdmap e11: 2 total, 1 up, 2 in 2024-05-03T14:38:28.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:27 smithi090 ceph-mon[33568]: from='osd.1 [v2:172.21.15.90:6800/720063242,v1:172.21.15.90:6801/720063242]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-05-03T14:38:28.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:27 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T14:38:28.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:27 smithi090 ceph-mon[33568]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-05-03T14:38:28.126 INFO:journalctl@ceph.osd.1.smithi090.stdout:May 03 14:38:27 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-1[38156]: 2024-05-03T14:38:27.876+0000 7f2b25324700 -1 osd.1 0 waiting for initial osdmap 2024-05-03T14:38:28.126 INFO:journalctl@ceph.osd.1.smithi090.stdout:May 03 14:38:27 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-1[38156]: 2024-05-03T14:38:27.884+0000 7f2b21117700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-03T14:38:28.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:27 smithi060 ceph-mon[28873]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-03T14:38:28.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:27 smithi060 ceph-mon[28873]: osdmap e11: 2 total, 1 up, 2 in 2024-05-03T14:38:28.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:27 smithi060 ceph-mon[28873]: from='osd.1 [v2:172.21.15.90:6800/720063242,v1:172.21.15.90:6801/720063242]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-05-03T14:38:28.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:27 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T14:38:28.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:27 smithi060 ceph-mon[28873]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-05-03T14:38:28.669 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-03T14:38:28.670 INFO:teuthology.orchestra.run.smithi149.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-03T14:38:28.670 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10+0 records in 2024-05-03T14:38:28.670 INFO:teuthology.orchestra.run.smithi149.stderr:10+0 records out 2024-05-03T14:38:28.670 INFO:teuthology.orchestra.run.smithi149.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112998 s, 928 MB/s 2024-05-03T14:38:28.670 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 2024-05-03T14:38:28.670 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping successful for: 2024-05-03T14:38:28.973 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:28 smithi149 ceph-mon[33332]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-05-03T14:38:28.973 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:28 smithi149 ceph-mon[33332]: osdmap e12: 2 total, 1 up, 2 in 2024-05-03T14:38:28.973 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:28 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T14:38:28.973 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:28 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T14:38:28.973 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:28 smithi149 ceph-mon[33332]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T14:38:29.225 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi149:vg_nvme/lv_4 2024-05-03T14:38:29.243 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:28 smithi090 ceph-mon[33568]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-05-03T14:38:29.243 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:28 smithi090 ceph-mon[33568]: osdmap e12: 2 total, 1 up, 2 in 2024-05-03T14:38:29.243 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:28 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T14:38:29.244 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:28 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T14:38:29.244 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:28 smithi090 ceph-mon[33568]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T14:38:29.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:28 smithi060 ceph-mon[28873]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-05-03T14:38:29.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:28 smithi060 ceph-mon[28873]: osdmap e12: 2 total, 1 up, 2 in 2024-05-03T14:38:29.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:28 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T14:38:29.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:28 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T14:38:29.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:28 smithi060 ceph-mon[28873]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T14:38:29.470 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T14:38:30.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:29 smithi149 ceph-mon[33332]: purged_snaps scrub starts 2024-05-03T14:38:30.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:29 smithi149 ceph-mon[33332]: purged_snaps scrub ok 2024-05-03T14:38:30.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:29 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T14:38:30.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:29 smithi149 ceph-mon[33332]: osd.1 [v2:172.21.15.90:6800/720063242,v1:172.21.15.90:6801/720063242] boot 2024-05-03T14:38:30.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:29 smithi149 ceph-mon[33332]: osdmap e13: 2 total, 2 up, 2 in 2024-05-03T14:38:30.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:29 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T14:38:30.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:29 smithi149 ceph-mon[33332]: Detected new or changed devices on smithi090 2024-05-03T14:38:30.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:29 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:30.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:29 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:30.175 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:29 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:38:30.175 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:29 smithi149 ceph-mon[33332]: Adjusting osd_memory_target on smithi090 to 17088M 2024-05-03T14:38:30.175 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:29 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:30.175 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:29 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:38:30.175 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:29 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:38:30.175 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:29 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:30.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:29 smithi060 ceph-mon[28873]: purged_snaps scrub starts 2024-05-03T14:38:30.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:29 smithi060 ceph-mon[28873]: purged_snaps scrub ok 2024-05-03T14:38:30.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:29 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T14:38:30.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:29 smithi060 ceph-mon[28873]: osd.1 [v2:172.21.15.90:6800/720063242,v1:172.21.15.90:6801/720063242] boot 2024-05-03T14:38:30.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:29 smithi060 ceph-mon[28873]: osdmap e13: 2 total, 2 up, 2 in 2024-05-03T14:38:30.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:29 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T14:38:30.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:29 smithi060 ceph-mon[28873]: Detected new or changed devices on smithi090 2024-05-03T14:38:30.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:29 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:30.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:29 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:30.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:29 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:38:30.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:29 smithi060 ceph-mon[28873]: Adjusting osd_memory_target on smithi090 to 17088M 2024-05-03T14:38:30.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:29 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:30.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:29 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:38:30.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:29 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:38:30.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:29 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:30.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:29 smithi090 ceph-mon[33568]: purged_snaps scrub starts 2024-05-03T14:38:30.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:29 smithi090 ceph-mon[33568]: purged_snaps scrub ok 2024-05-03T14:38:30.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:29 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T14:38:30.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:29 smithi090 ceph-mon[33568]: osd.1 [v2:172.21.15.90:6800/720063242,v1:172.21.15.90:6801/720063242] boot 2024-05-03T14:38:30.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:29 smithi090 ceph-mon[33568]: osdmap e13: 2 total, 2 up, 2 in 2024-05-03T14:38:30.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:29 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T14:38:30.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:29 smithi090 ceph-mon[33568]: Detected new or changed devices on smithi090 2024-05-03T14:38:30.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:29 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:30.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:29 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:30.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:29 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:38:30.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:29 smithi090 ceph-mon[33568]: Adjusting osd_memory_target on smithi090 to 17088M 2024-05-03T14:38:30.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:29 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:30.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:29 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:38:30.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:29 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:38:30.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:29 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:31.063 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:38:31.057+0000 7f93b33ae700 1 -- 172.21.15.149:0/740281382 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93ac1051f0 msgr2=0x7f93ac10baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:38:31.063 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:38:31.057+0000 7f93b33ae700 1 --2- 172.21.15.149:0/740281382 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93ac1051f0 0x7f93ac10baa0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f93a400b0a0 tx=0x7f93a400b3b0 comp rx=0 tx=0).stop 2024-05-03T14:38:31.063 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:38:31.058+0000 7f93b33ae700 1 -- 172.21.15.149:0/740281382 shutdown_connections 2024-05-03T14:38:31.064 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:38:31.058+0000 7f93b33ae700 1 --2- 172.21.15.149:0/740281382 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93ac1051f0 0x7f93ac10baa0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:38:31.064 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:38:31.058+0000 7f93b33ae700 1 --2- 172.21.15.149:0/740281382 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f93ac1017d0 0x7f93ac101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:38:31.064 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:38:31.058+0000 7f93b33ae700 1 --2- 172.21.15.149:0/740281382 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93ac104850 0x7f93ac104cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:38:31.064 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:38:31.058+0000 7f93b33ae700 1 -- 172.21.15.149:0/740281382 >> 172.21.15.149:0/740281382 conn(0x7f93ac0fd110 msgr2=0x7f93ac0ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:38:31.064 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:38:31.058+0000 7f93b33ae700 1 -- 172.21.15.149:0/740281382 shutdown_connections 2024-05-03T14:38:31.064 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:38:31.058+0000 7f93b33ae700 1 -- 172.21.15.149:0/740281382 wait complete. 2024-05-03T14:38:31.064 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:38:31.058+0000 7f93b33ae700 1 Processor -- start 2024-05-03T14:38:31.064 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:38:31.058+0000 7f93b33ae700 1 -- start start 2024-05-03T14:38:31.064 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:38:31.059+0000 7f93b33ae700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93ac1017d0 0x7f93ac199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:38:31.064 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:38:31.059+0000 7f93b33ae700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f93ac104850 0x7f93ac199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:38:31.064 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:38:31.059+0000 7f93b33ae700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93ac1051f0 0x7f93ac1a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:38:31.064 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:38:31.059+0000 7f93b33ae700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f93ac10f2f0 con 0x7f93ac1017d0 2024-05-03T14:38:31.064 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:38:31.059+0000 7f93b33ae700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f93ac10f170 con 0x7f93ac104850 2024-05-03T14:38:31.064 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:38:31.059+0000 7f93b33ae700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f93ac10f470 con 0x7f93ac1051f0 2024-05-03T14:38:31.065 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:38:31.059+0000 7f93b194b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93ac1051f0 0x7f93ac1a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:38:31.065 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:38:31.059+0000 7f93b194b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93ac1051f0 0x7f93ac1a0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:38096/0 (socket says 172.21.15.149:38096) 2024-05-03T14:38:31.065 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:38:31.059+0000 7f93b194b700 1 -- 172.21.15.149:0/3874692245 learned_addr learned my addr 172.21.15.149:0/3874692245 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-05-03T14:38:31.065 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:38:31.059+0000 7f93b194b700 1 -- 172.21.15.149:0/3874692245 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f93ac104850 msgr2=0x7f93ac199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:38:31.065 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:38:31.059+0000 7f93b194b700 1 --2- 172.21.15.149:0/3874692245 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f93ac104850 0x7f93ac199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:38:31.065 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:38:31.059+0000 7f93b194b700 1 -- 172.21.15.149:0/3874692245 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93ac1017d0 msgr2=0x7f93ac199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:38:31.065 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:38:31.059+0000 7f93b194b700 1 --2- 172.21.15.149:0/3874692245 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93ac1017d0 0x7f93ac199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:38:31.065 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:38:31.059+0000 7f93b194b700 1 -- 172.21.15.149:0/3874692245 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93ac1a1470 con 0x7f93ac1051f0 2024-05-03T14:38:31.065 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:38:31.059+0000 7f93b194b700 1 --2- 172.21.15.149:0/3874692245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93ac1051f0 0x7f93ac1a0dd0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f93a4006070 tx=0x7f93a4007dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:38:31.065 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:38:31.059+0000 7f93a27fc700 1 -- 172.21.15.149:0/3874692245 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a4003e10 con 0x7f93ac1051f0 2024-05-03T14:38:31.065 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:38:31.059+0000 7f93a27fc700 1 -- 172.21.15.149:0/3874692245 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f93a4005570 con 0x7f93ac1051f0 2024-05-03T14:38:31.065 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:38:31.059+0000 7f93a27fc700 1 -- 172.21.15.149:0/3874692245 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a401bc50 con 0x7f93ac1051f0 2024-05-03T14:38:31.066 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:38:31.059+0000 7f93b33ae700 1 -- 172.21.15.149:0/3874692245 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93ac1a1700 con 0x7f93ac1051f0 2024-05-03T14:38:31.066 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:38:31.060+0000 7f93b33ae700 1 -- 172.21.15.149:0/3874692245 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93ac1a1b80 con 0x7f93ac1051f0 2024-05-03T14:38:31.066 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:38:31.060+0000 7f93b33ae700 1 -- 172.21.15.149:0/3874692245 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93ac02e510 con 0x7f93ac1051f0 2024-05-03T14:38:31.066 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:38:31.063+0000 7f93a27fc700 1 -- 172.21.15.149:0/3874692245 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7f93a4005720 con 0x7f93ac1051f0 2024-05-03T14:38:31.067 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:38:31.063+0000 7f93a27fc700 1 --2- 172.21.15.149:0/3874692245 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f93980b2870 0x7f93980b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:38:31.067 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:38:31.063+0000 7f93a27fc700 1 -- 172.21.15.149:0/3874692245 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f93a400f020 con 0x7f93ac1051f0 2024-05-03T14:38:31.067 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:38:31.064+0000 7f93b114a700 1 --2- 172.21.15.149:0/3874692245 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f93980b2870 0x7f93980b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:38:31.067 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:38:31.065+0000 7f93b114a700 1 --2- 172.21.15.149:0/3874692245 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f93980b2870 0x7f93980b4d30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f93ac06b9b0 tx=0x7f93a8005ca0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:38:31.070 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:38:31.067+0000 7f93a27fc700 1 -- 172.21.15.149:0/3874692245 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f93a40ab6a0 con 0x7f93ac1051f0 2024-05-03T14:38:31.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:30 smithi149 ceph-mon[33332]: osdmap e14: 2 total, 2 up, 2 in 2024-05-03T14:38:31.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:30 smithi149 ceph-mon[33332]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-03T14:38:31.282 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:38:31.279+0000 7f93b33ae700 1 -- 172.21.15.149:0/3874692245 --> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f93ac0638e0 con 0x7f93980b2870 2024-05-03T14:38:31.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:30 smithi060 ceph-mon[28873]: osdmap e14: 2 total, 2 up, 2 in 2024-05-03T14:38:31.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:30 smithi060 ceph-mon[28873]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-03T14:38:31.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:30 smithi090 ceph-mon[33568]: osdmap e14: 2 total, 2 up, 2 in 2024-05-03T14:38:31.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:30 smithi090 ceph-mon[33568]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-03T14:38:32.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:31 smithi090 ceph-mon[33568]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:38:32.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:31 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T14:38:32.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:31 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T14:38:32.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:31 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:38:32.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:31 smithi149 ceph-mon[33332]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:38:32.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:31 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T14:38:32.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:31 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T14:38:32.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:31 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:38:32.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:31 smithi060 ceph-mon[28873]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:38:32.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:31 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T14:38:32.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:31 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T14:38:32.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:31 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:38:33.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:32 smithi149 ceph-mon[33332]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-03T14:38:33.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:32 smithi060 ceph-mon[28873]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-03T14:38:33.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:32 smithi090 ceph-mon[33568]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-03T14:38:34.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:34 smithi060 ceph-mon[28873]: from='client.? 172.21.15.149:0/3802290831' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ce95fb7c-1611-4dd1-aee7-97c420d6a466"}]: dispatch 2024-05-03T14:38:34.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:34 smithi060 ceph-mon[28873]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ce95fb7c-1611-4dd1-aee7-97c420d6a466"}]: dispatch 2024-05-03T14:38:34.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:34 smithi060 ceph-mon[28873]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ce95fb7c-1611-4dd1-aee7-97c420d6a466"}]': finished 2024-05-03T14:38:34.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:34 smithi060 ceph-mon[28873]: osdmap e15: 3 total, 2 up, 3 in 2024-05-03T14:38:34.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:34 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T14:38:34.344 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:34 smithi149 ceph-mon[33332]: from='client.? 172.21.15.149:0/3802290831' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ce95fb7c-1611-4dd1-aee7-97c420d6a466"}]: dispatch 2024-05-03T14:38:34.344 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:34 smithi149 ceph-mon[33332]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ce95fb7c-1611-4dd1-aee7-97c420d6a466"}]: dispatch 2024-05-03T14:38:34.344 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:34 smithi149 ceph-mon[33332]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ce95fb7c-1611-4dd1-aee7-97c420d6a466"}]': finished 2024-05-03T14:38:34.344 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:34 smithi149 ceph-mon[33332]: osdmap e15: 3 total, 2 up, 3 in 2024-05-03T14:38:34.344 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:34 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T14:38:34.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:34 smithi090 ceph-mon[33568]: from='client.? 172.21.15.149:0/3802290831' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ce95fb7c-1611-4dd1-aee7-97c420d6a466"}]: dispatch 2024-05-03T14:38:34.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:34 smithi090 ceph-mon[33568]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ce95fb7c-1611-4dd1-aee7-97c420d6a466"}]: dispatch 2024-05-03T14:38:34.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:34 smithi090 ceph-mon[33568]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ce95fb7c-1611-4dd1-aee7-97c420d6a466"}]': finished 2024-05-03T14:38:34.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:34 smithi090 ceph-mon[33568]: osdmap e15: 3 total, 2 up, 3 in 2024-05-03T14:38:34.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:34 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T14:38:35.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:35 smithi060 ceph-mon[28873]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-03T14:38:35.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:35 smithi060 ceph-mon[28873]: from='client.? 172.21.15.149:0/3261714823' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T14:38:35.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:35 smithi090 ceph-mon[33568]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-03T14:38:35.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:35 smithi090 ceph-mon[33568]: from='client.? 172.21.15.149:0/3261714823' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T14:38:35.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:35 smithi149 ceph-mon[33332]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-03T14:38:35.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:35 smithi149 ceph-mon[33332]: from='client.? 172.21.15.149:0/3261714823' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T14:38:37.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:37 smithi060 ceph-mon[28873]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-03T14:38:37.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:37 smithi090 ceph-mon[33568]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-03T14:38:37.630 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:37 smithi149 ceph-mon[33332]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-03T14:38:39.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:39 smithi149 ceph-mon[33332]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-03T14:38:39.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:39 smithi060 ceph-mon[28873]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-03T14:38:39.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:39 smithi090 ceph-mon[33568]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-03T14:38:41.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:41 smithi060 ceph-mon[28873]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-03T14:38:41.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:41 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-03T14:38:41.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:41 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:38:41.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:41 smithi090 ceph-mon[33568]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-03T14:38:41.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:41 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-03T14:38:41.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:41 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:38:41.642 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:41 smithi149 ceph-mon[33332]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-03T14:38:41.643 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:41 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-03T14:38:41.643 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:41 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:38:42.556 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:42 smithi149 ceph-mon[33332]: Deploying daemon osd.2 on smithi149 2024-05-03T14:38:42.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:42 smithi060 ceph-mon[28873]: Deploying daemon osd.2 on smithi149 2024-05-03T14:38:42.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:42 smithi090 ceph-mon[33568]: Deploying daemon osd.2 on smithi149 2024-05-03T14:38:43.520 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:43 smithi149 ceph-mon[33332]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-03T14:38:43.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:43 smithi060 ceph-mon[28873]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-03T14:38:43.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:43 smithi090 ceph-mon[33568]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-03T14:38:45.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:45 smithi149 ceph-mon[33332]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-03T14:38:45.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:45 smithi060 ceph-mon[28873]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-03T14:38:45.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:45 smithi090 ceph-mon[33568]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-03T14:38:47.634 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:47 smithi149 ceph-mon[33332]: pgmap v86: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-03T14:38:47.634 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:47 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:47.635 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:47 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:38:47.635 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:47 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:47.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:47 smithi060 ceph-mon[28873]: pgmap v86: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-03T14:38:47.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:47 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:47.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:47 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:38:47.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:47 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:47.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:47 smithi090 ceph-mon[33568]: pgmap v86: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-03T14:38:47.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:47 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:47.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:47 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:38:47.876 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:47 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:49.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:49 smithi149 ceph-mon[33332]: pgmap v87: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-03T14:38:49.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:49 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:49.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:49 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:49.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:49 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:38:49.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:49 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:38:49.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:49 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:49.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:49 smithi060 ceph-mon[28873]: pgmap v87: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-03T14:38:49.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:49 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:49.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:49 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:49.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:49 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:38:49.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:49 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:38:49.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:49 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:49.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:49 smithi090 ceph-mon[33568]: pgmap v87: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-03T14:38:49.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:49 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:49.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:49 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:49.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:49 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:38:49.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:49 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:38:49.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:49 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:50.082 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:38:50.079+0000 7f93a27fc700 1 -- 172.21.15.149:0/3874692245 <== mgr.14152 v2:172.21.15.60:6800/3066142956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f93ac0638e0 con 0x7f93980b2870 2024-05-03T14:38:50.082 INFO:teuthology.orchestra.run.smithi149.stdout:Created osd(s) 2 on host 'smithi149' 2024-05-03T14:38:50.084 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:38:50.082+0000 7f93b33ae700 1 -- 172.21.15.149:0/3874692245 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f93980b2870 msgr2=0x7f93980b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:38:50.085 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:38:50.082+0000 7f93b33ae700 1 --2- 172.21.15.149:0/3874692245 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f93980b2870 0x7f93980b4d30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f93ac06b9b0 tx=0x7f93a8005ca0 comp rx=0 tx=0).stop 2024-05-03T14:38:50.085 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:38:50.082+0000 7f93b33ae700 1 -- 172.21.15.149:0/3874692245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93ac1051f0 msgr2=0x7f93ac1a0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:38:50.085 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:38:50.082+0000 7f93b33ae700 1 --2- 172.21.15.149:0/3874692245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93ac1051f0 0x7f93ac1a0dd0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f93a4006070 tx=0x7f93a4007dc0 comp rx=0 tx=0).stop 2024-05-03T14:38:50.085 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:38:50.082+0000 7f93b33ae700 1 -- 172.21.15.149:0/3874692245 shutdown_connections 2024-05-03T14:38:50.085 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:38:50.082+0000 7f93b33ae700 1 --2- 172.21.15.149:0/3874692245 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f93980b2870 0x7f93980b4d30 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:38:50.085 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:38:50.082+0000 7f93b33ae700 1 --2- 172.21.15.149:0/3874692245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93ac1051f0 0x7f93ac1a0dd0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:38:50.085 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:38:50.082+0000 7f93b33ae700 1 --2- 172.21.15.149:0/3874692245 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93ac1017d0 0x7f93ac199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:38:50.085 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:38:50.082+0000 7f93b33ae700 1 --2- 172.21.15.149:0/3874692245 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f93ac104850 0x7f93ac199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:38:50.085 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:38:50.082+0000 7f93b33ae700 1 -- 172.21.15.149:0/3874692245 >> 172.21.15.149:0/3874692245 conn(0x7f93ac0fd110 msgr2=0x7f93ac107a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:38:50.085 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:38:50.082+0000 7f93b33ae700 1 -- 172.21.15.149:0/3874692245 shutdown_connections 2024-05-03T14:38:50.085 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:38:50.082+0000 7f93b33ae700 1 -- 172.21.15.149:0/3874692245 wait complete. 2024-05-03T14:38:50.703 DEBUG:teuthology.orchestra.run.smithi149:osd.2> sudo journalctl -f -n 0 -u ceph-285d64b2-095a-11ef-bc97-c7b262605968@osd.2.service 2024-05-03T14:38:50.706 INFO:tasks.cephadm:Deploying osd.3 on smithi060 with /dev/vg_nvme/lv_3... 2024-05-03T14:38:50.707 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-03T14:38:51.156 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:38:51.306 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:51 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:51.306 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:51 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:51.306 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:51 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:38:51.306 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:51 smithi060 ceph-mon[28873]: pgmap v88: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-03T14:38:51.306 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:51 smithi060 ceph-mon[28873]: from='osd.2 [v2:172.21.15.149:6800/1359662077,v1:172.21.15.149:6801/1359662077]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-03T14:38:51.306 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:51 smithi060 ceph-mon[28873]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-03T14:38:51.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:51 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:51.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:51 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:51.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:51 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:38:51.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:51 smithi090 ceph-mon[33568]: pgmap v88: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-03T14:38:51.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:51 smithi090 ceph-mon[33568]: from='osd.2 [v2:172.21.15.149:6800/1359662077,v1:172.21.15.149:6801/1359662077]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-03T14:38:51.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:51 smithi090 ceph-mon[33568]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-03T14:38:51.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:51 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:51.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:51 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:51.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:51 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:38:51.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:51 smithi149 ceph-mon[33332]: pgmap v88: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-03T14:38:51.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:51 smithi149 ceph-mon[33332]: from='osd.2 [v2:172.21.15.149:6800/1359662077,v1:172.21.15.149:6801/1359662077]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-03T14:38:51.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:51 smithi149 ceph-mon[33332]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-03T14:38:52.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:52 smithi060 ceph-mon[28873]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-03T14:38:52.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:52 smithi060 ceph-mon[28873]: from='osd.2 [v2:172.21.15.149:6800/1359662077,v1:172.21.15.149:6801/1359662077]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-03T14:38:52.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:52 smithi060 ceph-mon[28873]: osdmap e16: 3 total, 2 up, 3 in 2024-05-03T14:38:52.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:52 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T14:38:52.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:52 smithi060 ceph-mon[28873]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-03T14:38:52.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:52 smithi090 ceph-mon[33568]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-03T14:38:52.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:52 smithi090 ceph-mon[33568]: from='osd.2 [v2:172.21.15.149:6800/1359662077,v1:172.21.15.149:6801/1359662077]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-03T14:38:52.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:52 smithi090 ceph-mon[33568]: osdmap e16: 3 total, 2 up, 3 in 2024-05-03T14:38:52.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:52 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T14:38:52.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:52 smithi090 ceph-mon[33568]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-03T14:38:52.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:52 smithi149 ceph-mon[33332]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-03T14:38:52.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:52 smithi149 ceph-mon[33332]: from='osd.2 [v2:172.21.15.149:6800/1359662077,v1:172.21.15.149:6801/1359662077]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-03T14:38:52.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:52 smithi149 ceph-mon[33332]: osdmap e16: 3 total, 2 up, 3 in 2024-05-03T14:38:52.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:52 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T14:38:52.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:52 smithi149 ceph-mon[33332]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-03T14:38:52.924 INFO:journalctl@ceph.osd.2.smithi149.stdout:May 03 14:38:52 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-2[38497]: 2024-05-03T14:38:52.527+0000 7f0fc8fa9700 -1 osd.2 0 waiting for initial osdmap 2024-05-03T14:38:52.924 INFO:journalctl@ceph.osd.2.smithi149.stdout:May 03 14:38:52 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-2[38497]: 2024-05-03T14:38:52.531+0000 7f0fc0593700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-03T14:38:53.304 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:53 smithi060 ceph-mon[28873]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-05-03T14:38:53.304 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:53 smithi060 ceph-mon[28873]: osdmap e17: 3 total, 2 up, 3 in 2024-05-03T14:38:53.304 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:53 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T14:38:53.304 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:53 smithi060 ceph-mon[28873]: pgmap v91: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-03T14:38:53.304 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:53 smithi060 ceph-mon[28873]: from='osd.2 ' entity='osd.2' 2024-05-03T14:38:53.304 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:53 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T14:38:53.332 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:53 smithi149 ceph-mon[33332]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-05-03T14:38:53.333 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:53 smithi149 ceph-mon[33332]: osdmap e17: 3 total, 2 up, 3 in 2024-05-03T14:38:53.333 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:53 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T14:38:53.333 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:53 smithi149 ceph-mon[33332]: pgmap v91: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-03T14:38:53.333 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:53 smithi149 ceph-mon[33332]: from='osd.2 ' entity='osd.2' 2024-05-03T14:38:53.333 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:53 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T14:38:53.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:53 smithi090 ceph-mon[33568]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-05-03T14:38:53.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:53 smithi090 ceph-mon[33568]: osdmap e17: 3 total, 2 up, 3 in 2024-05-03T14:38:53.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:53 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T14:38:53.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:53 smithi090 ceph-mon[33568]: pgmap v91: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-03T14:38:53.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:53 smithi090 ceph-mon[33568]: from='osd.2 ' entity='osd.2' 2024-05-03T14:38:53.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:53 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T14:38:54.075 INFO:teuthology.orchestra.run.smithi060.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-03T14:38:54.075 INFO:teuthology.orchestra.run.smithi060.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-03T14:38:54.075 INFO:teuthology.orchestra.run.smithi060.stderr: stderr: 10+0 records in 2024-05-03T14:38:54.075 INFO:teuthology.orchestra.run.smithi060.stderr:10+0 records out 2024-05-03T14:38:54.075 INFO:teuthology.orchestra.run.smithi060.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0359963 s, 291 MB/s 2024-05-03T14:38:54.075 INFO:teuthology.orchestra.run.smithi060.stderr:--> Zapping successful for: 2024-05-03T14:38:54.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:54 smithi060 ceph-mon[28873]: purged_snaps scrub starts 2024-05-03T14:38:54.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:54 smithi060 ceph-mon[28873]: purged_snaps scrub ok 2024-05-03T14:38:54.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:54 smithi060 ceph-mon[28873]: osd.2 [v2:172.21.15.149:6800/1359662077,v1:172.21.15.149:6801/1359662077] boot 2024-05-03T14:38:54.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:54 smithi060 ceph-mon[28873]: osdmap e18: 3 total, 3 up, 3 in 2024-05-03T14:38:54.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:54 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T14:38:54.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:54 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:54.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:54 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:54.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:54 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:38:54.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:54 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:54.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:54 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:38:54.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:54 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:38:54.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:54 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:54.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:54 smithi090 ceph-mon[33568]: purged_snaps scrub starts 2024-05-03T14:38:54.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:54 smithi090 ceph-mon[33568]: purged_snaps scrub ok 2024-05-03T14:38:54.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:54 smithi090 ceph-mon[33568]: osd.2 [v2:172.21.15.149:6800/1359662077,v1:172.21.15.149:6801/1359662077] boot 2024-05-03T14:38:54.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:54 smithi090 ceph-mon[33568]: osdmap e18: 3 total, 3 up, 3 in 2024-05-03T14:38:54.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:54 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T14:38:54.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:54 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:54.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:54 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:54.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:54 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:38:54.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:54 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:54.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:54 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:38:54.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:54 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:38:54.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:54 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:54.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:54 smithi149 ceph-mon[33332]: purged_snaps scrub starts 2024-05-03T14:38:54.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:54 smithi149 ceph-mon[33332]: purged_snaps scrub ok 2024-05-03T14:38:54.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:54 smithi149 ceph-mon[33332]: osd.2 [v2:172.21.15.149:6800/1359662077,v1:172.21.15.149:6801/1359662077] boot 2024-05-03T14:38:54.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:54 smithi149 ceph-mon[33332]: osdmap e18: 3 total, 3 up, 3 in 2024-05-03T14:38:54.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:54 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T14:38:54.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:54 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:54.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:54 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:54.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:54 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:38:54.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:54 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:54.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:54 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:38:54.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:54 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:38:54.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:54 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:54.893 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi060:vg_nvme/lv_3 2024-05-03T14:38:55.335 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:38:55.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:55 smithi090 ceph-mon[33568]: Detected new or changed devices on smithi149 2024-05-03T14:38:55.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:55 smithi090 ceph-mon[33568]: Adjusting osd_memory_target on smithi149 to 5816M 2024-05-03T14:38:55.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:55 smithi090 ceph-mon[33568]: pgmap v93: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-03T14:38:55.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:55 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-03T14:38:55.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:55 smithi149 ceph-mon[33332]: Detected new or changed devices on smithi149 2024-05-03T14:38:55.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:55 smithi149 ceph-mon[33332]: Adjusting osd_memory_target on smithi149 to 5816M 2024-05-03T14:38:55.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:55 smithi149 ceph-mon[33332]: pgmap v93: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-03T14:38:55.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:55 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-03T14:38:55.481 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:55 smithi060 ceph-mon[28873]: Detected new or changed devices on smithi149 2024-05-03T14:38:55.481 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:55 smithi060 ceph-mon[28873]: Adjusting osd_memory_target on smithi149 to 5816M 2024-05-03T14:38:55.481 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:55 smithi060 ceph-mon[28873]: pgmap v93: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-03T14:38:55.481 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:55 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-03T14:38:56.250 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:56 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-03T14:38:56.251 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:56 smithi060 ceph-mon[28873]: osdmap e19: 3 total, 3 up, 3 in 2024-05-03T14:38:56.251 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:56 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-03T14:38:56.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:56 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-03T14:38:56.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:56 smithi090 ceph-mon[33568]: osdmap e19: 3 total, 3 up, 3 in 2024-05-03T14:38:56.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:56 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-03T14:38:56.375 INFO:journalctl@ceph.osd.1.smithi090.stdout:May 03 14:38:56 smithi090 sudo[42389]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-03T14:38:56.375 INFO:journalctl@ceph.osd.1.smithi090.stdout:May 03 14:38:56 smithi090 sudo[42389]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-03T14:38:56.376 INFO:journalctl@ceph.osd.1.smithi090.stdout:May 03 14:38:56 smithi090 sudo[42389]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-03T14:38:56.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:56 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-03T14:38:56.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:56 smithi149 ceph-mon[33332]: osdmap e19: 3 total, 3 up, 3 in 2024-05-03T14:38:56.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:56 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-03T14:38:56.505 INFO:journalctl@ceph.osd.0.smithi060.stdout:May 03 14:38:56 smithi060 sudo[45106]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-03T14:38:56.505 INFO:journalctl@ceph.osd.0.smithi060.stdout:May 03 14:38:56 smithi060 sudo[45106]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-03T14:38:56.505 INFO:journalctl@ceph.osd.0.smithi060.stdout:May 03 14:38:56 smithi060 sudo[45106]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-03T14:38:56.505 INFO:journalctl@ceph.osd.0.smithi060.stdout:May 03 14:38:56 smithi060 sudo[45106]: pam_unix(sudo:session): session closed for user root 2024-05-03T14:38:56.505 INFO:journalctl@ceph.osd.0.smithi060.stdout:May 03 14:38:56 smithi060 sudo[45109]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-03T14:38:56.505 INFO:journalctl@ceph.osd.0.smithi060.stdout:May 03 14:38:56 smithi060 sudo[45109]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-03T14:38:56.505 INFO:journalctl@ceph.osd.0.smithi060.stdout:May 03 14:38:56 smithi060 sudo[45109]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-03T14:38:56.505 INFO:journalctl@ceph.osd.0.smithi060.stdout:May 03 14:38:56 smithi060 sudo[45109]: pam_unix(sudo:session): session closed for user root 2024-05-03T14:38:56.677 INFO:journalctl@ceph.osd.1.smithi090.stdout:May 03 14:38:56 smithi090 sudo[42389]: pam_unix(sudo:session): session closed for user root 2024-05-03T14:38:56.677 INFO:journalctl@ceph.osd.1.smithi090.stdout:May 03 14:38:56 smithi090 sudo[42392]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-03T14:38:56.677 INFO:journalctl@ceph.osd.1.smithi090.stdout:May 03 14:38:56 smithi090 sudo[42392]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-03T14:38:56.677 INFO:journalctl@ceph.osd.1.smithi090.stdout:May 03 14:38:56 smithi090 sudo[42392]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-03T14:38:56.677 INFO:journalctl@ceph.osd.1.smithi090.stdout:May 03 14:38:56 smithi090 sudo[42392]: pam_unix(sudo:session): session closed for user root 2024-05-03T14:38:56.832 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:56 smithi060 sudo[45115]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-03T14:38:56.832 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:56 smithi060 sudo[45115]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-03T14:38:56.832 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:56 smithi060 sudo[45115]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-03T14:38:56.832 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:56 smithi060 sudo[45115]: pam_unix(sudo:session): session closed for user root 2024-05-03T14:38:56.832 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:56 smithi060 sudo[45118]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-03T14:38:56.832 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:56 smithi060 sudo[45118]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-03T14:38:56.832 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:56 smithi060 sudo[45118]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-03T14:38:56.832 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:56 smithi060 sudo[45118]: pam_unix(sudo:session): session closed for user root 2024-05-03T14:38:56.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:56 smithi149 sudo[42574]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-03T14:38:56.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:56 smithi149 sudo[42574]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-03T14:38:56.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:56 smithi149 sudo[42574]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-03T14:38:56.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:56 smithi149 sudo[42574]: pam_unix(sudo:session): session closed for user root 2024-05-03T14:38:56.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:56 smithi149 sudo[42577]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-03T14:38:56.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:56 smithi149 sudo[42577]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-03T14:38:56.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:56 smithi149 sudo[42577]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-03T14:38:56.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:56 smithi149 sudo[42577]: pam_unix(sudo:session): session closed for user root 2024-05-03T14:38:56.924 INFO:journalctl@ceph.osd.2.smithi149.stdout:May 03 14:38:56 smithi149 sudo[42568]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-03T14:38:56.924 INFO:journalctl@ceph.osd.2.smithi149.stdout:May 03 14:38:56 smithi149 sudo[42568]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-03T14:38:56.925 INFO:journalctl@ceph.osd.2.smithi149.stdout:May 03 14:38:56 smithi149 sudo[42568]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-03T14:38:56.925 INFO:journalctl@ceph.osd.2.smithi149.stdout:May 03 14:38:56 smithi149 sudo[42568]: pam_unix(sudo:session): session closed for user root 2024-05-03T14:38:56.925 INFO:journalctl@ceph.osd.2.smithi149.stdout:May 03 14:38:56 smithi149 sudo[42571]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-03T14:38:56.925 INFO:journalctl@ceph.osd.2.smithi149.stdout:May 03 14:38:56 smithi149 sudo[42571]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-03T14:38:56.925 INFO:journalctl@ceph.osd.2.smithi149.stdout:May 03 14:38:56 smithi149 sudo[42571]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-03T14:38:56.925 INFO:journalctl@ceph.osd.2.smithi149.stdout:May 03 14:38:56 smithi149 sudo[42571]: pam_unix(sudo:session): session closed for user root 2024-05-03T14:38:57.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:56 smithi090 sudo[42398]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-03T14:38:57.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:56 smithi090 sudo[42398]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-03T14:38:57.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:56 smithi090 sudo[42398]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-03T14:38:57.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:56 smithi090 sudo[42398]: pam_unix(sudo:session): session closed for user root 2024-05-03T14:38:57.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:56 smithi090 sudo[42401]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-03T14:38:57.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:56 smithi090 sudo[42401]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-03T14:38:57.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:56 smithi090 sudo[42401]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-03T14:38:57.126 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:56 smithi090 sudo[42401]: pam_unix(sudo:session): session closed for user root 2024-05-03T14:38:57.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:57 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-03T14:38:57.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:57 smithi149 ceph-mon[33332]: osdmap e20: 3 total, 3 up, 3 in 2024-05-03T14:38:57.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:57 smithi149 ceph-mon[33332]: pgmap v96: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-03T14:38:57.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:57 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:57.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:57 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:57.425 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:57 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:57.425 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:57 smithi149 ceph-mon[33332]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-03T14:38:57.425 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:57 smithi149 ceph-mon[33332]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-03T14:38:57.425 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:57 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T14:38:57.425 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:57 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T14:38:57.425 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:57 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T14:38:57.425 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:57 smithi149 ceph-mon[33332]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-03T14:38:57.425 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:57 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T14:38:57.425 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:57 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T14:38:57.425 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:57 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T14:38:57.425 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:57 smithi149 ceph-mon[33332]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-03T14:38:57.425 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:57 smithi149 ceph-mon[33332]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-03T14:38:57.425 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:57 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T14:38:57.426 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:57 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T14:38:57.426 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:57 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T14:38:57.426 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:57 smithi149 ceph-mon[33332]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-03T14:38:57.498 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:57 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-03T14:38:57.498 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:57 smithi090 ceph-mon[33568]: osdmap e20: 3 total, 3 up, 3 in 2024-05-03T14:38:57.498 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:57 smithi090 ceph-mon[33568]: pgmap v96: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-03T14:38:57.498 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:57 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:57.498 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:57 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:57.498 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:57 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:57.498 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:57 smithi090 ceph-mon[33568]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-03T14:38:57.498 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:57 smithi090 ceph-mon[33568]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-03T14:38:57.498 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:57 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T14:38:57.498 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:57 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T14:38:57.498 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:57 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T14:38:57.498 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:57 smithi090 ceph-mon[33568]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-03T14:38:57.499 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:57 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T14:38:57.499 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:57 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T14:38:57.499 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:57 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T14:38:57.499 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:57 smithi090 ceph-mon[33568]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-03T14:38:57.499 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:57 smithi090 ceph-mon[33568]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-03T14:38:57.499 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:57 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T14:38:57.499 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:57 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T14:38:57.499 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:57 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T14:38:57.499 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:57 smithi090 ceph-mon[33568]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-03T14:38:57.511 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:57 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-03T14:38:57.511 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:57 smithi060 ceph-mon[28873]: osdmap e20: 3 total, 3 up, 3 in 2024-05-03T14:38:57.511 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:57 smithi060 ceph-mon[28873]: pgmap v96: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-03T14:38:57.511 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:57 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:57.511 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:57 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:57.511 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:57 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:38:57.511 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:57 smithi060 ceph-mon[28873]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-03T14:38:57.511 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:57 smithi060 ceph-mon[28873]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-03T14:38:57.511 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:57 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T14:38:57.511 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:57 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T14:38:57.511 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:57 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T14:38:57.511 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:57 smithi060 ceph-mon[28873]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-03T14:38:57.512 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:57 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T14:38:57.512 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:57 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T14:38:57.512 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:57 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T14:38:57.512 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:57 smithi060 ceph-mon[28873]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-03T14:38:57.512 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:57 smithi060 ceph-mon[28873]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-03T14:38:57.512 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:57 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T14:38:57.512 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:57 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T14:38:57.512 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:57 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T14:38:57.512 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:57 smithi060 ceph-mon[28873]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-03T14:38:57.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:38:57.684+0000 7fc195e00700 1 -- 172.21.15.60:0/438798501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc190105350 msgr2=0x7fc1901057b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:38:57.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:38:57.684+0000 7fc195e00700 1 --2- 172.21.15.60:0/438798501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc190105350 0x7fc1901057b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc178009990 tx=0x7fc178009ca0 comp rx=0 tx=0).stop 2024-05-03T14:38:57.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:38:57.685+0000 7fc195e00700 1 -- 172.21.15.60:0/438798501 shutdown_connections 2024-05-03T14:38:57.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:38:57.685+0000 7fc195e00700 1 --2- 172.21.15.60:0/438798501 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc190105cf0 0x7fc19010db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:38:57.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:38:57.685+0000 7fc195e00700 1 --2- 172.21.15.60:0/438798501 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc190106e10 0x7fc190107230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:38:57.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:38:57.685+0000 7fc195e00700 1 --2- 172.21.15.60:0/438798501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc190105350 0x7fc1901057b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:38:57.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:38:57.685+0000 7fc195e00700 1 -- 172.21.15.60:0/438798501 >> 172.21.15.60:0/438798501 conn(0x7fc190100cf0 msgr2=0x7fc190103110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:38:57.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:38:57.685+0000 7fc195e00700 1 -- 172.21.15.60:0/438798501 shutdown_connections 2024-05-03T14:38:57.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:38:57.685+0000 7fc195e00700 1 -- 172.21.15.60:0/438798501 wait complete. 2024-05-03T14:38:57.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:38:57.686+0000 7fc195e00700 1 Processor -- start 2024-05-03T14:38:57.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:38:57.686+0000 7fc195e00700 1 -- start start 2024-05-03T14:38:57.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:38:57.686+0000 7fc195e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc190105350 0x7fc19019bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:38:57.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:38:57.686+0000 7fc195e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc190105cf0 0x7fc19019c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:38:57.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:38:57.686+0000 7fc195e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc190106e10 0x7fc1901a30d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:38:57.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:38:57.686+0000 7fc195e00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc1901115b0 con 0x7fc190105350 2024-05-03T14:38:57.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:38:57.686+0000 7fc195e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc190111430 con 0x7fc190106e10 2024-05-03T14:38:57.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:38:57.686+0000 7fc195e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc190111730 con 0x7fc190105cf0 2024-05-03T14:38:57.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:38:57.687+0000 7fc18f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc190106e10 0x7fc1901a30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:38:57.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:38:57.687+0000 7fc18f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc190106e10 0x7fc1901a30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.60:52258/0 (socket says 172.21.15.60:52258) 2024-05-03T14:38:57.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:38:57.687+0000 7fc18ea00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc190105350 0x7fc19019bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:38:57.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:38:57.687+0000 7fc18f400700 1 -- 172.21.15.60:0/4233113951 learned_addr learned my addr 172.21.15.60:0/4233113951 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:38:57.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:38:57.687+0000 7fc186000700 1 --2- 172.21.15.60:0/4233113951 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc190105cf0 0x7fc19019c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:38:57.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:38:57.687+0000 7fc18f400700 1 -- 172.21.15.60:0/4233113951 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc190105cf0 msgr2=0x7fc19019c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:38:57.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:38:57.687+0000 7fc18f400700 1 --2- 172.21.15.60:0/4233113951 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc190105cf0 0x7fc19019c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:38:57.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:38:57.687+0000 7fc18f400700 1 -- 172.21.15.60:0/4233113951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc190105350 msgr2=0x7fc19019bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:38:57.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:38:57.687+0000 7fc18f400700 1 --2- 172.21.15.60:0/4233113951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc190105350 0x7fc19019bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:38:57.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:38:57.687+0000 7fc18f400700 1 -- 172.21.15.60:0/4233113951 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1901a3770 con 0x7fc190106e10 2024-05-03T14:38:57.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:38:57.687+0000 7fc186000700 1 --2- 172.21.15.60:0/4233113951 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc190105cf0 0x7fc19019c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:38:57.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:38:57.688+0000 7fc18f400700 1 --2- 172.21.15.60:0/4233113951 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc190106e10 0x7fc1901a30d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc18800e940 tx=0x7fc18800ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:38:57.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:38:57.688+0000 7fc194a00700 1 -- 172.21.15.60:0/4233113951 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc188004a30 con 0x7fc190106e10 2024-05-03T14:38:57.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:38:57.688+0000 7fc195e00700 1 -- 172.21.15.60:0/4233113951 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1901a3a60 con 0x7fc190106e10 2024-05-03T14:38:57.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:38:57.688+0000 7fc194a00700 1 -- 172.21.15.60:0/4233113951 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7fc188005520 con 0x7fc190106e10 2024-05-03T14:38:57.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:38:57.688+0000 7fc194a00700 1 -- 172.21.15.60:0/4233113951 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18800ce80 con 0x7fc190106e10 2024-05-03T14:38:57.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:38:57.688+0000 7fc195e00700 1 -- 172.21.15.60:0/4233113951 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1901a3ff0 con 0x7fc190106e10 2024-05-03T14:38:57.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:38:57.689+0000 7fc195e00700 1 -- 172.21.15.60:0/4233113951 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc19006a190 con 0x7fc190106e10 2024-05-03T14:38:57.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:38:57.691+0000 7fc194a00700 1 -- 172.21.15.60:0/4233113951 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fc188005020 con 0x7fc190106e10 2024-05-03T14:38:57.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:38:57.693+0000 7fc194a00700 1 --2- 172.21.15.60:0/4233113951 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fc1740b2900 0x7fc1740b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:38:57.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:38:57.693+0000 7fc194a00700 1 -- 172.21.15.60:0/4233113951 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 3237+0+0 (secure 0 0 0) 0x7fc1880e1090 con 0x7fc190106e10 2024-05-03T14:38:57.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:38:57.693+0000 7fc18ea00700 1 --2- 172.21.15.60:0/4233113951 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fc1740b2900 0x7fc1740b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:38:57.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:38:57.694+0000 7fc18ea00700 1 --2- 172.21.15.60:0/4233113951 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fc1740b2900 0x7fc1740b4dc0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc190106790 tx=0x7fc18000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:38:57.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:38:57.697+0000 7fc194a00700 1 -- 172.21.15.60:0/4233113951 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fc1880aa0e0 con 0x7fc190106e10 2024-05-03T14:38:57.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:38:57.946+0000 7fc195e00700 1 -- 172.21.15.60:0/4233113951 --> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fc1900638e0 con 0x7fc1740b2900 2024-05-03T14:38:58.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:58 smithi149 ceph-mon[33332]: osdmap e21: 3 total, 3 up, 3 in 2024-05-03T14:38:58.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:58 smithi149 ceph-mon[33332]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-03T14:38:58.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:58 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T14:38:58.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:58 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T14:38:58.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:58 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:38:58.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:58 smithi060 ceph-mon[28873]: osdmap e21: 3 total, 3 up, 3 in 2024-05-03T14:38:58.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:58 smithi060 ceph-mon[28873]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-03T14:38:58.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:58 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T14:38:58.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:58 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T14:38:58.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:58 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:38:58.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:58 smithi090 ceph-mon[33568]: osdmap e21: 3 total, 3 up, 3 in 2024-05-03T14:38:58.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:58 smithi090 ceph-mon[33568]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-03T14:38:58.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:58 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T14:38:58.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:58 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T14:38:58.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:58 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:38:59.239 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:59 smithi060 ceph-mon[28873]: from='client.24191 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:38:59.240 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:38:59 smithi060 ceph-mon[28873]: pgmap v98: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-03T14:38:59.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:59 smithi149 ceph-mon[33332]: from='client.24191 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:38:59.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:38:59 smithi149 ceph-mon[33332]: pgmap v98: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-03T14:38:59.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:59 smithi090 ceph-mon[33568]: from='client.24191 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:38:59.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:38:59 smithi090 ceph-mon[33568]: pgmap v98: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-03T14:39:01.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:01 smithi060 ceph-mon[28873]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-03T14:39:01.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:01 smithi090 ceph-mon[33568]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-03T14:39:01.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:01 smithi149 ceph-mon[33332]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-03T14:39:02.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:02 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/2790880119' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef31136a-9b36-4b87-8389-f53d311bc5cc"}]: dispatch 2024-05-03T14:39:02.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:02 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/2790880119' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ef31136a-9b36-4b87-8389-f53d311bc5cc"}]': finished 2024-05-03T14:39:02.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:02 smithi060 ceph-mon[28873]: osdmap e22: 4 total, 3 up, 4 in 2024-05-03T14:39:02.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:02 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T14:39:02.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:02 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/2790880119' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef31136a-9b36-4b87-8389-f53d311bc5cc"}]: dispatch 2024-05-03T14:39:02.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:02 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/2790880119' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ef31136a-9b36-4b87-8389-f53d311bc5cc"}]': finished 2024-05-03T14:39:02.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:02 smithi090 ceph-mon[33568]: osdmap e22: 4 total, 3 up, 4 in 2024-05-03T14:39:02.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:02 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T14:39:02.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:02 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/2790880119' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef31136a-9b36-4b87-8389-f53d311bc5cc"}]: dispatch 2024-05-03T14:39:02.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:02 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/2790880119' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ef31136a-9b36-4b87-8389-f53d311bc5cc"}]': finished 2024-05-03T14:39:02.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:02 smithi149 ceph-mon[33332]: osdmap e22: 4 total, 3 up, 4 in 2024-05-03T14:39:02.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:02 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T14:39:03.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:03 smithi060 ceph-mon[28873]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-03T14:39:03.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:03 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/2426097455' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T14:39:03.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:03 smithi090 ceph-mon[33568]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-03T14:39:03.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:03 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/2426097455' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T14:39:03.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:03 smithi149 ceph-mon[33332]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-03T14:39:03.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:03 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/2426097455' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T14:39:05.513 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:05 smithi060 ceph-mon[28873]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-03T14:39:05.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:05 smithi090 ceph-mon[33568]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-03T14:39:05.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:05 smithi149 ceph-mon[33332]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-03T14:39:07.570 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:07 smithi090 ceph-mon[33568]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-03T14:39:07.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:07 smithi060 ceph-mon[28873]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-03T14:39:07.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:07 smithi149 ceph-mon[33332]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-03T14:39:09.486 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:09 smithi060 ceph-mon[28873]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-03T14:39:09.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:09 smithi090 ceph-mon[33568]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-03T14:39:09.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:09 smithi149 ceph-mon[33332]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-03T14:39:11.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:11 smithi060 ceph-mon[28873]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-03T14:39:11.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:11 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-03T14:39:11.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:11 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:39:11.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:11 smithi090 ceph-mon[33568]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-03T14:39:11.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:11 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-03T14:39:11.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:11 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:39:11.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:11 smithi149 ceph-mon[33332]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-03T14:39:11.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:11 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-03T14:39:11.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:11 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:39:12.554 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:12 smithi060 ceph-mon[28873]: Deploying daemon osd.3 on smithi060 2024-05-03T14:39:12.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:12 smithi090 ceph-mon[33568]: Deploying daemon osd.3 on smithi060 2024-05-03T14:39:12.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:12 smithi149 ceph-mon[33332]: Deploying daemon osd.3 on smithi060 2024-05-03T14:39:13.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:13 smithi060 ceph-mon[28873]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-03T14:39:13.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:13 smithi090 ceph-mon[33568]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-03T14:39:13.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:13 smithi149 ceph-mon[33332]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-03T14:39:15.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:15 smithi060 ceph-mon[28873]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-03T14:39:15.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:15 smithi090 ceph-mon[33568]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-03T14:39:15.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:15 smithi149 ceph-mon[33332]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-03T14:39:17.515 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:17 smithi060 ceph-mon[28873]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-03T14:39:17.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:17 smithi090 ceph-mon[33568]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-03T14:39:17.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:17 smithi149 ceph-mon[33332]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-03T14:39:19.463 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:19 smithi060 ceph-mon[28873]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-03T14:39:19.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:19 smithi090 ceph-mon[33568]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-03T14:39:19.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:19 smithi149 ceph-mon[33332]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-03T14:39:21.792 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:21 smithi060 ceph-mon[28873]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-03T14:39:21.792 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:21 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:21.792 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:21 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:21.792 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:21 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:39:21.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:21 smithi090 ceph-mon[33568]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-03T14:39:21.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:21 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:21.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:21 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:21.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:21 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:39:21.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:21 smithi149 ceph-mon[33332]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-03T14:39:21.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:21 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:21.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:21 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:21.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:21 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:39:23.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:23 smithi060 ceph-mon[28873]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-03T14:39:23.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:23 smithi090 ceph-mon[33568]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-03T14:39:23.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:23 smithi149 ceph-mon[33332]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-03T14:39:25.510 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:25 smithi060 ceph-mon[28873]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-03T14:39:25.511 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:25 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:25.511 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:25 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:25.511 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:25 smithi060 ceph-mon[28873]: from='osd.3 [v2:172.21.15.60:6810/3226133334,v1:172.21.15.60:6811/3226133334]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-03T14:39:25.511 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:25 smithi060 ceph-mon[28873]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-03T14:39:25.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:39:25.608+0000 7fc194a00700 1 -- 172.21.15.60:0/4233113951 <== mgr.14152 v2:172.21.15.60:6800/3066142956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc1900638e0 con 0x7fc1740b2900 2024-05-03T14:39:25.611 INFO:teuthology.orchestra.run.smithi060.stdout:Created osd(s) 3 on host 'smithi060' 2024-05-03T14:39:25.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:39:25.611+0000 7fc195e00700 1 -- 172.21.15.60:0/4233113951 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fc1740b2900 msgr2=0x7fc1740b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:39:25.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:39:25.611+0000 7fc195e00700 1 --2- 172.21.15.60:0/4233113951 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fc1740b2900 0x7fc1740b4dc0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc190106790 tx=0x7fc18000b040 comp rx=0 tx=0).stop 2024-05-03T14:39:25.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:39:25.611+0000 7fc195e00700 1 -- 172.21.15.60:0/4233113951 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc190106e10 msgr2=0x7fc1901a30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:39:25.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:39:25.611+0000 7fc195e00700 1 --2- 172.21.15.60:0/4233113951 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc190106e10 0x7fc1901a30d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc18800e940 tx=0x7fc18800ec50 comp rx=0 tx=0).stop 2024-05-03T14:39:25.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:39:25.611+0000 7fc195e00700 1 -- 172.21.15.60:0/4233113951 shutdown_connections 2024-05-03T14:39:25.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:39:25.611+0000 7fc195e00700 1 --2- 172.21.15.60:0/4233113951 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fc1740b2900 0x7fc1740b4dc0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:39:25.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:39:25.611+0000 7fc195e00700 1 --2- 172.21.15.60:0/4233113951 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc190106e10 0x7fc1901a30d0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:39:25.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:39:25.611+0000 7fc195e00700 1 --2- 172.21.15.60:0/4233113951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc190105350 0x7fc19019bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:39:25.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:39:25.611+0000 7fc195e00700 1 --2- 172.21.15.60:0/4233113951 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc190105cf0 0x7fc19019c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:39:25.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:39:25.611+0000 7fc195e00700 1 -- 172.21.15.60:0/4233113951 >> 172.21.15.60:0/4233113951 conn(0x7fc190100cf0 msgr2=0x7fc190109b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:39:25.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:39:25.611+0000 7fc195e00700 1 -- 172.21.15.60:0/4233113951 shutdown_connections 2024-05-03T14:39:25.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:39:25.611+0000 7fc195e00700 1 -- 172.21.15.60:0/4233113951 wait complete. 2024-05-03T14:39:25.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:25 smithi090 ceph-mon[33568]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-03T14:39:25.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:25 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:25.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:25 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:25.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:25 smithi090 ceph-mon[33568]: from='osd.3 [v2:172.21.15.60:6810/3226133334,v1:172.21.15.60:6811/3226133334]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-03T14:39:25.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:25 smithi090 ceph-mon[33568]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-03T14:39:25.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:25 smithi149 ceph-mon[33332]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-03T14:39:25.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:25 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:25.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:25 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:25.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:25 smithi149 ceph-mon[33332]: from='osd.3 [v2:172.21.15.60:6810/3226133334,v1:172.21.15.60:6811/3226133334]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-03T14:39:25.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:25 smithi149 ceph-mon[33332]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-03T14:39:26.369 DEBUG:teuthology.orchestra.run.smithi060:osd.3> sudo journalctl -f -n 0 -u ceph-285d64b2-095a-11ef-bc97-c7b262605968@osd.3.service 2024-05-03T14:39:26.375 INFO:tasks.cephadm:Deploying osd.4 on smithi090 with /dev/vg_nvme/lv_3... 2024-05-03T14:39:26.375 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-03T14:39:26.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:26 smithi060 ceph-mon[28873]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-03T14:39:26.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:26 smithi060 ceph-mon[28873]: osdmap e23: 4 total, 3 up, 4 in 2024-05-03T14:39:26.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:26 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T14:39:26.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:26 smithi060 ceph-mon[28873]: from='osd.3 [v2:172.21.15.60:6810/3226133334,v1:172.21.15.60:6811/3226133334]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-05-03T14:39:26.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:26 smithi060 ceph-mon[28873]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-05-03T14:39:26.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:26 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:39:26.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:26 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:39:26.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:26 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:26.589 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:26 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:26.589 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:26 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:26.589 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:26 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:39:26.618 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.b/config 2024-05-03T14:39:26.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:26 smithi090 ceph-mon[33568]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-03T14:39:26.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:26 smithi090 ceph-mon[33568]: osdmap e23: 4 total, 3 up, 4 in 2024-05-03T14:39:26.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:26 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T14:39:26.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:26 smithi090 ceph-mon[33568]: from='osd.3 [v2:172.21.15.60:6810/3226133334,v1:172.21.15.60:6811/3226133334]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-05-03T14:39:26.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:26 smithi090 ceph-mon[33568]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-05-03T14:39:26.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:26 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:39:26.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:26 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:39:26.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:26 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:26.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:26 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:26.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:26 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:26.626 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:26 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:39:26.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:26 smithi149 ceph-mon[33332]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-03T14:39:26.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:26 smithi149 ceph-mon[33332]: osdmap e23: 4 total, 3 up, 4 in 2024-05-03T14:39:26.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:26 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T14:39:26.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:26 smithi149 ceph-mon[33332]: from='osd.3 [v2:172.21.15.60:6810/3226133334,v1:172.21.15.60:6811/3226133334]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-05-03T14:39:26.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:26 smithi149 ceph-mon[33332]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-05-03T14:39:26.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:26 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:39:26.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:26 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:39:26.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:26 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:26.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:26 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:26.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:26 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:26.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:26 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:39:27.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:27 smithi060 ceph-mon[28873]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-03T14:39:27.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:27 smithi060 ceph-mon[28873]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-05-03T14:39:27.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:27 smithi060 ceph-mon[28873]: osdmap e24: 4 total, 3 up, 4 in 2024-05-03T14:39:27.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:27 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T14:39:27.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:27 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T14:39:27.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:27 smithi060 ceph-mon[28873]: osd.3 [v2:172.21.15.60:6810/3226133334,v1:172.21.15.60:6811/3226133334] boot 2024-05-03T14:39:27.607 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:27 smithi090 ceph-mon[33568]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-03T14:39:27.608 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:27 smithi090 ceph-mon[33568]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-05-03T14:39:27.608 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:27 smithi090 ceph-mon[33568]: osdmap e24: 4 total, 3 up, 4 in 2024-05-03T14:39:27.608 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:27 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T14:39:27.608 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:27 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T14:39:27.608 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:27 smithi090 ceph-mon[33568]: osd.3 [v2:172.21.15.60:6810/3226133334,v1:172.21.15.60:6811/3226133334] boot 2024-05-03T14:39:27.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:27 smithi149 ceph-mon[33332]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-03T14:39:27.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:27 smithi149 ceph-mon[33332]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-05-03T14:39:27.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:27 smithi149 ceph-mon[33332]: osdmap e24: 4 total, 3 up, 4 in 2024-05-03T14:39:27.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:27 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T14:39:27.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:27 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T14:39:27.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:27 smithi149 ceph-mon[33332]: osd.3 [v2:172.21.15.60:6810/3226133334,v1:172.21.15.60:6811/3226133334] boot 2024-05-03T14:39:28.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:28 smithi149 ceph-mon[33332]: purged_snaps scrub starts 2024-05-03T14:39:28.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:28 smithi149 ceph-mon[33332]: purged_snaps scrub ok 2024-05-03T14:39:28.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:28 smithi149 ceph-mon[33332]: osdmap e25: 4 total, 4 up, 4 in 2024-05-03T14:39:28.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:28 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T14:39:28.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:28 smithi149 ceph-mon[33332]: osdmap e26: 4 total, 4 up, 4 in 2024-05-03T14:39:28.453 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-03T14:39:28.453 INFO:teuthology.orchestra.run.smithi090.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-03T14:39:28.453 INFO:teuthology.orchestra.run.smithi090.stderr: stderr: 10+0 records in 2024-05-03T14:39:28.454 INFO:teuthology.orchestra.run.smithi090.stderr:10+0 records out 2024-05-03T14:39:28.454 INFO:teuthology.orchestra.run.smithi090.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115335 s, 909 MB/s 2024-05-03T14:39:28.454 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping successful for: 2024-05-03T14:39:28.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:28 smithi090 ceph-mon[33568]: purged_snaps scrub starts 2024-05-03T14:39:28.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:28 smithi090 ceph-mon[33568]: purged_snaps scrub ok 2024-05-03T14:39:28.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:28 smithi090 ceph-mon[33568]: osdmap e25: 4 total, 4 up, 4 in 2024-05-03T14:39:28.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:28 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T14:39:28.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:28 smithi090 ceph-mon[33568]: osdmap e26: 4 total, 4 up, 4 in 2024-05-03T14:39:28.652 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:28 smithi060 ceph-mon[28873]: purged_snaps scrub starts 2024-05-03T14:39:28.652 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:28 smithi060 ceph-mon[28873]: purged_snaps scrub ok 2024-05-03T14:39:28.652 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:28 smithi060 ceph-mon[28873]: osdmap e25: 4 total, 4 up, 4 in 2024-05-03T14:39:28.652 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:28 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T14:39:28.652 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:28 smithi060 ceph-mon[28873]: osdmap e26: 4 total, 4 up, 4 in 2024-05-03T14:39:29.078 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi090:vg_nvme/lv_3 2024-05-03T14:39:29.317 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.b/config 2024-05-03T14:39:29.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:29 smithi090 ceph-mon[33568]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-03T14:39:29.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:29 smithi090 ceph-mon[33568]: osdmap e27: 4 total, 4 up, 4 in 2024-05-03T14:39:29.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:29 smithi149 ceph-mon[33332]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-03T14:39:29.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:29 smithi149 ceph-mon[33332]: osdmap e27: 4 total, 4 up, 4 in 2024-05-03T14:39:29.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:29 smithi060 ceph-mon[28873]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-03T14:39:29.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:29 smithi060 ceph-mon[28873]: osdmap e27: 4 total, 4 up, 4 in 2024-05-03T14:39:30.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:39:30.915+0000 7f0e9ab6a700 1 -- 172.21.15.90:0/2737552358 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e94103fa0 msgr2=0x7f0e9410a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:39:30.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:39:30.915+0000 7f0e9ab6a700 1 --2- 172.21.15.90:0/2737552358 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e94103fa0 0x7f0e9410a850 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c00b0d0 tx=0x7f0e8c00b3e0 comp rx=0 tx=0).stop 2024-05-03T14:39:30.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:39:30.915+0000 7f0e9ab6a700 1 -- 172.21.15.90:0/2737552358 shutdown_connections 2024-05-03T14:39:30.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:39:30.915+0000 7f0e9ab6a700 1 --2- 172.21.15.90:0/2737552358 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e94103fa0 0x7f0e9410a850 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:39:30.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:39:30.915+0000 7f0e9ab6a700 1 --2- 172.21.15.90:0/2737552358 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0e94102400 0x7f0e94102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:39:30.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:39:30.915+0000 7f0e9ab6a700 1 --2- 172.21.15.90:0/2737552358 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e94103600 0x7f0e94103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:39:30.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:39:30.915+0000 7f0e9ab6a700 1 -- 172.21.15.90:0/2737552358 >> 172.21.15.90:0/2737552358 conn(0x7f0e940fd9c0 msgr2=0x7f0e940ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:39:30.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:39:30.915+0000 7f0e9ab6a700 1 -- 172.21.15.90:0/2737552358 shutdown_connections 2024-05-03T14:39:30.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:39:30.915+0000 7f0e9ab6a700 1 -- 172.21.15.90:0/2737552358 wait complete. 2024-05-03T14:39:30.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:39:30.916+0000 7f0e9ab6a700 1 Processor -- start 2024-05-03T14:39:30.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:39:30.916+0000 7f0e9ab6a700 1 -- start start 2024-05-03T14:39:30.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:39:30.916+0000 7f0e9ab6a700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e94102400 0x7f0e941997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:39:30.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:39:30.916+0000 7f0e9ab6a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e94103600 0x7f0e94199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:39:30.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:39:30.916+0000 7f0e9ab6a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0e94103fa0 0x7f0e941a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:39:30.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:39:30.916+0000 7f0e9ab6a700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9410e1c0 con 0x7f0e94103fa0 2024-05-03T14:39:30.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:39:30.916+0000 7f0e9ab6a700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9410e040 con 0x7f0e94102400 2024-05-03T14:39:30.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:39:30.916+0000 7f0e9ab6a700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9410e340 con 0x7f0e94103600 2024-05-03T14:39:30.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:39:30.916+0000 7f0e98906700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e94102400 0x7f0e941997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:39:30.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:39:30.916+0000 7f0e98906700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e94102400 0x7f0e941997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57042/0 (socket says 172.21.15.90:57042) 2024-05-03T14:39:30.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:39:30.916+0000 7f0e98906700 1 -- 172.21.15.90:0/1535788603 learned_addr learned my addr 172.21.15.90:0/1535788603 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-03T14:39:30.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:39:30.916+0000 7f0e93fff700 1 --2- 172.21.15.90:0/1535788603 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e94103600 0x7f0e94199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:39:30.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:39:30.916+0000 7f0e98906700 1 -- 172.21.15.90:0/1535788603 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e94103600 msgr2=0x7f0e94199d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:39:30.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:39:30.916+0000 7f0e98906700 1 --2- 172.21.15.90:0/1535788603 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e94103600 0x7f0e94199d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:39:30.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:39:30.916+0000 7f0e98906700 1 -- 172.21.15.90:0/1535788603 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0e94103fa0 msgr2=0x7f0e941a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:39:30.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:39:30.916+0000 7f0e98906700 1 --2- 172.21.15.90:0/1535788603 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0e94103fa0 0x7f0e941a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:39:30.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:39:30.916+0000 7f0e98906700 1 -- 172.21.15.90:0/1535788603 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e941a1430 con 0x7f0e94102400 2024-05-03T14:39:30.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:39:30.916+0000 7f0e98906700 1 --2- 172.21.15.90:0/1535788603 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e94102400 0x7f0e941997d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0e8800b500 tx=0x7f0e8800b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:39:30.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:39:30.916+0000 7f0e91ffb700 1 -- 172.21.15.90:0/1535788603 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e88015020 con 0x7f0e94102400 2024-05-03T14:39:30.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:39:30.916+0000 7f0e91ffb700 1 -- 172.21.15.90:0/1535788603 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f0e88011d30 con 0x7f0e94102400 2024-05-03T14:39:30.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:39:30.916+0000 7f0e9ab6a700 1 -- 172.21.15.90:0/1535788603 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e941a1720 con 0x7f0e94102400 2024-05-03T14:39:30.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:39:30.916+0000 7f0e91ffb700 1 -- 172.21.15.90:0/1535788603 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e8800f2e0 con 0x7f0e94102400 2024-05-03T14:39:30.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:39:30.917+0000 7f0e9ab6a700 1 -- 172.21.15.90:0/1535788603 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e941a1d30 con 0x7f0e94102400 2024-05-03T14:39:30.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:39:30.918+0000 7f0e9ab6a700 1 -- 172.21.15.90:0/1535788603 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e9406a190 con 0x7f0e94102400 2024-05-03T14:39:30.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:39:30.919+0000 7f0e91ffb700 1 -- 172.21.15.90:0/1535788603 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f0e88018020 con 0x7f0e94102400 2024-05-03T14:39:30.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:39:30.919+0000 7f0e91ffb700 1 --2- 172.21.15.90:0/1535788603 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f0e840b2900 0x7f0e840b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:39:30.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:39:30.920+0000 7f0e91ffb700 1 -- 172.21.15.90:0/1535788603 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3669+0+0 (secure 0 0 0) 0x7f0e880e06d0 con 0x7f0e94102400 2024-05-03T14:39:30.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:39:30.920+0000 7f0e93fff700 1 --2- 172.21.15.90:0/1535788603 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f0e840b2900 0x7f0e840b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:39:30.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:39:30.921+0000 7f0e93fff700 1 --2- 172.21.15.90:0/1535788603 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f0e840b2900 0x7f0e840b4dc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0e80009c60 tx=0x7f0e80009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:39:30.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:39:30.924+0000 7f0e91ffb700 1 -- 172.21.15.90:0/1535788603 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f0e8801f020 con 0x7f0e94102400 2024-05-03T14:39:31.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:39:31.135+0000 7f0e9ab6a700 1 -- 172.21.15.90:0/1535788603 --> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f0e940ffa60 con 0x7f0e840b2900 2024-05-03T14:39:31.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:31 smithi060 ceph-mon[28873]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-03T14:39:31.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:31 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T14:39:31.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:31 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T14:39:31.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:31 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:39:31.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:31 smithi090 ceph-mon[33568]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-03T14:39:31.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:31 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T14:39:31.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:31 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T14:39:31.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:31 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:39:31.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:31 smithi149 ceph-mon[33332]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-03T14:39:31.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:31 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T14:39:31.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:31 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T14:39:31.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:31 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:39:32.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:32 smithi149 ceph-mon[33332]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:39:32.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:32 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:32.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:32 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:32.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:32 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:39:32.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:32 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:39:32.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:32 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:32.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:32 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:39:32.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:32 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:39:32.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:32 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:32.725 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:32 smithi090 ceph-mon[33568]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:39:32.725 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:32 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:32.725 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:32 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:32.725 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:32 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:39:32.725 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:32 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:39:32.725 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:32 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:32.726 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:32 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:39:32.726 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:32 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:39:32.726 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:32 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:32.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:32 smithi060 ceph-mon[28873]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:39:32.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:32 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:32.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:32 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:32.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:32 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:39:32.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:32 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:39:32.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:32 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:32.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:32 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:39:32.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:32 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:39:32.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:32 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:33.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:33 smithi149 ceph-mon[33332]: Detected new or changed devices on smithi060 2024-05-03T14:39:33.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:33 smithi149 ceph-mon[33332]: Adjusting osd_memory_target on smithi060 to 8547M 2024-05-03T14:39:33.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:33 smithi149 ceph-mon[33332]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-05-03T14:39:33.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:33 smithi060 ceph-mon[28873]: Detected new or changed devices on smithi060 2024-05-03T14:39:33.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:33 smithi060 ceph-mon[28873]: Adjusting osd_memory_target on smithi060 to 8547M 2024-05-03T14:39:33.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:33 smithi060 ceph-mon[28873]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-05-03T14:39:33.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:33 smithi090 ceph-mon[33568]: Detected new or changed devices on smithi060 2024-05-03T14:39:33.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:33 smithi090 ceph-mon[33568]: Adjusting osd_memory_target on smithi060 to 8547M 2024-05-03T14:39:33.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:33 smithi090 ceph-mon[33568]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-05-03T14:39:34.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:34 smithi090 ceph-mon[33568]: from='client.? 172.21.15.90:0/1820985376' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73db7408-a2ff-47e9-b13d-d31688a93610"}]: dispatch 2024-05-03T14:39:34.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:34 smithi090 ceph-mon[33568]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73db7408-a2ff-47e9-b13d-d31688a93610"}]: dispatch 2024-05-03T14:39:34.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:34 smithi090 ceph-mon[33568]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "73db7408-a2ff-47e9-b13d-d31688a93610"}]': finished 2024-05-03T14:39:34.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:34 smithi090 ceph-mon[33568]: osdmap e28: 5 total, 4 up, 5 in 2024-05-03T14:39:34.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:34 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T14:39:34.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:34 smithi149 ceph-mon[33332]: from='client.? 172.21.15.90:0/1820985376' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73db7408-a2ff-47e9-b13d-d31688a93610"}]: dispatch 2024-05-03T14:39:34.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:34 smithi149 ceph-mon[33332]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73db7408-a2ff-47e9-b13d-d31688a93610"}]: dispatch 2024-05-03T14:39:34.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:34 smithi149 ceph-mon[33332]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "73db7408-a2ff-47e9-b13d-d31688a93610"}]': finished 2024-05-03T14:39:34.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:34 smithi149 ceph-mon[33332]: osdmap e28: 5 total, 4 up, 5 in 2024-05-03T14:39:34.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:34 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T14:39:34.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:34 smithi060 ceph-mon[28873]: from='client.? 172.21.15.90:0/1820985376' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73db7408-a2ff-47e9-b13d-d31688a93610"}]: dispatch 2024-05-03T14:39:34.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:34 smithi060 ceph-mon[28873]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73db7408-a2ff-47e9-b13d-d31688a93610"}]: dispatch 2024-05-03T14:39:34.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:34 smithi060 ceph-mon[28873]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "73db7408-a2ff-47e9-b13d-d31688a93610"}]': finished 2024-05-03T14:39:34.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:34 smithi060 ceph-mon[28873]: osdmap e28: 5 total, 4 up, 5 in 2024-05-03T14:39:34.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:34 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T14:39:35.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:35 smithi149 ceph-mon[33332]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-03T14:39:35.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:35 smithi149 ceph-mon[33332]: from='client.? 172.21.15.90:0/504102547' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T14:39:35.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:35 smithi060 ceph-mon[28873]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-03T14:39:35.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:35 smithi060 ceph-mon[28873]: from='client.? 172.21.15.90:0/504102547' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T14:39:35.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:35 smithi090 ceph-mon[33568]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-03T14:39:35.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:35 smithi090 ceph-mon[33568]: from='client.? 172.21.15.90:0/504102547' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T14:39:37.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:37 smithi149 ceph-mon[33332]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-05-03T14:39:37.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:37 smithi060 ceph-mon[28873]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-05-03T14:39:37.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:37 smithi090 ceph-mon[33568]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-05-03T14:39:39.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:39 smithi090 ceph-mon[33568]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-05-03T14:39:39.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:39 smithi149 ceph-mon[33332]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-05-03T14:39:39.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:39 smithi060 ceph-mon[28873]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-05-03T14:39:41.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:41 smithi149 ceph-mon[33332]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-03T14:39:41.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:41 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-03T14:39:41.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:41 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:39:41.762 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:41 smithi090 ceph-mon[33568]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-03T14:39:41.762 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:41 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-03T14:39:41.762 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:41 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:39:41.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:41 smithi060 ceph-mon[28873]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-03T14:39:41.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:41 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-03T14:39:41.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:41 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:39:42.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:42 smithi149 ceph-mon[33332]: Deploying daemon osd.4 on smithi090 2024-05-03T14:39:42.790 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:42 smithi090 ceph-mon[33568]: Deploying daemon osd.4 on smithi090 2024-05-03T14:39:42.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:42 smithi060 ceph-mon[28873]: Deploying daemon osd.4 on smithi090 2024-05-03T14:39:43.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:43 smithi149 ceph-mon[33332]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-03T14:39:43.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:43 smithi060 ceph-mon[28873]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-03T14:39:43.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:43 smithi090 ceph-mon[33568]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-03T14:39:45.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:45 smithi090 ceph-mon[33568]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-03T14:39:45.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:45 smithi149 ceph-mon[33332]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-03T14:39:45.776 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:45 smithi060 ceph-mon[28873]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-03T14:39:47.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:47 smithi060 ceph-mon[28873]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-03T14:39:47.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:47 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:47.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:47 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:47.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:47 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:39:47.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:47 smithi090 ceph-mon[33568]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-03T14:39:47.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:47 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:47.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:47 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:47.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:47 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:39:47.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:47 smithi149 ceph-mon[33332]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-03T14:39:47.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:47 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:47.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:47 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:47.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:47 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:39:49.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:49 smithi060 ceph-mon[28873]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-03T14:39:49.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:49 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:49.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:49 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:49.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:49 smithi090 ceph-mon[33568]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-03T14:39:49.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:49 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:49.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:49 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:49.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:49 smithi149 ceph-mon[33332]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-03T14:39:49.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:49 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:49.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:49 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:50.528 INFO:teuthology.orchestra.run.smithi090.stdout:Created osd(s) 4 on host 'smithi090' 2024-05-03T14:39:50.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:39:50.525+0000 7f0e91ffb700 1 -- 172.21.15.90:0/1535788603 <== mgr.14152 v2:172.21.15.60:6800/3066142956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0e940ffa60 con 0x7f0e840b2900 2024-05-03T14:39:50.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:39:50.527+0000 7f0e9ab6a700 1 -- 172.21.15.90:0/1535788603 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f0e840b2900 msgr2=0x7f0e840b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:39:50.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:39:50.528+0000 7f0e9ab6a700 1 --2- 172.21.15.90:0/1535788603 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f0e840b2900 0x7f0e840b4dc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0e80009c60 tx=0x7f0e80009260 comp rx=0 tx=0).stop 2024-05-03T14:39:50.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:39:50.528+0000 7f0e9ab6a700 1 -- 172.21.15.90:0/1535788603 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e94102400 msgr2=0x7f0e941997d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:39:50.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:39:50.528+0000 7f0e9ab6a700 1 --2- 172.21.15.90:0/1535788603 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e94102400 0x7f0e941997d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0e8800b500 tx=0x7f0e8800b810 comp rx=0 tx=0).stop 2024-05-03T14:39:50.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:39:50.528+0000 7f0e9ab6a700 1 -- 172.21.15.90:0/1535788603 shutdown_connections 2024-05-03T14:39:50.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:39:50.528+0000 7f0e9ab6a700 1 --2- 172.21.15.90:0/1535788603 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f0e840b2900 0x7f0e840b4dc0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:39:50.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:39:50.528+0000 7f0e9ab6a700 1 --2- 172.21.15.90:0/1535788603 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0e94103fa0 0x7f0e941a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:39:50.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:39:50.528+0000 7f0e9ab6a700 1 --2- 172.21.15.90:0/1535788603 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e94102400 0x7f0e941997d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:39:50.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:39:50.528+0000 7f0e9ab6a700 1 --2- 172.21.15.90:0/1535788603 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e94103600 0x7f0e94199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:39:50.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:39:50.528+0000 7f0e9ab6a700 1 -- 172.21.15.90:0/1535788603 >> 172.21.15.90:0/1535788603 conn(0x7f0e940fd9c0 msgr2=0x7f0e940ff350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:39:50.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:39:50.528+0000 7f0e9ab6a700 1 -- 172.21.15.90:0/1535788603 shutdown_connections 2024-05-03T14:39:50.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:39:50.528+0000 7f0e9ab6a700 1 -- 172.21.15.90:0/1535788603 wait complete. 2024-05-03T14:39:50.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:50 smithi060 ceph-mon[28873]: from='osd.4 [v2:172.21.15.90:6808/2201391883,v1:172.21.15.90:6809/2201391883]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-03T14:39:50.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:50 smithi060 ceph-mon[28873]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-03T14:39:50.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:50 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:50.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:50 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:50.866 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:50 smithi090 ceph-mon[33568]: from='osd.4 [v2:172.21.15.90:6808/2201391883,v1:172.21.15.90:6809/2201391883]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-03T14:39:50.866 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:50 smithi090 ceph-mon[33568]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-03T14:39:50.866 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:50 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:50.866 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:50 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:50.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:50 smithi149 ceph-mon[33332]: from='osd.4 [v2:172.21.15.90:6808/2201391883,v1:172.21.15.90:6809/2201391883]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-03T14:39:50.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:50 smithi149 ceph-mon[33332]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-03T14:39:50.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:50 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:50.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:50 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:51.231 DEBUG:teuthology.orchestra.run.smithi090:osd.4> sudo journalctl -f -n 0 -u ceph-285d64b2-095a-11ef-bc97-c7b262605968@osd.4.service 2024-05-03T14:39:51.233 INFO:tasks.cephadm:Deploying osd.5 on smithi149 with /dev/vg_nvme/lv_3... 2024-05-03T14:39:51.233 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-03T14:39:51.478 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T14:39:51.839 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:51 smithi149 ceph-mon[33332]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-03T14:39:51.839 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:51 smithi149 ceph-mon[33332]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-03T14:39:51.839 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:51 smithi149 ceph-mon[33332]: from='osd.4 [v2:172.21.15.90:6808/2201391883,v1:172.21.15.90:6809/2201391883]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-05-03T14:39:51.839 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:51 smithi149 ceph-mon[33332]: osdmap e29: 5 total, 4 up, 5 in 2024-05-03T14:39:51.839 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:51 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T14:39:51.839 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:51 smithi149 ceph-mon[33332]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-05-03T14:39:51.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:51 smithi090 ceph-mon[33568]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-03T14:39:51.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:51 smithi090 ceph-mon[33568]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-03T14:39:51.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:51 smithi090 ceph-mon[33568]: from='osd.4 [v2:172.21.15.90:6808/2201391883,v1:172.21.15.90:6809/2201391883]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-05-03T14:39:51.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:51 smithi090 ceph-mon[33568]: osdmap e29: 5 total, 4 up, 5 in 2024-05-03T14:39:51.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:51 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T14:39:51.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:51 smithi090 ceph-mon[33568]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-05-03T14:39:52.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:51 smithi060 ceph-mon[28873]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-03T14:39:52.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:51 smithi060 ceph-mon[28873]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-03T14:39:52.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:51 smithi060 ceph-mon[28873]: from='osd.4 [v2:172.21.15.90:6808/2201391883,v1:172.21.15.90:6809/2201391883]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-05-03T14:39:52.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:51 smithi060 ceph-mon[28873]: osdmap e29: 5 total, 4 up, 5 in 2024-05-03T14:39:52.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:51 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T14:39:52.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:51 smithi060 ceph-mon[28873]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-05-03T14:39:52.375 INFO:journalctl@ceph.osd.4.smithi090.stdout:May 03 14:39:52 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-4[45126]: 2024-05-03T14:39:52.045+0000 7ff3a5c80700 -1 osd.4 0 waiting for initial osdmap 2024-05-03T14:39:52.375 INFO:journalctl@ceph.osd.4.smithi090.stdout:May 03 14:39:52 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-4[45126]: 2024-05-03T14:39:52.052+0000 7ff39da6b700 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-03T14:39:52.695 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:52 smithi090 ceph-mon[33568]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-05-03T14:39:52.695 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:52 smithi090 ceph-mon[33568]: osdmap e30: 5 total, 4 up, 5 in 2024-05-03T14:39:52.695 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:52 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T14:39:52.695 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:52 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T14:39:52.695 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:52 smithi090 ceph-mon[33568]: from='osd.4 ' entity='osd.4' 2024-05-03T14:39:52.845 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:52 smithi149 ceph-mon[33332]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-05-03T14:39:52.845 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:52 smithi149 ceph-mon[33332]: osdmap e30: 5 total, 4 up, 5 in 2024-05-03T14:39:52.845 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:52 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T14:39:52.845 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:52 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T14:39:52.845 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:52 smithi149 ceph-mon[33332]: from='osd.4 ' entity='osd.4' 2024-05-03T14:39:53.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:52 smithi060 ceph-mon[28873]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-05-03T14:39:53.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:52 smithi060 ceph-mon[28873]: osdmap e30: 5 total, 4 up, 5 in 2024-05-03T14:39:53.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:52 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T14:39:53.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:52 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T14:39:53.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:52 smithi060 ceph-mon[28873]: from='osd.4 ' entity='osd.4' 2024-05-03T14:39:53.530 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-03T14:39:53.530 INFO:teuthology.orchestra.run.smithi149.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-03T14:39:53.531 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10+0 records in 2024-05-03T14:39:53.531 INFO:teuthology.orchestra.run.smithi149.stderr:10+0 records out 2024-05-03T14:39:53.531 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112906 s, 929 MB/s 2024-05-03T14:39:53.531 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping successful for: 2024-05-03T14:39:53.788 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:53 smithi090 ceph-mon[33568]: purged_snaps scrub starts 2024-05-03T14:39:53.788 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:53 smithi090 ceph-mon[33568]: purged_snaps scrub ok 2024-05-03T14:39:53.788 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:53 smithi090 ceph-mon[33568]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-03T14:39:53.788 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:53 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T14:39:53.788 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:53 smithi090 ceph-mon[33568]: osd.4 [v2:172.21.15.90:6808/2201391883,v1:172.21.15.90:6809/2201391883] boot 2024-05-03T14:39:53.788 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:53 smithi090 ceph-mon[33568]: osdmap e31: 5 total, 5 up, 5 in 2024-05-03T14:39:53.788 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:53 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T14:39:53.850 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:53 smithi149 ceph-mon[33332]: purged_snaps scrub starts 2024-05-03T14:39:53.851 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:53 smithi149 ceph-mon[33332]: purged_snaps scrub ok 2024-05-03T14:39:53.851 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:53 smithi149 ceph-mon[33332]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-03T14:39:53.851 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:53 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T14:39:53.851 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:53 smithi149 ceph-mon[33332]: osd.4 [v2:172.21.15.90:6808/2201391883,v1:172.21.15.90:6809/2201391883] boot 2024-05-03T14:39:53.851 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:53 smithi149 ceph-mon[33332]: osdmap e31: 5 total, 5 up, 5 in 2024-05-03T14:39:53.851 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:53 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T14:39:54.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:53 smithi060 ceph-mon[28873]: purged_snaps scrub starts 2024-05-03T14:39:54.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:53 smithi060 ceph-mon[28873]: purged_snaps scrub ok 2024-05-03T14:39:54.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:53 smithi060 ceph-mon[28873]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-03T14:39:54.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:53 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T14:39:54.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:53 smithi060 ceph-mon[28873]: osd.4 [v2:172.21.15.90:6808/2201391883,v1:172.21.15.90:6809/2201391883] boot 2024-05-03T14:39:54.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:53 smithi060 ceph-mon[28873]: osdmap e31: 5 total, 5 up, 5 in 2024-05-03T14:39:54.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:53 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T14:39:54.214 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi149:vg_nvme/lv_3 2024-05-03T14:39:54.448 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T14:39:55.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:55 smithi149 ceph-mon[33332]: osdmap e32: 5 total, 5 up, 5 in 2024-05-03T14:39:55.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:55 smithi149 ceph-mon[33332]: Detected new or changed devices on smithi090 2024-05-03T14:39:55.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:55 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:55.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:55 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:55.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:55 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:39:55.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:55 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:39:55.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:55 smithi149 ceph-mon[33332]: Adjusting osd_memory_target on smithi090 to 8544M 2024-05-03T14:39:55.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:55 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:55.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:55 smithi149 ceph-mon[33332]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-05-03T14:39:55.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:55 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:39:55.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:55 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:39:55.175 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:55 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:55.175 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:55 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:39:55.175 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:55 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:39:55.175 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:55 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:39:55.175 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:55 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:55.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:55 smithi060 ceph-mon[28873]: osdmap e32: 5 total, 5 up, 5 in 2024-05-03T14:39:55.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:55 smithi060 ceph-mon[28873]: Detected new or changed devices on smithi090 2024-05-03T14:39:55.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:55 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:55.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:55 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:55.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:55 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:39:55.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:55 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:39:55.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:55 smithi060 ceph-mon[28873]: Adjusting osd_memory_target on smithi090 to 8544M 2024-05-03T14:39:55.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:55 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:55.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:55 smithi060 ceph-mon[28873]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-05-03T14:39:55.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:55 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:39:55.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:55 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:39:55.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:55 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:55.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:55 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:39:55.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:55 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:39:55.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:55 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:39:55.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:55 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:55.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:55 smithi090 ceph-mon[33568]: osdmap e32: 5 total, 5 up, 5 in 2024-05-03T14:39:55.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:55 smithi090 ceph-mon[33568]: Detected new or changed devices on smithi090 2024-05-03T14:39:55.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:55 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:55.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:55 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:55.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:55 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:39:55.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:55 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:39:55.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:55 smithi090 ceph-mon[33568]: Adjusting osd_memory_target on smithi090 to 8544M 2024-05-03T14:39:55.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:55 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:55.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:55 smithi090 ceph-mon[33568]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-05-03T14:39:55.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:55 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:39:55.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:55 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:39:55.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:55 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:55.377 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:55 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:39:55.377 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:55 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:39:55.377 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:55 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:39:55.377 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:55 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:39:56.160 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:39:56.156+0000 7fe5f92fe700 1 -- 172.21.15.149:0/2475664458 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe5f4103fd0 msgr2=0x7fe5f410a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:39:56.160 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:39:56.156+0000 7fe5f92fe700 1 --2- 172.21.15.149:0/2475664458 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe5f4103fd0 0x7fe5f410a880 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec00b0a0 tx=0x7fe5ec00b3b0 comp rx=0 tx=0).stop 2024-05-03T14:39:56.161 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:39:56.156+0000 7fe5f92fe700 1 -- 172.21.15.149:0/2475664458 shutdown_connections 2024-05-03T14:39:56.161 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:39:56.156+0000 7fe5f92fe700 1 --2- 172.21.15.149:0/2475664458 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe5f4103fd0 0x7fe5f410a880 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:39:56.161 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:39:56.156+0000 7fe5f92fe700 1 --2- 172.21.15.149:0/2475664458 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5f4102430 0x7fe5f4102850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:39:56.161 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:39:56.156+0000 7fe5f92fe700 1 --2- 172.21.15.149:0/2475664458 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5f4103630 0x7fe5f4103a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:39:56.161 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:39:56.156+0000 7fe5f92fe700 1 -- 172.21.15.149:0/2475664458 >> 172.21.15.149:0/2475664458 conn(0x7fe5f40fd9f0 msgr2=0x7fe5f40ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:39:56.161 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:39:56.156+0000 7fe5f92fe700 1 -- 172.21.15.149:0/2475664458 shutdown_connections 2024-05-03T14:39:56.161 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:39:56.156+0000 7fe5f92fe700 1 -- 172.21.15.149:0/2475664458 wait complete. 2024-05-03T14:39:56.161 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:39:56.157+0000 7fe5f92fe700 1 Processor -- start 2024-05-03T14:39:56.161 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:39:56.157+0000 7fe5f92fe700 1 -- start start 2024-05-03T14:39:56.161 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:39:56.157+0000 7fe5f92fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5f4102430 0x7fe5f41986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:39:56.161 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:39:56.157+0000 7fe5f92fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe5f4103630 0x7fe5f4198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:39:56.161 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:39:56.157+0000 7fe5f92fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5f4103fd0 0x7fe5f419fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:39:56.161 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:39:56.157+0000 7fe5f92fe700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f410e1d0 con 0x7fe5f4103fd0 2024-05-03T14:39:56.162 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:39:56.157+0000 7fe5f92fe700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f410e050 con 0x7fe5f4102430 2024-05-03T14:39:56.162 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:39:56.157+0000 7fe5f92fe700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f410e350 con 0x7fe5f4103630 2024-05-03T14:39:56.162 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:39:56.157+0000 7fe5f27fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe5f4103630 0x7fe5f4198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:39:56.162 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:39:56.157+0000 7fe5f27fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe5f4103630 0x7fe5f4198c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:37440/0 (socket says 172.21.15.149:37440) 2024-05-03T14:39:56.162 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:39:56.157+0000 7fe5f27fc700 1 -- 172.21.15.149:0/1836974263 learned_addr learned my addr 172.21.15.149:0/1836974263 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-05-03T14:39:56.162 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:39:56.157+0000 7fe5f2ffd700 1 --2- 172.21.15.149:0/1836974263 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5f4102430 0x7fe5f41986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:39:56.162 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:39:56.158+0000 7fe5f27fc700 1 -- 172.21.15.149:0/1836974263 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5f4102430 msgr2=0x7fe5f41986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:39:56.162 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:39:56.158+0000 7fe5f27fc700 1 --2- 172.21.15.149:0/1836974263 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5f4102430 0x7fe5f41986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:39:56.162 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:39:56.158+0000 7fe5f27fc700 1 -- 172.21.15.149:0/1836974263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5f4103fd0 msgr2=0x7fe5f419fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:39:56.162 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:39:56.158+0000 7fe5f27fc700 1 --2- 172.21.15.149:0/1836974263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5f4103fd0 0x7fe5f419fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:39:56.162 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:39:56.158+0000 7fe5f27fc700 1 -- 172.21.15.149:0/1836974263 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5f41a0320 con 0x7fe5f4103630 2024-05-03T14:39:56.162 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:39:56.158+0000 7fe5f27fc700 1 --2- 172.21.15.149:0/1836974263 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe5f4103630 0x7fe5f4198c00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe5e400b560 tx=0x7fe5e400b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:39:56.162 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:39:56.158+0000 7fe5ebfff700 1 -- 172.21.15.149:0/1836974263 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e4015020 con 0x7fe5f4103630 2024-05-03T14:39:56.162 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:39:56.158+0000 7fe5f92fe700 1 -- 172.21.15.149:0/1836974263 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5f406bbd0 con 0x7fe5f4103630 2024-05-03T14:39:56.163 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:39:56.158+0000 7fe5ebfff700 1 -- 172.21.15.149:0/1836974263 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7fe5e4011d30 con 0x7fe5f4103630 2024-05-03T14:39:56.163 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:39:56.158+0000 7fe5ebfff700 1 -- 172.21.15.149:0/1836974263 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e400f2e0 con 0x7fe5f4103630 2024-05-03T14:39:56.163 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:39:56.158+0000 7fe5f92fe700 1 -- 172.21.15.149:0/1836974263 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5f406c1e0 con 0x7fe5f4103630 2024-05-03T14:39:56.163 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:39:56.159+0000 7fe5f92fe700 1 -- 172.21.15.149:0/1836974263 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5f406a190 con 0x7fe5f4103630 2024-05-03T14:39:56.164 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:39:56.161+0000 7fe5ebfff700 1 -- 172.21.15.149:0/1836974263 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fe5e4018020 con 0x7fe5f4103630 2024-05-03T14:39:56.164 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:39:56.161+0000 7fe5ebfff700 1 --2- 172.21.15.149:0/1836974263 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fe5e00b2880 0x7fe5e00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:39:56.164 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:39:56.162+0000 7fe5ebfff700 1 -- 172.21.15.149:0/1836974263 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4101+0+0 (secure 0 0 0) 0x7fe5e40e0f10 con 0x7fe5f4103630 2024-05-03T14:39:56.164 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:39:56.162+0000 7fe5f2ffd700 1 --2- 172.21.15.149:0/1836974263 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fe5e00b2880 0x7fe5e00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:39:56.165 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:39:56.163+0000 7fe5f2ffd700 1 --2- 172.21.15.149:0/1836974263 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fe5e00b2880 0x7fe5e00b4d40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4103490 tx=0x7fe5dc009580 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:39:56.167 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:39:56.166+0000 7fe5ebfff700 1 -- 172.21.15.149:0/1836974263 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fe5e401f020 con 0x7fe5f4103630 2024-05-03T14:39:56.381 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:39:56.379+0000 7fe5f92fe700 1 -- 172.21.15.149:0/1836974263 --> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fe5f406c4a0 con 0x7fe5e00b2880 2024-05-03T14:39:57.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:57 smithi060 ceph-mon[28873]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-05-03T14:39:57.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:57 smithi060 ceph-mon[28873]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:39:57.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:57 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T14:39:57.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:57 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T14:39:57.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:57 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:39:57.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:57 smithi090 ceph-mon[33568]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-05-03T14:39:57.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:57 smithi090 ceph-mon[33568]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:39:57.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:57 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T14:39:57.626 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:57 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T14:39:57.626 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:57 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:39:57.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:57 smithi149 ceph-mon[33332]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-05-03T14:39:57.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:57 smithi149 ceph-mon[33332]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:39:57.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:57 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T14:39:57.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:57 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T14:39:57.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:57 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:39:59.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:59 smithi060 ceph-mon[28873]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-03T14:39:59.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:59 smithi060 ceph-mon[28873]: from='client.? 172.21.15.149:0/1799345652' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aefb78a2-522a-4b35-92b8-e5c04fa47b04"}]: dispatch 2024-05-03T14:39:59.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:59 smithi060 ceph-mon[28873]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aefb78a2-522a-4b35-92b8-e5c04fa47b04"}]: dispatch 2024-05-03T14:39:59.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:59 smithi060 ceph-mon[28873]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aefb78a2-522a-4b35-92b8-e5c04fa47b04"}]': finished 2024-05-03T14:39:59.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:59 smithi060 ceph-mon[28873]: osdmap e33: 6 total, 5 up, 6 in 2024-05-03T14:39:59.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:39:59 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T14:39:59.613 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:59 smithi149 ceph-mon[33332]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-03T14:39:59.614 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:59 smithi149 ceph-mon[33332]: from='client.? 172.21.15.149:0/1799345652' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aefb78a2-522a-4b35-92b8-e5c04fa47b04"}]: dispatch 2024-05-03T14:39:59.614 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:59 smithi149 ceph-mon[33332]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aefb78a2-522a-4b35-92b8-e5c04fa47b04"}]: dispatch 2024-05-03T14:39:59.614 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:59 smithi149 ceph-mon[33332]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aefb78a2-522a-4b35-92b8-e5c04fa47b04"}]': finished 2024-05-03T14:39:59.614 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:59 smithi149 ceph-mon[33332]: osdmap e33: 6 total, 5 up, 6 in 2024-05-03T14:39:59.614 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:39:59 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T14:39:59.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:59 smithi090 ceph-mon[33568]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-03T14:39:59.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:59 smithi090 ceph-mon[33568]: from='client.? 172.21.15.149:0/1799345652' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aefb78a2-522a-4b35-92b8-e5c04fa47b04"}]: dispatch 2024-05-03T14:39:59.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:59 smithi090 ceph-mon[33568]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aefb78a2-522a-4b35-92b8-e5c04fa47b04"}]: dispatch 2024-05-03T14:39:59.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:59 smithi090 ceph-mon[33568]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aefb78a2-522a-4b35-92b8-e5c04fa47b04"}]': finished 2024-05-03T14:39:59.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:59 smithi090 ceph-mon[33568]: osdmap e33: 6 total, 5 up, 6 in 2024-05-03T14:39:59.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:39:59 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T14:40:00.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:00 smithi060 ceph-mon[28873]: from='client.? 172.21.15.149:0/2185600300' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T14:40:00.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:00 smithi060 ceph-mon[28873]: overall HEALTH_OK 2024-05-03T14:40:00.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:00 smithi090 ceph-mon[33568]: from='client.? 172.21.15.149:0/2185600300' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T14:40:00.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:00 smithi090 ceph-mon[33568]: overall HEALTH_OK 2024-05-03T14:40:00.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:00 smithi149 ceph-mon[33332]: from='client.? 172.21.15.149:0/2185600300' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T14:40:00.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:00 smithi149 ceph-mon[33332]: overall HEALTH_OK 2024-05-03T14:40:01.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:01 smithi060 ceph-mon[28873]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-03T14:40:01.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:01 smithi090 ceph-mon[33568]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-03T14:40:01.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:01 smithi149 ceph-mon[33332]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-03T14:40:03.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:03 smithi060 ceph-mon[28873]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-03T14:40:03.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:03 smithi149 ceph-mon[33332]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-03T14:40:03.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:03 smithi090 ceph-mon[33568]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-03T14:40:05.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:05 smithi149 ceph-mon[33332]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-03T14:40:05.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:05 smithi060 ceph-mon[28873]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-03T14:40:05.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:05 smithi090 ceph-mon[33568]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-03T14:40:07.459 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:07 smithi149 ceph-mon[33332]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-03T14:40:07.459 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:07 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-03T14:40:07.459 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:07 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:40:07.459 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:07 smithi149 ceph-mon[33332]: Deploying daemon osd.5 on smithi149 2024-05-03T14:40:07.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:07 smithi060 ceph-mon[28873]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-03T14:40:07.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:07 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-03T14:40:07.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:07 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:40:07.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:07 smithi060 ceph-mon[28873]: Deploying daemon osd.5 on smithi149 2024-05-03T14:40:07.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:07 smithi090 ceph-mon[33568]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-03T14:40:07.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:07 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-03T14:40:07.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:07 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:40:07.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:07 smithi090 ceph-mon[33568]: Deploying daemon osd.5 on smithi149 2024-05-03T14:40:09.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:09 smithi060 ceph-mon[28873]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-03T14:40:09.618 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:09 smithi149 ceph-mon[33332]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-03T14:40:09.624 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:09 smithi090 ceph-mon[33568]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-03T14:40:11.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:11 smithi060 ceph-mon[28873]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-03T14:40:11.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:11 smithi090 ceph-mon[33568]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-03T14:40:11.661 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:11 smithi149 ceph-mon[33332]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-03T14:40:13.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:13 smithi060 ceph-mon[28873]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-03T14:40:13.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:13 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:13.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:13 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:13.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:13 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:40:13.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:13 smithi090 ceph-mon[33568]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-03T14:40:13.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:13 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:13.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:13 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:13.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:13 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:40:13.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:13 smithi149 ceph-mon[33332]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-03T14:40:13.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:13 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:13.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:13 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:13.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:13 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:40:15.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:15 smithi060 ceph-mon[28873]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-03T14:40:15.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:15 smithi090 ceph-mon[33568]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-03T14:40:15.656 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:15 smithi149 ceph-mon[33332]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-03T14:40:16.323 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:40:16.320+0000 7fe5ebfff700 1 -- 172.21.15.149:0/1836974263 <== mgr.14152 v2:172.21.15.60:6800/3066142956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe5f406c4a0 con 0x7fe5e00b2880 2024-05-03T14:40:16.323 INFO:teuthology.orchestra.run.smithi149.stdout:Created osd(s) 5 on host 'smithi149' 2024-05-03T14:40:16.325 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:40:16.322+0000 7fe5f92fe700 1 -- 172.21.15.149:0/1836974263 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fe5e00b2880 msgr2=0x7fe5e00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:40:16.325 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:40:16.322+0000 7fe5f92fe700 1 --2- 172.21.15.149:0/1836974263 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fe5e00b2880 0x7fe5e00b4d40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4103490 tx=0x7fe5dc009580 comp rx=0 tx=0).stop 2024-05-03T14:40:16.325 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:40:16.322+0000 7fe5f92fe700 1 -- 172.21.15.149:0/1836974263 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe5f4103630 msgr2=0x7fe5f4198c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:40:16.326 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:40:16.322+0000 7fe5f92fe700 1 --2- 172.21.15.149:0/1836974263 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe5f4103630 0x7fe5f4198c00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe5e400b560 tx=0x7fe5e400b870 comp rx=0 tx=0).stop 2024-05-03T14:40:16.326 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:40:16.322+0000 7fe5f92fe700 1 -- 172.21.15.149:0/1836974263 shutdown_connections 2024-05-03T14:40:16.326 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:40:16.322+0000 7fe5f92fe700 1 --2- 172.21.15.149:0/1836974263 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fe5e00b2880 0x7fe5e00b4d40 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:40:16.326 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:40:16.322+0000 7fe5f92fe700 1 --2- 172.21.15.149:0/1836974263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5f4103fd0 0x7fe5f419fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:40:16.326 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:40:16.322+0000 7fe5f92fe700 1 --2- 172.21.15.149:0/1836974263 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5f4102430 0x7fe5f41986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:40:16.326 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:40:16.322+0000 7fe5f92fe700 1 --2- 172.21.15.149:0/1836974263 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe5f4103630 0x7fe5f4198c00 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:40:16.326 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:40:16.322+0000 7fe5f92fe700 1 -- 172.21.15.149:0/1836974263 >> 172.21.15.149:0/1836974263 conn(0x7fe5f40fd9f0 msgr2=0x7fe5f4106860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:40:16.326 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:40:16.322+0000 7fe5f92fe700 1 -- 172.21.15.149:0/1836974263 shutdown_connections 2024-05-03T14:40:16.326 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:40:16.322+0000 7fe5f92fe700 1 -- 172.21.15.149:0/1836974263 wait complete. 2024-05-03T14:40:16.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:16 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:16.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:16 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:16.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:16 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:40:16.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:16 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:40:16.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:16 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:16.626 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:16 smithi090 ceph-mon[33568]: from='osd.5 [v2:172.21.15.149:6808/2139732836,v1:172.21.15.149:6809/2139732836]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-03T14:40:16.626 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:16 smithi090 ceph-mon[33568]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-03T14:40:16.626 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:16 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:16.626 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:16 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:16.626 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:16 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:40:16.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:16 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:16.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:16 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:16.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:16 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:40:16.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:16 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:40:16.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:16 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:16.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:16 smithi149 ceph-mon[33332]: from='osd.5 [v2:172.21.15.149:6808/2139732836,v1:172.21.15.149:6809/2139732836]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-03T14:40:16.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:16 smithi149 ceph-mon[33332]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-03T14:40:16.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:16 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:16.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:16 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:16.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:16 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:40:16.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:16 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:16.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:16 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:16.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:16 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:40:16.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:16 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:40:16.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:16 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:16.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:16 smithi060 ceph-mon[28873]: from='osd.5 [v2:172.21.15.149:6808/2139732836,v1:172.21.15.149:6809/2139732836]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-03T14:40:16.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:16 smithi060 ceph-mon[28873]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-03T14:40:16.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:16 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:16.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:16 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:16.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:16 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:40:16.987 DEBUG:teuthology.orchestra.run.smithi149:osd.5> sudo journalctl -f -n 0 -u ceph-285d64b2-095a-11ef-bc97-c7b262605968@osd.5.service 2024-05-03T14:40:16.989 INFO:tasks.cephadm:Deploying osd.6 on smithi060 with /dev/vg_nvme/lv_2... 2024-05-03T14:40:16.989 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-03T14:40:17.458 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:40:17.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:17 smithi149 ceph-mon[33332]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-03T14:40:17.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:17 smithi149 ceph-mon[33332]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-03T14:40:17.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:17 smithi149 ceph-mon[33332]: from='osd.5 [v2:172.21.15.149:6808/2139732836,v1:172.21.15.149:6809/2139732836]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-03T14:40:17.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:17 smithi149 ceph-mon[33332]: osdmap e34: 6 total, 5 up, 6 in 2024-05-03T14:40:17.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:17 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T14:40:17.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:17 smithi149 ceph-mon[33332]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-03T14:40:17.807 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:17 smithi060 ceph-mon[28873]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-03T14:40:17.807 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:17 smithi060 ceph-mon[28873]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-03T14:40:17.807 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:17 smithi060 ceph-mon[28873]: from='osd.5 [v2:172.21.15.149:6808/2139732836,v1:172.21.15.149:6809/2139732836]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-03T14:40:17.807 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:17 smithi060 ceph-mon[28873]: osdmap e34: 6 total, 5 up, 6 in 2024-05-03T14:40:17.807 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:17 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T14:40:17.808 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:17 smithi060 ceph-mon[28873]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-03T14:40:17.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:17 smithi090 ceph-mon[33568]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-03T14:40:17.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:17 smithi090 ceph-mon[33568]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-03T14:40:17.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:17 smithi090 ceph-mon[33568]: from='osd.5 [v2:172.21.15.149:6808/2139732836,v1:172.21.15.149:6809/2139732836]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-03T14:40:17.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:17 smithi090 ceph-mon[33568]: osdmap e34: 6 total, 5 up, 6 in 2024-05-03T14:40:17.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:17 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T14:40:17.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:17 smithi090 ceph-mon[33568]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-03T14:40:18.174 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 03 14:40:17 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-5[45461]: 2024-05-03T14:40:17.842+0000 7f50eaf89700 -1 osd.5 0 waiting for initial osdmap 2024-05-03T14:40:18.174 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 03 14:40:17 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-5[45461]: 2024-05-03T14:40:17.849+0000 7f50e2d74700 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-03T14:40:18.450 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:18 smithi149 ceph-mon[33332]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-05-03T14:40:18.450 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:18 smithi149 ceph-mon[33332]: osdmap e35: 6 total, 5 up, 6 in 2024-05-03T14:40:18.450 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:18 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T14:40:18.450 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:18 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T14:40:18.451 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:18 smithi149 ceph-mon[33332]: from='osd.5 ' entity='osd.5' 2024-05-03T14:40:18.577 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:18 smithi060 ceph-mon[28873]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-05-03T14:40:18.577 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:18 smithi060 ceph-mon[28873]: osdmap e35: 6 total, 5 up, 6 in 2024-05-03T14:40:18.577 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:18 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T14:40:18.577 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:18 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T14:40:18.578 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:18 smithi060 ceph-mon[28873]: from='osd.5 ' entity='osd.5' 2024-05-03T14:40:18.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:18 smithi090 ceph-mon[33568]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-05-03T14:40:18.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:18 smithi090 ceph-mon[33568]: osdmap e35: 6 total, 5 up, 6 in 2024-05-03T14:40:18.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:18 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T14:40:18.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:18 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T14:40:18.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:18 smithi090 ceph-mon[33568]: from='osd.5 ' entity='osd.5' 2024-05-03T14:40:19.638 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:19 smithi060 ceph-mon[28873]: purged_snaps scrub starts 2024-05-03T14:40:19.638 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:19 smithi060 ceph-mon[28873]: purged_snaps scrub ok 2024-05-03T14:40:19.638 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:19 smithi060 ceph-mon[28873]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-03T14:40:19.638 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:19 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T14:40:19.638 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:19 smithi060 ceph-mon[28873]: osd.5 [v2:172.21.15.149:6808/2139732836,v1:172.21.15.149:6809/2139732836] boot 2024-05-03T14:40:19.638 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:19 smithi060 ceph-mon[28873]: osdmap e36: 6 total, 6 up, 6 in 2024-05-03T14:40:19.638 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:19 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T14:40:19.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:19 smithi149 ceph-mon[33332]: purged_snaps scrub starts 2024-05-03T14:40:19.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:19 smithi149 ceph-mon[33332]: purged_snaps scrub ok 2024-05-03T14:40:19.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:19 smithi149 ceph-mon[33332]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-03T14:40:19.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:19 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T14:40:19.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:19 smithi149 ceph-mon[33332]: osd.5 [v2:172.21.15.149:6808/2139732836,v1:172.21.15.149:6809/2139732836] boot 2024-05-03T14:40:19.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:19 smithi149 ceph-mon[33332]: osdmap e36: 6 total, 6 up, 6 in 2024-05-03T14:40:19.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:19 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T14:40:19.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:19 smithi090 ceph-mon[33568]: purged_snaps scrub starts 2024-05-03T14:40:19.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:19 smithi090 ceph-mon[33568]: purged_snaps scrub ok 2024-05-03T14:40:19.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:19 smithi090 ceph-mon[33568]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-03T14:40:19.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:19 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T14:40:19.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:19 smithi090 ceph-mon[33568]: osd.5 [v2:172.21.15.149:6808/2139732836,v1:172.21.15.149:6809/2139732836] boot 2024-05-03T14:40:19.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:19 smithi090 ceph-mon[33568]: osdmap e36: 6 total, 6 up, 6 in 2024-05-03T14:40:19.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:19 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T14:40:20.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:20 smithi149 ceph-mon[33332]: osdmap e37: 6 total, 6 up, 6 in 2024-05-03T14:40:20.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:20 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:20.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:20 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:20.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:20 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:40:20.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:20 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:40:20.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:20 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:20.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:20 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:40:20.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:20 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:40:20.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:20 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:20.733 INFO:teuthology.orchestra.run.smithi060.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-03T14:40:20.733 INFO:teuthology.orchestra.run.smithi060.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-03T14:40:20.733 INFO:teuthology.orchestra.run.smithi060.stderr: stderr: 10+0 records in 2024-05-03T14:40:20.734 INFO:teuthology.orchestra.run.smithi060.stderr:10+0 records out 2024-05-03T14:40:20.734 INFO:teuthology.orchestra.run.smithi060.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0324412 s, 323 MB/s 2024-05-03T14:40:20.734 INFO:teuthology.orchestra.run.smithi060.stderr:--> Zapping successful for: 2024-05-03T14:40:20.734 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:20 smithi060 ceph-mon[28873]: osdmap e37: 6 total, 6 up, 6 in 2024-05-03T14:40:20.734 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:20 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:20.734 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:20 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:20.734 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:20 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:40:20.734 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:20 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:40:20.734 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:20 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:20.734 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:20 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:40:20.734 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:20 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:40:20.735 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:20 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:20.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:20 smithi090 ceph-mon[33568]: osdmap e37: 6 total, 6 up, 6 in 2024-05-03T14:40:20.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:20 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:20.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:20 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:20.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:20 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:40:20.876 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:20 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:40:20.876 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:20 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:20.876 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:20 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:40:20.876 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:20 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:40:20.876 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:20 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:21.684 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi060:vg_nvme/lv_2 2024-05-03T14:40:21.767 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:21 smithi060 ceph-mon[28873]: pgmap v155: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-03T14:40:21.767 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:21 smithi060 ceph-mon[28873]: Detected new or changed devices on smithi149 2024-05-03T14:40:21.767 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:21 smithi060 ceph-mon[28873]: Adjusting osd_memory_target on smithi149 to 2908M 2024-05-03T14:40:21.767 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:21 smithi060 ceph-mon[28873]: osdmap e38: 6 total, 6 up, 6 in 2024-05-03T14:40:21.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:21 smithi090 ceph-mon[33568]: pgmap v155: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-03T14:40:21.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:21 smithi090 ceph-mon[33568]: Detected new or changed devices on smithi149 2024-05-03T14:40:21.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:21 smithi090 ceph-mon[33568]: Adjusting osd_memory_target on smithi149 to 2908M 2024-05-03T14:40:21.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:21 smithi090 ceph-mon[33568]: osdmap e38: 6 total, 6 up, 6 in 2024-05-03T14:40:21.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:21 smithi149 ceph-mon[33332]: pgmap v155: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-03T14:40:21.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:21 smithi149 ceph-mon[33332]: Detected new or changed devices on smithi149 2024-05-03T14:40:21.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:21 smithi149 ceph-mon[33332]: Adjusting osd_memory_target on smithi149 to 2908M 2024-05-03T14:40:21.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:21 smithi149 ceph-mon[33332]: osdmap e38: 6 total, 6 up, 6 in 2024-05-03T14:40:22.142 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:40:23.804 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:23 smithi060 ceph-mon[28873]: pgmap v157: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-03T14:40:23.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:23 smithi090 ceph-mon[33568]: pgmap v157: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-03T14:40:23.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:23 smithi149 ceph-mon[33332]: pgmap v157: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-03T14:40:24.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:40:24.342+0000 7f1197200700 1 -- 172.21.15.60:0/42594784 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11980774d0 msgr2=0x7f1198075930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:40:24.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:40:24.342+0000 7f1197200700 1 --2- 172.21.15.60:0/42594784 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11980774d0 0x7f1198075930 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1188009990 tx=0x7f1188009ca0 comp rx=0 tx=0).stop 2024-05-03T14:40:24.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:40:24.342+0000 7f1197200700 1 -- 172.21.15.60:0/42594784 shutdown_connections 2024-05-03T14:40:24.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:40:24.342+0000 7f1197200700 1 --2- 172.21.15.60:0/42594784 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f119810fa20 0x7f1198111e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:40:24.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:40:24.342+0000 7f1197200700 1 --2- 172.21.15.60:0/42594784 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11980774d0 0x7f1198075930 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:40:24.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:40:24.342+0000 7f1197200700 1 --2- 172.21.15.60:0/42594784 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1198075f00 0x7f1198076380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:40:24.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:40:24.342+0000 7f1197200700 1 -- 172.21.15.60:0/42594784 >> 172.21.15.60:0/42594784 conn(0x7f11980fe9e0 msgr2=0x7f1198100e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:40:24.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:40:24.342+0000 7f1197200700 1 -- 172.21.15.60:0/42594784 shutdown_connections 2024-05-03T14:40:24.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:40:24.343+0000 7f1197200700 1 -- 172.21.15.60:0/42594784 wait complete. 2024-05-03T14:40:24.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:40:24.344+0000 7f1197200700 1 Processor -- start 2024-05-03T14:40:24.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:40:24.344+0000 7f1197200700 1 -- start start 2024-05-03T14:40:24.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:40:24.345+0000 7f1197200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1198075f00 0x7f119819fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:40:24.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:40:24.345+0000 7f1197200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f11980774d0 0x7f11981a0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:40:24.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:40:24.345+0000 7f1197200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f119810fa20 0x7f11981a72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:40:24.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:40:24.345+0000 7f1197200700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f1198115740 con 0x7f1198075f00 2024-05-03T14:40:24.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:40:24.345+0000 7f1197200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f11981155c0 con 0x7f119810fa20 2024-05-03T14:40:24.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:40:24.345+0000 7f1197200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f11981158c0 con 0x7f11980774d0 2024-05-03T14:40:24.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:40:24.345+0000 7f1195400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f11980774d0 0x7f11981a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:40:24.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:40:24.345+0000 7f1195e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1198075f00 0x7f119819fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:40:24.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:40:24.345+0000 7f1195400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f11980774d0 0x7f11981a0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.60:44338/0 (socket says 172.21.15.60:44338) 2024-05-03T14:40:24.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:40:24.345+0000 7f1195400700 1 -- 172.21.15.60:0/3624308052 learned_addr learned my addr 172.21.15.60:0/3624308052 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:40:24.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:40:24.345+0000 7f1196800700 1 --2- 172.21.15.60:0/3624308052 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f119810fa20 0x7f11981a72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:40:24.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:40:24.345+0000 7f1195400700 1 -- 172.21.15.60:0/3624308052 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f119810fa20 msgr2=0x7f11981a72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:40:24.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:40:24.345+0000 7f1195400700 1 --2- 172.21.15.60:0/3624308052 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f119810fa20 0x7f11981a72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:40:24.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:40:24.345+0000 7f1195400700 1 -- 172.21.15.60:0/3624308052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1198075f00 msgr2=0x7f119819fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:40:24.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:40:24.345+0000 7f1195400700 1 --2- 172.21.15.60:0/3624308052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1198075f00 0x7f119819fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:40:24.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:40:24.346+0000 7f1195400700 1 -- 172.21.15.60:0/3624308052 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11981a7990 con 0x7f11980774d0 2024-05-03T14:40:24.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:40:24.346+0000 7f1195400700 1 --2- 172.21.15.60:0/3624308052 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f11980774d0 0x7f11981a0280 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f118400ea50 tx=0x7f118400ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:40:24.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:40:24.346+0000 7f118ea00700 1 -- 172.21.15.60:0/3624308052 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118400cbd0 con 0x7f11980774d0 2024-05-03T14:40:24.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:40:24.346+0000 7f1197200700 1 -- 172.21.15.60:0/3624308052 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11981a7c80 con 0x7f11980774d0 2024-05-03T14:40:24.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:40:24.346+0000 7f118ea00700 1 -- 172.21.15.60:0/3624308052 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f118400cd70 con 0x7f11980774d0 2024-05-03T14:40:24.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:40:24.346+0000 7f118ea00700 1 -- 172.21.15.60:0/3624308052 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11840187a0 con 0x7f11980774d0 2024-05-03T14:40:24.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:40:24.347+0000 7f1197200700 1 -- 172.21.15.60:0/3624308052 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11981a81c0 con 0x7f11980774d0 2024-05-03T14:40:24.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:40:24.348+0000 7f1197200700 1 -- 172.21.15.60:0/3624308052 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f119806a190 con 0x7f11980774d0 2024-05-03T14:40:24.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:40:24.351+0000 7f118ea00700 1 -- 172.21.15.60:0/3624308052 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f118401f020 con 0x7f11980774d0 2024-05-03T14:40:24.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:40:24.352+0000 7f118ea00700 1 --2- 172.21.15.60:0/3624308052 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f117c0b2900 0x7f117c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:40:24.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:40:24.352+0000 7f118ea00700 1 -- 172.21.15.60:0/3624308052 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4533+0+0 (secure 0 0 0) 0x7f11840e1320 con 0x7f11980774d0 2024-05-03T14:40:24.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:40:24.353+0000 7f1195e00700 1 --2- 172.21.15.60:0/3624308052 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f117c0b2900 0x7f117c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:40:24.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:40:24.353+0000 7f1195e00700 1 --2- 172.21.15.60:0/3624308052 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f117c0b2900 0x7f117c0b4dc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f11981a1220 tx=0x7f118800b8c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:40:24.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:40:24.357+0000 7f118ea00700 1 -- 172.21.15.60:0/3624308052 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f11840a9e60 con 0x7f11980774d0 2024-05-03T14:40:24.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:40:24.608+0000 7f1197200700 1 -- 172.21.15.60:0/3624308052 --> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f11980638e0 con 0x7f117c0b2900 2024-05-03T14:40:25.740 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:25 smithi060 ceph-mon[28873]: pgmap v158: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-03T14:40:25.740 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:25 smithi060 ceph-mon[28873]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:40:25.740 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:25 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T14:40:25.740 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:25 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T14:40:25.740 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:25 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:40:25.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:25 smithi090 ceph-mon[33568]: pgmap v158: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-03T14:40:25.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:25 smithi090 ceph-mon[33568]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:40:25.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:25 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T14:40:25.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:25 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T14:40:25.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:25 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:40:25.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:25 smithi149 ceph-mon[33332]: pgmap v158: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-03T14:40:25.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:25 smithi149 ceph-mon[33332]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:40:25.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:25 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T14:40:25.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:25 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T14:40:25.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:25 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:40:27.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:27 smithi060 ceph-mon[28873]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-05-03T14:40:27.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:27 smithi090 ceph-mon[33568]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-05-03T14:40:27.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:27 smithi149 ceph-mon[33332]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-05-03T14:40:29.767 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:29 smithi149 ceph-mon[33332]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-05-03T14:40:29.767 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:29 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/3204865777' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7c887819-6182-4636-8cfd-d845c30bd13c"}]: dispatch 2024-05-03T14:40:29.767 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:29 smithi149 ceph-mon[33332]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7c887819-6182-4636-8cfd-d845c30bd13c"}]: dispatch 2024-05-03T14:40:29.767 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:29 smithi149 ceph-mon[33332]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7c887819-6182-4636-8cfd-d845c30bd13c"}]': finished 2024-05-03T14:40:29.767 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:29 smithi149 ceph-mon[33332]: osdmap e39: 7 total, 6 up, 7 in 2024-05-03T14:40:29.767 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:29 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T14:40:29.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:29 smithi060 ceph-mon[28873]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-05-03T14:40:29.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:29 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/3204865777' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7c887819-6182-4636-8cfd-d845c30bd13c"}]: dispatch 2024-05-03T14:40:29.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:29 smithi060 ceph-mon[28873]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7c887819-6182-4636-8cfd-d845c30bd13c"}]: dispatch 2024-05-03T14:40:29.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:29 smithi060 ceph-mon[28873]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7c887819-6182-4636-8cfd-d845c30bd13c"}]': finished 2024-05-03T14:40:29.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:29 smithi060 ceph-mon[28873]: osdmap e39: 7 total, 6 up, 7 in 2024-05-03T14:40:29.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:29 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T14:40:29.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:29 smithi090 ceph-mon[33568]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-05-03T14:40:29.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:29 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/3204865777' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7c887819-6182-4636-8cfd-d845c30bd13c"}]: dispatch 2024-05-03T14:40:29.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:29 smithi090 ceph-mon[33568]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7c887819-6182-4636-8cfd-d845c30bd13c"}]: dispatch 2024-05-03T14:40:29.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:29 smithi090 ceph-mon[33568]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7c887819-6182-4636-8cfd-d845c30bd13c"}]': finished 2024-05-03T14:40:29.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:29 smithi090 ceph-mon[33568]: osdmap e39: 7 total, 6 up, 7 in 2024-05-03T14:40:29.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:29 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T14:40:30.740 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:30 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/2716420770' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T14:40:30.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:30 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/2716420770' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T14:40:30.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:30 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/2716420770' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T14:40:31.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:31 smithi060 ceph-mon[28873]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-03T14:40:31.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:31 smithi090 ceph-mon[33568]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-03T14:40:31.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:31 smithi149 ceph-mon[33332]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-03T14:40:33.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:33 smithi060 ceph-mon[28873]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T14:40:33.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:33 smithi090 ceph-mon[33568]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T14:40:33.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:33 smithi149 ceph-mon[33332]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T14:40:35.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:35 smithi060 ceph-mon[28873]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T14:40:35.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:35 smithi090 ceph-mon[33568]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T14:40:35.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:35 smithi149 ceph-mon[33332]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T14:40:37.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:37 smithi060 ceph-mon[28873]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-03T14:40:37.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:37 smithi090 ceph-mon[33568]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-03T14:40:37.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:37 smithi149 ceph-mon[33332]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-03T14:40:38.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:38 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-03T14:40:38.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:38 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:40:38.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:38 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-03T14:40:38.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:38 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:40:38.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:38 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-03T14:40:38.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:38 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:40:39.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:39 smithi060 ceph-mon[28873]: Deploying daemon osd.6 on smithi060 2024-05-03T14:40:39.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:39 smithi060 ceph-mon[28873]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-03T14:40:39.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:39 smithi090 ceph-mon[33568]: Deploying daemon osd.6 on smithi060 2024-05-03T14:40:39.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:39 smithi090 ceph-mon[33568]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-03T14:40:39.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:39 smithi149 ceph-mon[33332]: Deploying daemon osd.6 on smithi060 2024-05-03T14:40:39.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:39 smithi149 ceph-mon[33332]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-03T14:40:41.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:41 smithi060 ceph-mon[28873]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-03T14:40:41.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:41 smithi090 ceph-mon[33568]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-03T14:40:41.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:41 smithi149 ceph-mon[33332]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-03T14:40:43.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:43 smithi060 ceph-mon[28873]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-03T14:40:43.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:43 smithi090 ceph-mon[33568]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-03T14:40:43.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:43 smithi149 ceph-mon[33332]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-03T14:40:45.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:45 smithi060 ceph-mon[28873]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-03T14:40:45.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:45 smithi090 ceph-mon[33568]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-03T14:40:45.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:45 smithi149 ceph-mon[33332]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-03T14:40:47.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:47 smithi060 ceph-mon[28873]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-03T14:40:47.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:47 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:47.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:47 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:47.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:47 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:40:47.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:47 smithi090 ceph-mon[33568]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-03T14:40:47.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:47 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:47.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:47 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:47.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:47 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:40:47.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:47 smithi149 ceph-mon[33332]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-03T14:40:47.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:47 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:47.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:47 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:47.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:47 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:40:49.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:49 smithi060 ceph-mon[28873]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-03T14:40:49.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:49 smithi090 ceph-mon[33568]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-03T14:40:49.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:49 smithi149 ceph-mon[33332]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-03T14:40:51.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:51 smithi060 ceph-mon[28873]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-03T14:40:51.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:51 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:51.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:51 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:51.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:51 smithi060 ceph-mon[28873]: from='osd.6 [v2:172.21.15.60:6818/3738521002,v1:172.21.15.60:6819/3738521002]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-03T14:40:51.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:51 smithi060 ceph-mon[28873]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-03T14:40:51.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:51 smithi090 ceph-mon[33568]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-03T14:40:51.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:51 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:51.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:51 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:51.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:51 smithi090 ceph-mon[33568]: from='osd.6 [v2:172.21.15.60:6818/3738521002,v1:172.21.15.60:6819/3738521002]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-03T14:40:51.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:51 smithi090 ceph-mon[33568]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-03T14:40:51.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:51 smithi149 ceph-mon[33332]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-03T14:40:51.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:51 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:51.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:51 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:51.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:51 smithi149 ceph-mon[33332]: from='osd.6 [v2:172.21.15.60:6818/3738521002,v1:172.21.15.60:6819/3738521002]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-03T14:40:51.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:51 smithi149 ceph-mon[33332]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-03T14:40:52.062 INFO:teuthology.orchestra.run.smithi060.stdout:Created osd(s) 6 on host 'smithi060' 2024-05-03T14:40:52.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:40:52.058+0000 7f118ea00700 1 -- 172.21.15.60:0/3624308052 <== mgr.14152 v2:172.21.15.60:6800/3066142956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f11980638e0 con 0x7f117c0b2900 2024-05-03T14:40:52.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:40:52.062+0000 7f1197200700 1 -- 172.21.15.60:0/3624308052 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f117c0b2900 msgr2=0x7f117c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:40:52.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:40:52.062+0000 7f1197200700 1 --2- 172.21.15.60:0/3624308052 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f117c0b2900 0x7f117c0b4dc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f11981a1220 tx=0x7f118800b8c0 comp rx=0 tx=0).stop 2024-05-03T14:40:52.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:40:52.062+0000 7f1197200700 1 -- 172.21.15.60:0/3624308052 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f11980774d0 msgr2=0x7f11981a0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:40:52.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:40:52.062+0000 7f1197200700 1 --2- 172.21.15.60:0/3624308052 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f11980774d0 0x7f11981a0280 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f118400ea50 tx=0x7f118400ed60 comp rx=0 tx=0).stop 2024-05-03T14:40:52.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:40:52.062+0000 7f1197200700 1 -- 172.21.15.60:0/3624308052 shutdown_connections 2024-05-03T14:40:52.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:40:52.062+0000 7f1197200700 1 --2- 172.21.15.60:0/3624308052 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f117c0b2900 0x7f117c0b4dc0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:40:52.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:40:52.062+0000 7f1197200700 1 --2- 172.21.15.60:0/3624308052 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f119810fa20 0x7f11981a72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:40:52.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:40:52.062+0000 7f1197200700 1 --2- 172.21.15.60:0/3624308052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1198075f00 0x7f119819fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:40:52.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:40:52.062+0000 7f1197200700 1 --2- 172.21.15.60:0/3624308052 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f11980774d0 0x7f11981a0280 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:40:52.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:40:52.062+0000 7f1197200700 1 -- 172.21.15.60:0/3624308052 >> 172.21.15.60:0/3624308052 conn(0x7f11980fe9e0 msgr2=0x7f1198102dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:40:52.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:40:52.062+0000 7f1197200700 1 -- 172.21.15.60:0/3624308052 shutdown_connections 2024-05-03T14:40:52.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:40:52.062+0000 7f1197200700 1 -- 172.21.15.60:0/3624308052 wait complete. 2024-05-03T14:40:52.836 DEBUG:teuthology.orchestra.run.smithi060:osd.6> sudo journalctl -f -n 0 -u ceph-285d64b2-095a-11ef-bc97-c7b262605968@osd.6.service 2024-05-03T14:40:52.841 INFO:tasks.cephadm:Deploying osd.7 on smithi090 with /dev/vg_nvme/lv_2... 2024-05-03T14:40:52.841 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-03T14:40:53.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:52 smithi060 ceph-mon[28873]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-03T14:40:53.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:52 smithi060 ceph-mon[28873]: osdmap e40: 7 total, 6 up, 7 in 2024-05-03T14:40:53.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:52 smithi060 ceph-mon[28873]: from='osd.6 [v2:172.21.15.60:6818/3738521002,v1:172.21.15.60:6819/3738521002]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-05-03T14:40:53.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:52 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T14:40:53.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:52 smithi060 ceph-mon[28873]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-05-03T14:40:53.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:52 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:53.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:52 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:53.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:52 smithi060 ceph-mon[28873]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-03T14:40:53.088 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.b/config 2024-05-03T14:40:53.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:52 smithi090 ceph-mon[33568]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-03T14:40:53.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:52 smithi090 ceph-mon[33568]: osdmap e40: 7 total, 6 up, 7 in 2024-05-03T14:40:53.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:52 smithi090 ceph-mon[33568]: from='osd.6 [v2:172.21.15.60:6818/3738521002,v1:172.21.15.60:6819/3738521002]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-05-03T14:40:53.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:52 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T14:40:53.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:52 smithi090 ceph-mon[33568]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-05-03T14:40:53.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:52 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:53.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:52 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:53.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:52 smithi090 ceph-mon[33568]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-03T14:40:53.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:52 smithi149 ceph-mon[33332]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-03T14:40:53.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:52 smithi149 ceph-mon[33332]: osdmap e40: 7 total, 6 up, 7 in 2024-05-03T14:40:53.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:52 smithi149 ceph-mon[33332]: from='osd.6 [v2:172.21.15.60:6818/3738521002,v1:172.21.15.60:6819/3738521002]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-05-03T14:40:53.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:52 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T14:40:53.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:52 smithi149 ceph-mon[33332]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-05-03T14:40:53.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:52 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:53.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:52 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:53.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:52 smithi149 ceph-mon[33332]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-03T14:40:53.786 INFO:journalctl@ceph.osd.6.smithi060.stdout:May 03 14:40:53 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-6[54395]: 2024-05-03T14:40:53.392+0000 7f6dcd000700 -1 osd.6 0 waiting for initial osdmap 2024-05-03T14:40:53.787 INFO:journalctl@ceph.osd.6.smithi060.stdout:May 03 14:40:53 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-6[54395]: 2024-05-03T14:40:53.430+0000 7f6dc2e00700 -1 osd.6 41 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-03T14:40:53.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:53 smithi090 ceph-mon[33568]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-05-03T14:40:53.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:53 smithi090 ceph-mon[33568]: osdmap e41: 7 total, 6 up, 7 in 2024-05-03T14:40:53.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:53 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T14:40:53.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:53 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T14:40:53.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:53 smithi090 ceph-mon[33568]: from='osd.6 ' entity='osd.6' 2024-05-03T14:40:54.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:53 smithi060 ceph-mon[28873]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-05-03T14:40:54.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:53 smithi060 ceph-mon[28873]: osdmap e41: 7 total, 6 up, 7 in 2024-05-03T14:40:54.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:53 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T14:40:54.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:53 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T14:40:54.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:53 smithi060 ceph-mon[28873]: from='osd.6 ' entity='osd.6' 2024-05-03T14:40:54.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:53 smithi149 ceph-mon[33332]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-05-03T14:40:54.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:53 smithi149 ceph-mon[33332]: osdmap e41: 7 total, 6 up, 7 in 2024-05-03T14:40:54.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:53 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T14:40:54.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:53 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T14:40:54.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:53 smithi149 ceph-mon[33332]: from='osd.6 ' entity='osd.6' 2024-05-03T14:40:55.021 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-03T14:40:55.021 INFO:teuthology.orchestra.run.smithi090.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-03T14:40:55.021 INFO:teuthology.orchestra.run.smithi090.stderr: stderr: 10+0 records in 2024-05-03T14:40:55.021 INFO:teuthology.orchestra.run.smithi090.stderr:10+0 records out 2024-05-03T14:40:55.021 INFO:teuthology.orchestra.run.smithi090.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0124845 s, 840 MB/s 2024-05-03T14:40:55.021 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping successful for: 2024-05-03T14:40:55.076 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:54 smithi090 ceph-mon[33568]: purged_snaps scrub starts 2024-05-03T14:40:55.076 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:54 smithi090 ceph-mon[33568]: purged_snaps scrub ok 2024-05-03T14:40:55.076 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:54 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T14:40:55.076 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:54 smithi090 ceph-mon[33568]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-03T14:40:55.076 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:54 smithi090 ceph-mon[33568]: osd.6 [v2:172.21.15.60:6818/3738521002,v1:172.21.15.60:6819/3738521002] boot 2024-05-03T14:40:55.076 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:54 smithi090 ceph-mon[33568]: osdmap e42: 7 total, 7 up, 7 in 2024-05-03T14:40:55.077 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:54 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T14:40:55.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:54 smithi060 ceph-mon[28873]: purged_snaps scrub starts 2024-05-03T14:40:55.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:54 smithi060 ceph-mon[28873]: purged_snaps scrub ok 2024-05-03T14:40:55.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:54 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T14:40:55.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:54 smithi060 ceph-mon[28873]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-03T14:40:55.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:54 smithi060 ceph-mon[28873]: osd.6 [v2:172.21.15.60:6818/3738521002,v1:172.21.15.60:6819/3738521002] boot 2024-05-03T14:40:55.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:54 smithi060 ceph-mon[28873]: osdmap e42: 7 total, 7 up, 7 in 2024-05-03T14:40:55.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:54 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T14:40:55.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:54 smithi149 ceph-mon[33332]: purged_snaps scrub starts 2024-05-03T14:40:55.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:54 smithi149 ceph-mon[33332]: purged_snaps scrub ok 2024-05-03T14:40:55.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:54 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T14:40:55.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:54 smithi149 ceph-mon[33332]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-03T14:40:55.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:54 smithi149 ceph-mon[33332]: osd.6 [v2:172.21.15.60:6818/3738521002,v1:172.21.15.60:6819/3738521002] boot 2024-05-03T14:40:55.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:54 smithi149 ceph-mon[33332]: osdmap e42: 7 total, 7 up, 7 in 2024-05-03T14:40:55.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:54 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T14:40:55.676 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi090:vg_nvme/lv_2 2024-05-03T14:40:55.913 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.b/config 2024-05-03T14:40:57.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:56 smithi060 ceph-mon[28873]: osdmap e43: 7 total, 7 up, 7 in 2024-05-03T14:40:57.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:56 smithi060 ceph-mon[28873]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-03T14:40:57.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:56 smithi090 ceph-mon[33568]: osdmap e43: 7 total, 7 up, 7 in 2024-05-03T14:40:57.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:56 smithi090 ceph-mon[33568]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-03T14:40:57.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:56 smithi149 ceph-mon[33332]: osdmap e43: 7 total, 7 up, 7 in 2024-05-03T14:40:57.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:56 smithi149 ceph-mon[33332]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-03T14:40:57.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:40:57.712+0000 7f266bfcc700 1 -- 172.21.15.90:0/4292904753 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2664103fc0 msgr2=0x7f266410a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:40:57.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:40:57.712+0000 7f266bfcc700 1 --2- 172.21.15.90:0/4292904753 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2664103fc0 0x7f266410a870 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f265c00b0d0 tx=0x7f265c00b3e0 comp rx=0 tx=0).stop 2024-05-03T14:40:57.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:40:57.712+0000 7f266bfcc700 1 -- 172.21.15.90:0/4292904753 shutdown_connections 2024-05-03T14:40:57.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:40:57.712+0000 7f266bfcc700 1 --2- 172.21.15.90:0/4292904753 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2664103fc0 0x7f266410a870 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:40:57.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:40:57.712+0000 7f266bfcc700 1 --2- 172.21.15.90:0/4292904753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2664102420 0x7f2664102840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:40:57.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:40:57.712+0000 7f266bfcc700 1 --2- 172.21.15.90:0/4292904753 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2664103620 0x7f2664103a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:40:57.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:40:57.712+0000 7f266bfcc700 1 -- 172.21.15.90:0/4292904753 >> 172.21.15.90:0/4292904753 conn(0x7f26640fd9c0 msgr2=0x7f26640ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:40:57.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:40:57.712+0000 7f266bfcc700 1 -- 172.21.15.90:0/4292904753 shutdown_connections 2024-05-03T14:40:57.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:40:57.712+0000 7f266bfcc700 1 -- 172.21.15.90:0/4292904753 wait complete. 2024-05-03T14:40:57.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:40:57.713+0000 7f266bfcc700 1 Processor -- start 2024-05-03T14:40:57.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:40:57.713+0000 7f266bfcc700 1 -- start start 2024-05-03T14:40:57.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:40:57.713+0000 7f266bfcc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2664102420 0x7f266406c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:40:57.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:40:57.713+0000 7f266bfcc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2664103620 0x7f266419e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:40:57.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:40:57.713+0000 7f266bfcc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2664103fc0 0x7f26641a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:40:57.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:40:57.713+0000 7f266bfcc700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f266410e1a0 con 0x7f2664103620 2024-05-03T14:40:57.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:40:57.713+0000 7f266bfcc700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f266410e020 con 0x7f2664102420 2024-05-03T14:40:57.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:40:57.713+0000 7f266bfcc700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f266410e320 con 0x7f2664103fc0 2024-05-03T14:40:57.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:40:57.713+0000 7f2669d68700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2664102420 0x7f266406c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:40:57.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:40:57.713+0000 7f2669d68700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2664102420 0x7f266406c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39226/0 (socket says 172.21.15.90:39226) 2024-05-03T14:40:57.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:40:57.713+0000 7f2669d68700 1 -- 172.21.15.90:0/1284864635 learned_addr learned my addr 172.21.15.90:0/1284864635 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-03T14:40:57.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:40:57.714+0000 7f266a569700 1 --2- 172.21.15.90:0/1284864635 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2664103fc0 0x7f26641a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:40:57.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:40:57.714+0000 7f2669d68700 1 -- 172.21.15.90:0/1284864635 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2664103fc0 msgr2=0x7f26641a0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:40:57.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:40:57.714+0000 7f2669d68700 1 --2- 172.21.15.90:0/1284864635 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2664103fc0 0x7f26641a0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:40:57.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:40:57.714+0000 7f2669d68700 1 -- 172.21.15.90:0/1284864635 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2664103620 msgr2=0x7f266419e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:40:57.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:40:57.714+0000 7f2669d68700 1 --2- 172.21.15.90:0/1284864635 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2664103620 0x7f266419e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:40:57.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:40:57.714+0000 7f2669d68700 1 -- 172.21.15.90:0/1284864635 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26641a1380 con 0x7f2664102420 2024-05-03T14:40:57.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:40:57.714+0000 7f2669d68700 1 --2- 172.21.15.90:0/1284864635 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2664102420 0x7f266406c750 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f265400c990 tx=0x7f265400cca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:40:57.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:40:57.714+0000 7f265affd700 1 -- 172.21.15.90:0/1284864635 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2654007b00 con 0x7f2664102420 2024-05-03T14:40:57.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:40:57.714+0000 7f265affd700 1 -- 172.21.15.90:0/1284864635 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f2654007ca0 con 0x7f2664102420 2024-05-03T14:40:57.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:40:57.714+0000 7f266bfcc700 1 -- 172.21.15.90:0/1284864635 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26641a1670 con 0x7f2664102420 2024-05-03T14:40:57.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:40:57.714+0000 7f265affd700 1 -- 172.21.15.90:0/1284864635 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2654018780 con 0x7f2664102420 2024-05-03T14:40:57.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:40:57.714+0000 7f266bfcc700 1 -- 172.21.15.90:0/1284864635 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26641a1c30 con 0x7f2664102420 2024-05-03T14:40:57.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:40:57.715+0000 7f266bfcc700 1 -- 172.21.15.90:0/1284864635 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f266406a190 con 0x7f2664102420 2024-05-03T14:40:57.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:40:57.716+0000 7f265affd700 1 -- 172.21.15.90:0/1284864635 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f2654018920 con 0x7f2664102420 2024-05-03T14:40:57.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:40:57.717+0000 7f265affd700 1 --2- 172.21.15.90:0/1284864635 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f26500b2880 0x7f26500b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:40:57.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:40:57.717+0000 7f265affd700 1 -- 172.21.15.90:0/1284864635 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4965+0+0 (secure 0 0 0) 0x7f26540e1490 con 0x7f2664102420 2024-05-03T14:40:57.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:40:57.717+0000 7f2669567700 1 --2- 172.21.15.90:0/1284864635 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f26500b2880 0x7f26500b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:40:57.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:40:57.718+0000 7f2669567700 1 --2- 172.21.15.90:0/1284864635 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f26500b2880 0x7f26500b4d40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f266406ccb0 tx=0x7f2660006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:40:57.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:40:57.721+0000 7f265affd700 1 -- 172.21.15.90:0/1284864635 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f2654014030 con 0x7f2664102420 2024-05-03T14:40:57.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:40:57.935+0000 7f266bfcc700 1 -- 172.21.15.90:0/1284864635 --> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f26640638e0 con 0x7f26500b2880 2024-05-03T14:40:58.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:58 smithi090 ceph-mon[33568]: Detected new or changed devices on smithi060 2024-05-03T14:40:58.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:58 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:58.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:58 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:58.876 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:58 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:40:58.876 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:58 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:40:58.876 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:58 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:40:58.876 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:58 smithi090 ceph-mon[33568]: Adjusting osd_memory_target on smithi060 to 5698M 2024-05-03T14:40:58.876 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:58 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:58.876 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:58 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:40:58.876 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:58 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:40:58.876 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:58 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:58.876 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:58 smithi090 ceph-mon[33568]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:40:58.877 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:58 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T14:40:58.877 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:58 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T14:40:58.877 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:58 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:40:58.877 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:58 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:40:58.877 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:58 smithi090 ceph-mon[33568]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-03T14:40:59.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:58 smithi060 ceph-mon[28873]: Detected new or changed devices on smithi060 2024-05-03T14:40:59.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:58 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:59.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:58 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:59.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:58 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:40:59.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:58 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:40:59.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:58 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:40:59.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:58 smithi060 ceph-mon[28873]: Adjusting osd_memory_target on smithi060 to 5698M 2024-05-03T14:40:59.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:58 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:59.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:58 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:40:59.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:58 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:40:59.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:58 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:59.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:58 smithi060 ceph-mon[28873]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:40:59.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:58 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T14:40:59.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:58 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T14:40:59.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:58 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:40:59.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:58 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:40:59.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:58 smithi060 ceph-mon[28873]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-03T14:40:59.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:58 smithi149 ceph-mon[33332]: Detected new or changed devices on smithi060 2024-05-03T14:40:59.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:58 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:59.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:58 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:59.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:58 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:40:59.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:58 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:40:59.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:58 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:40:59.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:58 smithi149 ceph-mon[33332]: Adjusting osd_memory_target on smithi060 to 5698M 2024-05-03T14:40:59.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:58 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:59.175 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:58 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:40:59.175 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:58 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:40:59.175 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:58 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:40:59.175 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:58 smithi149 ceph-mon[33332]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:40:59.175 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:58 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T14:40:59.175 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:58 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T14:40:59.175 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:58 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:40:59.175 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:58 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:40:59.175 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:58 smithi149 ceph-mon[33332]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-03T14:40:59.910 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:59 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:40:59.910 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:59 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:40:59.910 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:40:59 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:00.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:59 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:41:00.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:59 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:41:00.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:40:59 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:00.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:59 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:41:00.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:59 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:41:00.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:40:59 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:01.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:00 smithi060 ceph-mon[28873]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-03T14:41:01.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:00 smithi060 ceph-mon[28873]: from='client.? 172.21.15.90:0/971857914' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ebce6acf-8873-4b35-bd5f-80de2163a307"}]: dispatch 2024-05-03T14:41:01.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:00 smithi060 ceph-mon[28873]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ebce6acf-8873-4b35-bd5f-80de2163a307"}]: dispatch 2024-05-03T14:41:01.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:00 smithi060 ceph-mon[28873]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ebce6acf-8873-4b35-bd5f-80de2163a307"}]': finished 2024-05-03T14:41:01.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:00 smithi060 ceph-mon[28873]: osdmap e44: 8 total, 7 up, 8 in 2024-05-03T14:41:01.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:00 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T14:41:01.107 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:00 smithi090 ceph-mon[33568]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-03T14:41:01.107 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:00 smithi090 ceph-mon[33568]: from='client.? 172.21.15.90:0/971857914' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ebce6acf-8873-4b35-bd5f-80de2163a307"}]: dispatch 2024-05-03T14:41:01.107 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:00 smithi090 ceph-mon[33568]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ebce6acf-8873-4b35-bd5f-80de2163a307"}]: dispatch 2024-05-03T14:41:01.107 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:00 smithi090 ceph-mon[33568]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ebce6acf-8873-4b35-bd5f-80de2163a307"}]': finished 2024-05-03T14:41:01.107 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:00 smithi090 ceph-mon[33568]: osdmap e44: 8 total, 7 up, 8 in 2024-05-03T14:41:01.107 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:00 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T14:41:01.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:00 smithi149 ceph-mon[33332]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-03T14:41:01.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:00 smithi149 ceph-mon[33332]: from='client.? 172.21.15.90:0/971857914' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ebce6acf-8873-4b35-bd5f-80de2163a307"}]: dispatch 2024-05-03T14:41:01.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:00 smithi149 ceph-mon[33332]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ebce6acf-8873-4b35-bd5f-80de2163a307"}]: dispatch 2024-05-03T14:41:01.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:00 smithi149 ceph-mon[33332]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ebce6acf-8873-4b35-bd5f-80de2163a307"}]': finished 2024-05-03T14:41:01.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:00 smithi149 ceph-mon[33332]: osdmap e44: 8 total, 7 up, 8 in 2024-05-03T14:41:01.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:00 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T14:41:02.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:01 smithi060 ceph-mon[28873]: from='client.? 172.21.15.90:0/2094951872' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T14:41:02.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:01 smithi090 ceph-mon[33568]: from='client.? 172.21.15.90:0/2094951872' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T14:41:02.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:01 smithi149 ceph-mon[33332]: from='client.? 172.21.15.90:0/2094951872' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T14:41:03.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:02 smithi060 ceph-mon[28873]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-03T14:41:03.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:02 smithi090 ceph-mon[33568]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-03T14:41:03.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:02 smithi149 ceph-mon[33332]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-03T14:41:05.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:05 smithi060 ceph-mon[28873]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-03T14:41:05.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:05 smithi090 ceph-mon[33568]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-03T14:41:05.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:05 smithi149 ceph-mon[33332]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-03T14:41:07.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:07 smithi060 ceph-mon[28873]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-03T14:41:07.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:07 smithi090 ceph-mon[33568]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-03T14:41:07.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:07 smithi149 ceph-mon[33332]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-03T14:41:08.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:08 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-03T14:41:08.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:08 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:41:08.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:08 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-03T14:41:08.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:08 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:41:08.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:08 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-03T14:41:08.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:08 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:41:09.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:09 smithi090 ceph-mon[33568]: Deploying daemon osd.7 on smithi090 2024-05-03T14:41:09.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:09 smithi090 ceph-mon[33568]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-03T14:41:09.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:09 smithi060 ceph-mon[28873]: Deploying daemon osd.7 on smithi090 2024-05-03T14:41:09.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:09 smithi060 ceph-mon[28873]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-03T14:41:09.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:09 smithi149 ceph-mon[33332]: Deploying daemon osd.7 on smithi090 2024-05-03T14:41:09.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:09 smithi149 ceph-mon[33332]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-03T14:41:11.494 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:11 smithi090 ceph-mon[33568]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-03T14:41:11.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:11 smithi060 ceph-mon[28873]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-03T14:41:11.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:11 smithi149 ceph-mon[33332]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-03T14:41:13.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:13 smithi060 ceph-mon[28873]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-03T14:41:13.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:13 smithi090 ceph-mon[33568]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-03T14:41:13.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:13 smithi149 ceph-mon[33332]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-03T14:41:14.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:14 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:41:14.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:14 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:14.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:14 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:14.623 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:14 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:41:14.623 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:14 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:14.623 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:14 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:14.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:14 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:41:14.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:14 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:14.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:14 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:15.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:15 smithi060 ceph-mon[28873]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-03T14:41:15.624 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:15 smithi090 ceph-mon[33568]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-03T14:41:15.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:15 smithi149 ceph-mon[33332]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-03T14:41:17.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:17 smithi060 ceph-mon[28873]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-03T14:41:17.624 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:17 smithi090 ceph-mon[33568]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-03T14:41:17.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:17 smithi149 ceph-mon[33332]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-03T14:41:18.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:41:18.188+0000 7f265affd700 1 -- 172.21.15.90:0/1284864635 <== mgr.14152 v2:172.21.15.60:6800/3066142956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f26640638e0 con 0x7f26500b2880 2024-05-03T14:41:18.191 INFO:teuthology.orchestra.run.smithi090.stdout:Created osd(s) 7 on host 'smithi090' 2024-05-03T14:41:18.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:41:18.190+0000 7f266bfcc700 1 -- 172.21.15.90:0/1284864635 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f26500b2880 msgr2=0x7f26500b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:41:18.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:41:18.190+0000 7f266bfcc700 1 --2- 172.21.15.90:0/1284864635 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f26500b2880 0x7f26500b4d40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f266406ccb0 tx=0x7f2660006d20 comp rx=0 tx=0).stop 2024-05-03T14:41:18.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:41:18.190+0000 7f266bfcc700 1 -- 172.21.15.90:0/1284864635 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2664102420 msgr2=0x7f266406c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:41:18.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:41:18.190+0000 7f266bfcc700 1 --2- 172.21.15.90:0/1284864635 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2664102420 0x7f266406c750 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f265400c990 tx=0x7f265400cca0 comp rx=0 tx=0).stop 2024-05-03T14:41:18.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:41:18.190+0000 7f266bfcc700 1 -- 172.21.15.90:0/1284864635 shutdown_connections 2024-05-03T14:41:18.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:41:18.190+0000 7f266bfcc700 1 --2- 172.21.15.90:0/1284864635 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f26500b2880 0x7f26500b4d40 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:41:18.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:41:18.190+0000 7f266bfcc700 1 --2- 172.21.15.90:0/1284864635 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2664103fc0 0x7f26641a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:41:18.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:41:18.190+0000 7f266bfcc700 1 --2- 172.21.15.90:0/1284864635 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2664102420 0x7f266406c750 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:41:18.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:41:18.190+0000 7f266bfcc700 1 --2- 172.21.15.90:0/1284864635 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2664103620 0x7f266419e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:41:18.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:41:18.190+0000 7f266bfcc700 1 -- 172.21.15.90:0/1284864635 >> 172.21.15.90:0/1284864635 conn(0x7f26640fd9c0 msgr2=0x7f2664106850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:41:18.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:41:18.190+0000 7f266bfcc700 1 -- 172.21.15.90:0/1284864635 shutdown_connections 2024-05-03T14:41:18.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:41:18.190+0000 7f266bfcc700 1 -- 172.21.15.90:0/1284864635 wait complete. 2024-05-03T14:41:18.513 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:18 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:18.513 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:18 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:18.513 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:18 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:41:18.514 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:18 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:41:18.514 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:18 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:18.514 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:18 smithi090 ceph-mon[33568]: from='osd.7 [v2:172.21.15.90:6816/1686298661,v1:172.21.15.90:6817/1686298661]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-03T14:41:18.514 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:18 smithi090 ceph-mon[33568]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-03T14:41:18.514 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:18 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:18.514 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:18 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:18.514 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:18 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:41:18.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:18 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:18.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:18 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:18.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:18 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:41:18.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:18 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:41:18.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:18 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:18.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:18 smithi149 ceph-mon[33332]: from='osd.7 [v2:172.21.15.90:6816/1686298661,v1:172.21.15.90:6817/1686298661]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-03T14:41:18.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:18 smithi149 ceph-mon[33332]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-03T14:41:18.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:18 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:18.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:18 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:18.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:18 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:41:18.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:18 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:18.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:18 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:18.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:18 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:41:18.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:18 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:41:18.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:18 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:18.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:18 smithi060 ceph-mon[28873]: from='osd.7 [v2:172.21.15.90:6816/1686298661,v1:172.21.15.90:6817/1686298661]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-03T14:41:18.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:18 smithi060 ceph-mon[28873]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-03T14:41:18.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:18 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:18.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:18 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:18.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:18 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:41:18.851 DEBUG:teuthology.orchestra.run.smithi090:osd.7> sudo journalctl -f -n 0 -u ceph-285d64b2-095a-11ef-bc97-c7b262605968@osd.7.service 2024-05-03T14:41:18.853 INFO:tasks.cephadm:Deploying osd.8 on smithi149 with /dev/vg_nvme/lv_2... 2024-05-03T14:41:18.854 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-03T14:41:19.102 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T14:41:19.475 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:19 smithi149 ceph-mon[33332]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-03T14:41:19.475 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:19 smithi149 ceph-mon[33332]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-03T14:41:19.475 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:19 smithi149 ceph-mon[33332]: from='osd.7 [v2:172.21.15.90:6816/1686298661,v1:172.21.15.90:6817/1686298661]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-05-03T14:41:19.475 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:19 smithi149 ceph-mon[33332]: osdmap e45: 8 total, 7 up, 8 in 2024-05-03T14:41:19.475 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:19 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T14:41:19.475 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:19 smithi149 ceph-mon[33332]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-05-03T14:41:19.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:19 smithi090 ceph-mon[33568]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-03T14:41:19.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:19 smithi090 ceph-mon[33568]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-03T14:41:19.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:19 smithi090 ceph-mon[33568]: from='osd.7 [v2:172.21.15.90:6816/1686298661,v1:172.21.15.90:6817/1686298661]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-05-03T14:41:19.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:19 smithi090 ceph-mon[33568]: osdmap e45: 8 total, 7 up, 8 in 2024-05-03T14:41:19.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:19 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T14:41:19.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:19 smithi090 ceph-mon[33568]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-05-03T14:41:19.625 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 14:41:19 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T14:41:19.388+0000 7f487d507700 -1 osd.7 0 waiting for initial osdmap 2024-05-03T14:41:19.626 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 14:41:19 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T14:41:19.396+0000 7f48792fa700 -1 osd.7 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-03T14:41:19.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:19 smithi060 ceph-mon[28873]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-03T14:41:19.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:19 smithi060 ceph-mon[28873]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-03T14:41:19.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:19 smithi060 ceph-mon[28873]: from='osd.7 [v2:172.21.15.90:6816/1686298661,v1:172.21.15.90:6817/1686298661]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-05-03T14:41:19.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:19 smithi060 ceph-mon[28873]: osdmap e45: 8 total, 7 up, 8 in 2024-05-03T14:41:19.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:19 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T14:41:19.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:19 smithi060 ceph-mon[28873]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-05-03T14:41:20.590 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:20 smithi149 ceph-mon[33332]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-05-03T14:41:20.590 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:20 smithi149 ceph-mon[33332]: osdmap e46: 8 total, 7 up, 8 in 2024-05-03T14:41:20.591 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:20 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T14:41:20.591 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:20 smithi149 ceph-mon[33332]: osd.7 [v2:172.21.15.90:6816/1686298661,v1:172.21.15.90:6817/1686298661] boot 2024-05-03T14:41:20.591 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:20 smithi149 ceph-mon[33332]: osdmap e47: 8 total, 8 up, 8 in 2024-05-03T14:41:20.591 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:20 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T14:41:20.724 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:20 smithi090 ceph-mon[33568]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-05-03T14:41:20.724 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:20 smithi090 ceph-mon[33568]: osdmap e46: 8 total, 7 up, 8 in 2024-05-03T14:41:20.724 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:20 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T14:41:20.724 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:20 smithi090 ceph-mon[33568]: osd.7 [v2:172.21.15.90:6816/1686298661,v1:172.21.15.90:6817/1686298661] boot 2024-05-03T14:41:20.724 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:20 smithi090 ceph-mon[33568]: osdmap e47: 8 total, 8 up, 8 in 2024-05-03T14:41:20.724 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:20 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T14:41:20.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:20 smithi060 ceph-mon[28873]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-05-03T14:41:20.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:20 smithi060 ceph-mon[28873]: osdmap e46: 8 total, 7 up, 8 in 2024-05-03T14:41:20.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:20 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T14:41:20.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:20 smithi060 ceph-mon[28873]: osd.7 [v2:172.21.15.90:6816/1686298661,v1:172.21.15.90:6817/1686298661] boot 2024-05-03T14:41:20.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:20 smithi060 ceph-mon[28873]: osdmap e47: 8 total, 8 up, 8 in 2024-05-03T14:41:20.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:20 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T14:41:21.173 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-03T14:41:21.173 INFO:teuthology.orchestra.run.smithi149.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-03T14:41:21.173 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10+0 records in 2024-05-03T14:41:21.173 INFO:teuthology.orchestra.run.smithi149.stderr:10+0 records out 2024-05-03T14:41:21.173 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113217 s, 926 MB/s 2024-05-03T14:41:21.173 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping successful for: 2024-05-03T14:41:21.482 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:21 smithi149 ceph-mon[33332]: purged_snaps scrub starts 2024-05-03T14:41:21.482 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:21 smithi149 ceph-mon[33332]: purged_snaps scrub ok 2024-05-03T14:41:21.482 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:21 smithi149 ceph-mon[33332]: pgmap v194: 1 pgs: 1 unknown; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-03T14:41:21.482 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:21 smithi149 ceph-mon[33332]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-05-03T14:41:21.482 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:21 smithi149 ceph-mon[33332]: osdmap e48: 8 total, 8 up, 8 in 2024-05-03T14:41:21.760 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:21 smithi090 ceph-mon[33568]: purged_snaps scrub starts 2024-05-03T14:41:21.761 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:21 smithi090 ceph-mon[33568]: purged_snaps scrub ok 2024-05-03T14:41:21.761 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:21 smithi090 ceph-mon[33568]: pgmap v194: 1 pgs: 1 unknown; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-03T14:41:21.761 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:21 smithi090 ceph-mon[33568]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-05-03T14:41:21.761 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:21 smithi090 ceph-mon[33568]: osdmap e48: 8 total, 8 up, 8 in 2024-05-03T14:41:21.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:21 smithi060 ceph-mon[28873]: purged_snaps scrub starts 2024-05-03T14:41:21.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:21 smithi060 ceph-mon[28873]: purged_snaps scrub ok 2024-05-03T14:41:21.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:21 smithi060 ceph-mon[28873]: pgmap v194: 1 pgs: 1 unknown; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-03T14:41:21.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:21 smithi060 ceph-mon[28873]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-05-03T14:41:21.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:21 smithi060 ceph-mon[28873]: osdmap e48: 8 total, 8 up, 8 in 2024-05-03T14:41:21.922 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi149:vg_nvme/lv_2 2024-05-03T14:41:22.166 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T14:41:23.549 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:23 smithi149 ceph-mon[33332]: Detected new or changed devices on smithi090 2024-05-03T14:41:23.549 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:23 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:23.549 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:23 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:23.549 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:23 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:41:23.549 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:23 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:41:23.550 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:23 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:41:23.550 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:23 smithi149 ceph-mon[33332]: Adjusting osd_memory_target on smithi090 to 5696M 2024-05-03T14:41:23.550 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:23 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:23.550 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:23 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:41:23.550 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:23 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:41:23.550 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:23 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:23.550 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:23 smithi149 ceph-mon[33332]: pgmap v197: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-03T14:41:23.550 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:23 smithi149 ceph-mon[33332]: osdmap e49: 8 total, 8 up, 8 in 2024-05-03T14:41:23.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:23 smithi060 ceph-mon[28873]: Detected new or changed devices on smithi090 2024-05-03T14:41:23.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:23 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:23.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:23 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:23.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:23 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:41:23.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:23 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:41:23.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:23 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:41:23.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:23 smithi060 ceph-mon[28873]: Adjusting osd_memory_target on smithi090 to 5696M 2024-05-03T14:41:23.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:23 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:23.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:23 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:41:23.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:23 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:41:23.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:23 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:23.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:23 smithi060 ceph-mon[28873]: pgmap v197: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-03T14:41:23.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:23 smithi060 ceph-mon[28873]: osdmap e49: 8 total, 8 up, 8 in 2024-05-03T14:41:23.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:23 smithi090 ceph-mon[33568]: Detected new or changed devices on smithi090 2024-05-03T14:41:23.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:23 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:23.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:23 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:23.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:23 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:41:23.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:23 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:41:23.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:23 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:41:23.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:23 smithi090 ceph-mon[33568]: Adjusting osd_memory_target on smithi090 to 5696M 2024-05-03T14:41:23.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:23 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:23.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:23 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:41:23.626 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:23 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:41:23.626 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:23 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:23.626 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:23 smithi090 ceph-mon[33568]: pgmap v197: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-03T14:41:23.626 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:23 smithi090 ceph-mon[33568]: osdmap e49: 8 total, 8 up, 8 in 2024-05-03T14:41:23.791 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:41:23.789+0000 7f1138d2d700 1 -- 172.21.15.149:0/2719588898 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f11341050a0 msgr2=0x7f113410b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:41:23.792 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:41:23.789+0000 7f1138d2d700 1 --2- 172.21.15.149:0/2719588898 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f11341050a0 0x7f113410b950 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f112c00b0a0 tx=0x7f112c00b3b0 comp rx=0 tx=0).stop 2024-05-03T14:41:23.792 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:41:23.789+0000 7f1138d2d700 1 -- 172.21.15.149:0/2719588898 shutdown_connections 2024-05-03T14:41:23.792 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:41:23.789+0000 7f1138d2d700 1 --2- 172.21.15.149:0/2719588898 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f11341050a0 0x7f113410b950 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:41:23.792 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:41:23.789+0000 7f1138d2d700 1 --2- 172.21.15.149:0/2719588898 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1134103500 0x7f1134103920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:41:23.792 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:41:23.789+0000 7f1138d2d700 1 --2- 172.21.15.149:0/2719588898 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1134104700 0x7f1134104b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:41:23.792 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:41:23.789+0000 7f1138d2d700 1 -- 172.21.15.149:0/2719588898 >> 172.21.15.149:0/2719588898 conn(0x7f11340feaa0 msgr2=0x7f1134100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:41:23.792 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:41:23.789+0000 7f1138d2d700 1 -- 172.21.15.149:0/2719588898 shutdown_connections 2024-05-03T14:41:23.792 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:41:23.789+0000 7f1138d2d700 1 -- 172.21.15.149:0/2719588898 wait complete. 2024-05-03T14:41:23.792 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:41:23.790+0000 7f1138d2d700 1 Processor -- start 2024-05-03T14:41:23.792 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:41:23.790+0000 7f1138d2d700 1 -- start start 2024-05-03T14:41:23.794 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:41:23.790+0000 7f1138d2d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1134103500 0x7f1134199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:41:23.794 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:41:23.790+0000 7f1138d2d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1134104700 0x7f1134199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:41:23.794 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:41:23.790+0000 7f1138d2d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f11341050a0 0x7f11341a0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:41:23.794 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:41:23.790+0000 7f1138d2d700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f113410f2d0 con 0x7f1134104700 2024-05-03T14:41:23.794 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:41:23.790+0000 7f1138d2d700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f113410f150 con 0x7f1134103500 2024-05-03T14:41:23.794 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:41:23.790+0000 7f1138d2d700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f113410f450 con 0x7f11341050a0 2024-05-03T14:41:23.794 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:41:23.790+0000 7f1132d9d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f11341050a0 0x7f11341a0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:41:23.794 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:41:23.790+0000 7f1132d9d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f11341050a0 0x7f11341a0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:34422/0 (socket says 172.21.15.149:34422) 2024-05-03T14:41:23.794 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:41:23.790+0000 7f1132d9d700 1 -- 172.21.15.149:0/2672563575 learned_addr learned my addr 172.21.15.149:0/2672563575 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-05-03T14:41:23.794 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:41:23.790+0000 7f113259c700 1 --2- 172.21.15.149:0/2672563575 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1134103500 0x7f1134199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:41:23.795 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:41:23.790+0000 7f1132d9d700 1 -- 172.21.15.149:0/2672563575 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1134103500 msgr2=0x7f1134199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:41:23.795 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:41:23.790+0000 7f1132d9d700 1 --2- 172.21.15.149:0/2672563575 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1134103500 0x7f1134199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:41:23.795 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:41:23.790+0000 7f1132d9d700 1 -- 172.21.15.149:0/2672563575 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1134104700 msgr2=0x7f1134199d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:41:23.795 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:41:23.790+0000 7f1132d9d700 1 --2- 172.21.15.149:0/2672563575 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1134104700 0x7f1134199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:41:23.795 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:41:23.790+0000 7f1132d9d700 1 -- 172.21.15.149:0/2672563575 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11341a1480 con 0x7f11341050a0 2024-05-03T14:41:23.795 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:41:23.790+0000 7f113259c700 1 --2- 172.21.15.149:0/2672563575 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1134103500 0x7f1134199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T14:41:23.795 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:41:23.791+0000 7f1132d9d700 1 --2- 172.21.15.149:0/2672563575 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f11341050a0 0x7f11341a0de0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f112c004e40 tx=0x7f112c004f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:41:23.795 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:41:23.791+0000 7f112bfff700 1 -- 172.21.15.149:0/2672563575 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f112c021020 con 0x7f11341050a0 2024-05-03T14:41:23.795 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:41:23.791+0000 7f1138d2d700 1 -- 172.21.15.149:0/2672563575 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11341a1710 con 0x7f11341050a0 2024-05-03T14:41:23.795 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:41:23.791+0000 7f112bfff700 1 -- 172.21.15.149:0/2672563575 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f112c007de0 con 0x7f11341050a0 2024-05-03T14:41:23.795 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:41:23.791+0000 7f112bfff700 1 -- 172.21.15.149:0/2672563575 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f112c01d9e0 con 0x7f11341050a0 2024-05-03T14:41:23.795 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:41:23.791+0000 7f1138d2d700 1 -- 172.21.15.149:0/2672563575 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11341a1c40 con 0x7f11341050a0 2024-05-03T14:41:23.795 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:41:23.792+0000 7f1138d2d700 1 -- 172.21.15.149:0/2672563575 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f113406a190 con 0x7f11341050a0 2024-05-03T14:41:23.796 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:41:23.793+0000 7f112bfff700 1 -- 172.21.15.149:0/2672563575 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f112c024020 con 0x7f11341050a0 2024-05-03T14:41:23.796 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:41:23.793+0000 7f112bfff700 1 --2- 172.21.15.149:0/2672563575 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f11200b2880 0x7f11200b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:41:23.796 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:41:23.794+0000 7f112bfff700 1 -- 172.21.15.149:0/2672563575 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5397+0+0 (secure 0 0 0) 0x7f112c00f020 con 0x7f11341050a0 2024-05-03T14:41:23.796 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:41:23.794+0000 7f113259c700 1 --2- 172.21.15.149:0/2672563575 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f11200b2880 0x7f11200b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:41:23.797 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:41:23.795+0000 7f113259c700 1 --2- 172.21.15.149:0/2672563575 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f11200b2880 0x7f11200b4d40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1134104560 tx=0x7f1124008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:41:23.800 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:41:23.798+0000 7f112bfff700 1 -- 172.21.15.149:0/2672563575 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f112c0ab330 con 0x7f11341050a0 2024-05-03T14:41:24.011 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:41:24.009+0000 7f1138d2d700 1 -- 172.21.15.149:0/2672563575 --> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f11340638e0 con 0x7f11200b2880 2024-05-03T14:41:24.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:24 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T14:41:24.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:24 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T14:41:24.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:24 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:41:24.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:24 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T14:41:24.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:24 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T14:41:24.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:24 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:41:24.657 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:24 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T14:41:24.657 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:24 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T14:41:24.657 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:24 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:41:25.526 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:25 smithi090 ceph-mon[33568]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:41:25.526 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:25 smithi090 ceph-mon[33568]: pgmap v199: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-03T14:41:25.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:25 smithi060 ceph-mon[28873]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:41:25.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:25 smithi060 ceph-mon[28873]: pgmap v199: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-03T14:41:25.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:25 smithi149 ceph-mon[33332]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:41:25.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:25 smithi149 ceph-mon[33332]: pgmap v199: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-03T14:41:26.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:26 smithi090 ceph-mon[33568]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-05-03T14:41:26.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:26 smithi090 ceph-mon[33568]: Cluster is now healthy 2024-05-03T14:41:26.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:26 smithi149 ceph-mon[33332]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-05-03T14:41:26.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:26 smithi149 ceph-mon[33332]: Cluster is now healthy 2024-05-03T14:41:26.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:26 smithi060 ceph-mon[28873]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-05-03T14:41:26.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:26 smithi060 ceph-mon[28873]: Cluster is now healthy 2024-05-03T14:41:27.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:27 smithi090 ceph-mon[33568]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 192 KiB/s, 0 objects/s recovering 2024-05-03T14:41:27.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:27 smithi090 ceph-mon[33568]: from='client.? 172.21.15.149:0/314389844' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ae04b98-e9b2-4956-83d9-17e5ac3c963f"}]: dispatch 2024-05-03T14:41:27.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:27 smithi090 ceph-mon[33568]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ae04b98-e9b2-4956-83d9-17e5ac3c963f"}]: dispatch 2024-05-03T14:41:27.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:27 smithi090 ceph-mon[33568]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4ae04b98-e9b2-4956-83d9-17e5ac3c963f"}]': finished 2024-05-03T14:41:27.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:27 smithi090 ceph-mon[33568]: osdmap e50: 9 total, 8 up, 9 in 2024-05-03T14:41:27.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:27 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T14:41:27.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:27 smithi149 ceph-mon[33332]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 192 KiB/s, 0 objects/s recovering 2024-05-03T14:41:27.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:27 smithi149 ceph-mon[33332]: from='client.? 172.21.15.149:0/314389844' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ae04b98-e9b2-4956-83d9-17e5ac3c963f"}]: dispatch 2024-05-03T14:41:27.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:27 smithi149 ceph-mon[33332]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ae04b98-e9b2-4956-83d9-17e5ac3c963f"}]: dispatch 2024-05-03T14:41:27.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:27 smithi149 ceph-mon[33332]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4ae04b98-e9b2-4956-83d9-17e5ac3c963f"}]': finished 2024-05-03T14:41:27.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:27 smithi149 ceph-mon[33332]: osdmap e50: 9 total, 8 up, 9 in 2024-05-03T14:41:27.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:27 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T14:41:27.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:27 smithi060 ceph-mon[28873]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 192 KiB/s, 0 objects/s recovering 2024-05-03T14:41:27.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:27 smithi060 ceph-mon[28873]: from='client.? 172.21.15.149:0/314389844' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ae04b98-e9b2-4956-83d9-17e5ac3c963f"}]: dispatch 2024-05-03T14:41:27.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:27 smithi060 ceph-mon[28873]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ae04b98-e9b2-4956-83d9-17e5ac3c963f"}]: dispatch 2024-05-03T14:41:27.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:27 smithi060 ceph-mon[28873]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4ae04b98-e9b2-4956-83d9-17e5ac3c963f"}]': finished 2024-05-03T14:41:27.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:27 smithi060 ceph-mon[28873]: osdmap e50: 9 total, 8 up, 9 in 2024-05-03T14:41:27.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:27 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T14:41:28.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:28 smithi090 ceph-mon[33568]: from='client.? 172.21.15.149:0/3541287536' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T14:41:28.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:28 smithi149 ceph-mon[33332]: from='client.? 172.21.15.149:0/3541287536' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T14:41:28.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:28 smithi060 ceph-mon[28873]: from='client.? 172.21.15.149:0/3541287536' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T14:41:29.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:29 smithi090 ceph-mon[33568]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 159 KiB/s, 0 objects/s recovering 2024-05-03T14:41:29.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:29 smithi149 ceph-mon[33332]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 159 KiB/s, 0 objects/s recovering 2024-05-03T14:41:29.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:29 smithi060 ceph-mon[28873]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 159 KiB/s, 0 objects/s recovering 2024-05-03T14:41:31.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:31 smithi149 ceph-mon[33332]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-03T14:41:31.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:31 smithi060 ceph-mon[28873]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-03T14:41:31.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:31 smithi090 ceph-mon[33568]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-03T14:41:33.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:33 smithi149 ceph-mon[33332]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-05-03T14:41:33.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:33 smithi060 ceph-mon[28873]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-05-03T14:41:33.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:33 smithi090 ceph-mon[33568]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-05-03T14:41:35.641 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:35 smithi060 ceph-mon[28873]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-03T14:41:35.641 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:35 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-03T14:41:35.641 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:35 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:41:35.641 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:35 smithi060 ceph-mon[28873]: Deploying daemon osd.8 on smithi149 2024-05-03T14:41:35.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:35 smithi149 ceph-mon[33332]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-03T14:41:35.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:35 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-03T14:41:35.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:35 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:41:35.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:35 smithi149 ceph-mon[33332]: Deploying daemon osd.8 on smithi149 2024-05-03T14:41:35.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:35 smithi090 ceph-mon[33568]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-03T14:41:35.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:35 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-03T14:41:35.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:35 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:41:35.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:35 smithi090 ceph-mon[33568]: Deploying daemon osd.8 on smithi149 2024-05-03T14:41:37.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:37 smithi090 ceph-mon[33568]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-03T14:41:37.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:37 smithi149 ceph-mon[33332]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-03T14:41:37.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:37 smithi060 ceph-mon[28873]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-03T14:41:39.483 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:39 smithi149 ceph-mon[33332]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-03T14:41:39.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:39 smithi060 ceph-mon[28873]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-03T14:41:39.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:39 smithi090 ceph-mon[33568]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-03T14:41:41.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:41 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:41.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:41 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:41.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:41 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:41:41.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:41 smithi060 ceph-mon[28873]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-03T14:41:41.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:41 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:41.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:41 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:41.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:41 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:41:41.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:41 smithi090 ceph-mon[33568]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-03T14:41:41.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:41 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:41.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:41 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:41.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:41 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:41:41.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:41 smithi149 ceph-mon[33332]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-03T14:41:43.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:43 smithi060 ceph-mon[28873]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-03T14:41:43.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:43 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:43.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:43 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:43.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:43 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:41:43.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:43 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:41:43.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:43 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:43.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:43 smithi090 ceph-mon[33568]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-03T14:41:43.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:43 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:43.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:43 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:43.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:43 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:41:43.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:43 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:41:43.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:43 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:43.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:43 smithi149 ceph-mon[33332]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-03T14:41:43.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:43 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:43.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:43 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:43.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:43 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:41:43.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:43 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:41:43.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:43 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:44.101 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:41:44.098+0000 7f112bfff700 1 -- 172.21.15.149:0/2672563575 <== mgr.14152 v2:172.21.15.60:6800/3066142956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f11340638e0 con 0x7f11200b2880 2024-05-03T14:41:44.101 INFO:teuthology.orchestra.run.smithi149.stdout:Created osd(s) 8 on host 'smithi149' 2024-05-03T14:41:44.103 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:41:44.101+0000 7f1138d2d700 1 -- 172.21.15.149:0/2672563575 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f11200b2880 msgr2=0x7f11200b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:41:44.103 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:41:44.101+0000 7f1138d2d700 1 --2- 172.21.15.149:0/2672563575 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f11200b2880 0x7f11200b4d40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1134104560 tx=0x7f1124008040 comp rx=0 tx=0).stop 2024-05-03T14:41:44.103 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:41:44.101+0000 7f1138d2d700 1 -- 172.21.15.149:0/2672563575 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f11341050a0 msgr2=0x7f11341a0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:41:44.103 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:41:44.101+0000 7f1138d2d700 1 --2- 172.21.15.149:0/2672563575 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f11341050a0 0x7f11341a0de0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f112c004e40 tx=0x7f112c004f20 comp rx=0 tx=0).stop 2024-05-03T14:41:44.103 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:41:44.101+0000 7f1138d2d700 1 -- 172.21.15.149:0/2672563575 shutdown_connections 2024-05-03T14:41:44.103 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:41:44.101+0000 7f1138d2d700 1 --2- 172.21.15.149:0/2672563575 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f11200b2880 0x7f11200b4d40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:41:44.103 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:41:44.101+0000 7f1138d2d700 1 --2- 172.21.15.149:0/2672563575 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f11341050a0 0x7f11341a0de0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:41:44.103 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:41:44.101+0000 7f1138d2d700 1 --2- 172.21.15.149:0/2672563575 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1134103500 0x7f1134199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:41:44.104 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:41:44.101+0000 7f1138d2d700 1 --2- 172.21.15.149:0/2672563575 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1134104700 0x7f1134199d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:41:44.104 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:41:44.101+0000 7f1138d2d700 1 -- 172.21.15.149:0/2672563575 >> 172.21.15.149:0/2672563575 conn(0x7f11340feaa0 msgr2=0x7f11341079a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:41:44.104 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:41:44.101+0000 7f1138d2d700 1 -- 172.21.15.149:0/2672563575 shutdown_connections 2024-05-03T14:41:44.104 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:41:44.101+0000 7f1138d2d700 1 -- 172.21.15.149:0/2672563575 wait complete. 2024-05-03T14:41:44.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:44 smithi060 ceph-mon[28873]: from='osd.8 [v2:172.21.15.149:6816/2608348248,v1:172.21.15.149:6817/2608348248]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-03T14:41:44.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:44 smithi060 ceph-mon[28873]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-03T14:41:44.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:44 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:44.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:44 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:44.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:44 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:41:44.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:44 smithi090 ceph-mon[33568]: from='osd.8 [v2:172.21.15.149:6816/2608348248,v1:172.21.15.149:6817/2608348248]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-03T14:41:44.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:44 smithi090 ceph-mon[33568]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-03T14:41:44.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:44 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:44.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:44 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:44.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:44 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:41:44.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:44 smithi149 ceph-mon[33332]: from='osd.8 [v2:172.21.15.149:6816/2608348248,v1:172.21.15.149:6817/2608348248]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-03T14:41:44.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:44 smithi149 ceph-mon[33332]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-03T14:41:44.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:44 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:44.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:44 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:44.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:44 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:41:44.752 DEBUG:teuthology.orchestra.run.smithi149:osd.8> sudo journalctl -f -n 0 -u ceph-285d64b2-095a-11ef-bc97-c7b262605968@osd.8.service 2024-05-03T14:41:44.754 INFO:tasks.cephadm:Deploying osd.9 on smithi060 with /dev/vg_nvme/lv_1... 2024-05-03T14:41:44.755 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-03T14:41:45.218 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:41:45.600 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:45 smithi060 ceph-mon[28873]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-03T14:41:45.600 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:45 smithi060 ceph-mon[28873]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-03T14:41:45.600 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:45 smithi060 ceph-mon[28873]: from='osd.8 [v2:172.21.15.149:6816/2608348248,v1:172.21.15.149:6817/2608348248]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-03T14:41:45.600 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:45 smithi060 ceph-mon[28873]: osdmap e51: 9 total, 8 up, 9 in 2024-05-03T14:41:45.600 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:45 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T14:41:45.600 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:45 smithi060 ceph-mon[28873]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-03T14:41:45.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:45 smithi090 ceph-mon[33568]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-03T14:41:45.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:45 smithi090 ceph-mon[33568]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-03T14:41:45.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:45 smithi090 ceph-mon[33568]: from='osd.8 [v2:172.21.15.149:6816/2608348248,v1:172.21.15.149:6817/2608348248]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-03T14:41:45.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:45 smithi090 ceph-mon[33568]: osdmap e51: 9 total, 8 up, 9 in 2024-05-03T14:41:45.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:45 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T14:41:45.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:45 smithi090 ceph-mon[33568]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-03T14:41:45.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:45 smithi149 ceph-mon[33332]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-03T14:41:45.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:45 smithi149 ceph-mon[33332]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-03T14:41:45.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:45 smithi149 ceph-mon[33332]: from='osd.8 [v2:172.21.15.149:6816/2608348248,v1:172.21.15.149:6817/2608348248]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-03T14:41:45.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:45 smithi149 ceph-mon[33332]: osdmap e51: 9 total, 8 up, 9 in 2024-05-03T14:41:45.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:45 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T14:41:45.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:45 smithi149 ceph-mon[33332]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-03T14:41:46.078 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 14:41:45 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T14:41:45.739+0000 7f25560cf700 -1 osd.8 0 waiting for initial osdmap 2024-05-03T14:41:46.078 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 14:41:45 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T14:41:45.749+0000 7f254deba700 -1 osd.8 52 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-03T14:41:46.605 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:46 smithi149 ceph-mon[33332]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-05-03T14:41:46.605 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:46 smithi149 ceph-mon[33332]: osdmap e52: 9 total, 8 up, 9 in 2024-05-03T14:41:46.606 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:46 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T14:41:46.606 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:46 smithi149 ceph-mon[33332]: from='osd.8 ' entity='osd.8' 2024-05-03T14:41:46.606 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:46 smithi149 ceph-mon[33332]: osd.8 [v2:172.21.15.149:6816/2608348248,v1:172.21.15.149:6817/2608348248] boot 2024-05-03T14:41:46.606 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:46 smithi149 ceph-mon[33332]: osdmap e53: 9 total, 9 up, 9 in 2024-05-03T14:41:46.606 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:46 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T14:41:46.624 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:46 smithi090 ceph-mon[33568]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-05-03T14:41:46.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:46 smithi090 ceph-mon[33568]: osdmap e52: 9 total, 8 up, 9 in 2024-05-03T14:41:46.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:46 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T14:41:46.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:46 smithi090 ceph-mon[33568]: from='osd.8 ' entity='osd.8' 2024-05-03T14:41:46.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:46 smithi090 ceph-mon[33568]: osd.8 [v2:172.21.15.149:6816/2608348248,v1:172.21.15.149:6817/2608348248] boot 2024-05-03T14:41:46.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:46 smithi090 ceph-mon[33568]: osdmap e53: 9 total, 9 up, 9 in 2024-05-03T14:41:46.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:46 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T14:41:46.784 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:46 smithi060 ceph-mon[28873]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-05-03T14:41:46.784 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:46 smithi060 ceph-mon[28873]: osdmap e52: 9 total, 8 up, 9 in 2024-05-03T14:41:46.784 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:46 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T14:41:46.784 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:46 smithi060 ceph-mon[28873]: from='osd.8 ' entity='osd.8' 2024-05-03T14:41:46.784 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:46 smithi060 ceph-mon[28873]: osd.8 [v2:172.21.15.149:6816/2608348248,v1:172.21.15.149:6817/2608348248] boot 2024-05-03T14:41:46.784 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:46 smithi060 ceph-mon[28873]: osdmap e53: 9 total, 9 up, 9 in 2024-05-03T14:41:46.784 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:46 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T14:41:47.539 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:47 smithi060 ceph-mon[28873]: purged_snaps scrub starts 2024-05-03T14:41:47.539 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:47 smithi060 ceph-mon[28873]: purged_snaps scrub ok 2024-05-03T14:41:47.539 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:47 smithi060 ceph-mon[28873]: pgmap v214: 1 pgs: 1 peering; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-03T14:41:47.539 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:47 smithi060 ceph-mon[28873]: osdmap e54: 9 total, 9 up, 9 in 2024-05-03T14:41:47.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:47 smithi090 ceph-mon[33568]: purged_snaps scrub starts 2024-05-03T14:41:47.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:47 smithi090 ceph-mon[33568]: purged_snaps scrub ok 2024-05-03T14:41:47.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:47 smithi090 ceph-mon[33568]: pgmap v214: 1 pgs: 1 peering; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-03T14:41:47.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:47 smithi090 ceph-mon[33568]: osdmap e54: 9 total, 9 up, 9 in 2024-05-03T14:41:47.626 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:47 smithi149 ceph-mon[33332]: purged_snaps scrub starts 2024-05-03T14:41:47.626 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:47 smithi149 ceph-mon[33332]: purged_snaps scrub ok 2024-05-03T14:41:47.626 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:47 smithi149 ceph-mon[33332]: pgmap v214: 1 pgs: 1 peering; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-03T14:41:47.626 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:47 smithi149 ceph-mon[33332]: osdmap e54: 9 total, 9 up, 9 in 2024-05-03T14:41:48.494 INFO:teuthology.orchestra.run.smithi060.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-03T14:41:48.494 INFO:teuthology.orchestra.run.smithi060.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-03T14:41:48.494 INFO:teuthology.orchestra.run.smithi060.stderr: stderr: 10+0 records in 2024-05-03T14:41:48.495 INFO:teuthology.orchestra.run.smithi060.stderr:10+0 records out 2024-05-03T14:41:48.495 INFO:teuthology.orchestra.run.smithi060.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0345817 s, 303 MB/s 2024-05-03T14:41:48.495 INFO:teuthology.orchestra.run.smithi060.stderr:--> Zapping successful for: 2024-05-03T14:41:49.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:49 smithi060 ceph-mon[28873]: osdmap e55: 9 total, 9 up, 9 in 2024-05-03T14:41:49.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:49 smithi060 ceph-mon[28873]: Detected new or changed devices on smithi149 2024-05-03T14:41:49.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:49 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:49.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:49 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:49.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:49 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:41:49.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:49 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:41:49.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:49 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:41:49.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:49 smithi060 ceph-mon[28873]: Adjusting osd_memory_target on smithi149 to 1938M 2024-05-03T14:41:49.340 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:49 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:49.340 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:49 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:41:49.340 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:49 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:41:49.340 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:49 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:49.340 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:49 smithi060 ceph-mon[28873]: pgmap v217: 1 pgs: 1 peering; 577 KiB data, 262 MiB used, 804 GiB / 805 GiB avail 2024-05-03T14:41:49.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:49 smithi090 ceph-mon[33568]: osdmap e55: 9 total, 9 up, 9 in 2024-05-03T14:41:49.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:49 smithi090 ceph-mon[33568]: Detected new or changed devices on smithi149 2024-05-03T14:41:49.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:49 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:49.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:49 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:49.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:49 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:41:49.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:49 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:41:49.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:49 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:41:49.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:49 smithi090 ceph-mon[33568]: Adjusting osd_memory_target on smithi149 to 1938M 2024-05-03T14:41:49.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:49 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:49.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:49 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:41:49.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:49 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:41:49.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:49 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:49.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:49 smithi090 ceph-mon[33568]: pgmap v217: 1 pgs: 1 peering; 577 KiB data, 262 MiB used, 804 GiB / 805 GiB avail 2024-05-03T14:41:49.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:49 smithi149 ceph-mon[33332]: osdmap e55: 9 total, 9 up, 9 in 2024-05-03T14:41:49.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:49 smithi149 ceph-mon[33332]: Detected new or changed devices on smithi149 2024-05-03T14:41:49.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:49 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:49.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:49 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:49.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:49 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:41:49.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:49 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:41:49.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:49 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:41:49.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:49 smithi149 ceph-mon[33332]: Adjusting osd_memory_target on smithi149 to 1938M 2024-05-03T14:41:49.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:49 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:49.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:49 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:41:49.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:49 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:41:49.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:49 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:41:49.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:49 smithi149 ceph-mon[33332]: pgmap v217: 1 pgs: 1 peering; 577 KiB data, 262 MiB used, 804 GiB / 805 GiB avail 2024-05-03T14:41:49.465 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi060:vg_nvme/lv_1 2024-05-03T14:41:49.933 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:41:51.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:51 smithi060 ceph-mon[28873]: pgmap v218: 1 pgs: 1 peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-03T14:41:51.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:51 smithi090 ceph-mon[33568]: pgmap v218: 1 pgs: 1 peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-03T14:41:51.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:51 smithi149 ceph-mon[33332]: pgmap v218: 1 pgs: 1 peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-03T14:41:52.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:41:52.274+0000 7f893b800700 1 -- 172.21.15.60:0/3568374252 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f89340774d0 msgr2=0x7f8934075930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:41:52.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:41:52.274+0000 7f893b800700 1 --2- 172.21.15.60:0/3568374252 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f89340774d0 0x7f8934075930 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f89280098f0 tx=0x7f8928009c00 comp rx=0 tx=0).stop 2024-05-03T14:41:52.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:41:52.274+0000 7f893b800700 1 -- 172.21.15.60:0/3568374252 shutdown_connections 2024-05-03T14:41:52.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:41:52.274+0000 7f893b800700 1 --2- 172.21.15.60:0/3568374252 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f893410faf0 0x7f8934111ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:41:52.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:41:52.274+0000 7f893b800700 1 --2- 172.21.15.60:0/3568374252 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f89340774d0 0x7f8934075930 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:41:52.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:41:52.274+0000 7f893b800700 1 --2- 172.21.15.60:0/3568374252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8934075f00 0x7f8934076380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:41:52.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:41:52.274+0000 7f893b800700 1 -- 172.21.15.60:0/3568374252 >> 172.21.15.60:0/3568374252 conn(0x7f89340fea70 msgr2=0x7f8934100ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:41:52.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:41:52.275+0000 7f893b800700 1 -- 172.21.15.60:0/3568374252 shutdown_connections 2024-05-03T14:41:52.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:41:52.275+0000 7f893b800700 1 -- 172.21.15.60:0/3568374252 wait complete. 2024-05-03T14:41:52.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:41:52.276+0000 7f893b800700 1 Processor -- start 2024-05-03T14:41:52.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:41:52.276+0000 7f893b800700 1 -- start start 2024-05-03T14:41:52.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:41:52.276+0000 7f893b800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8934075f00 0x7f893419fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:41:52.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:41:52.276+0000 7f893b800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f89340774d0 0x7f89341a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:41:52.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:41:52.276+0000 7f893b800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f893410faf0 0x7f89341a7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:41:52.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:41:52.276+0000 7f893b800700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f8934115880 con 0x7f8934075f00 2024-05-03T14:41:52.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:41:52.276+0000 7f893b800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8934115700 con 0x7f89340774d0 2024-05-03T14:41:52.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:41:52.276+0000 7f893b800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8934115a00 con 0x7f893410faf0 2024-05-03T14:41:52.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:41:52.276+0000 7f8938e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8934075f00 0x7f893419fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:41:52.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:41:52.277+0000 7f8938e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8934075f00 0x7f893419fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34066/0 (socket says 172.21.15.60:34066) 2024-05-03T14:41:52.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:41:52.277+0000 7f8938e00700 1 -- 172.21.15.60:0/2851121329 learned_addr learned my addr 172.21.15.60:0/2851121329 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:41:52.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:41:52.277+0000 7f8939800700 1 --2- 172.21.15.60:0/2851121329 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f893410faf0 0x7f89341a7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:41:52.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:41:52.277+0000 7f8933e00700 1 --2- 172.21.15.60:0/2851121329 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f89340774d0 0x7f89341a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:41:52.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:41:52.277+0000 7f8939800700 1 -- 172.21.15.60:0/2851121329 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f89340774d0 msgr2=0x7f89341a0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:41:52.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:41:52.277+0000 7f8939800700 1 --2- 172.21.15.60:0/2851121329 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f89340774d0 0x7f89341a0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:41:52.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:41:52.277+0000 7f8939800700 1 -- 172.21.15.60:0/2851121329 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8934075f00 msgr2=0x7f893419fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:41:52.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:41:52.277+0000 7f8939800700 1 --2- 172.21.15.60:0/2851121329 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8934075f00 0x7f893419fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:41:52.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:41:52.277+0000 7f8939800700 1 -- 172.21.15.60:0/2851121329 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89341a7b20 con 0x7f893410faf0 2024-05-03T14:41:52.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:41:52.277+0000 7f8938e00700 1 --2- 172.21.15.60:0/2851121329 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8934075f00 0x7f893419fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:41:52.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:41:52.277+0000 7f8939800700 1 --2- 172.21.15.60:0/2851121329 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f893410faf0 0x7f89341a7480 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f892c009f60 tx=0x7f892c009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:41:52.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:41:52.278+0000 7f8931600700 1 -- 172.21.15.60:0/2851121329 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892c0092e0 con 0x7f893410faf0 2024-05-03T14:41:52.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:41:52.278+0000 7f893b800700 1 -- 172.21.15.60:0/2851121329 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89341a7db0 con 0x7f893410faf0 2024-05-03T14:41:52.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:41:52.278+0000 7f8931600700 1 -- 172.21.15.60:0/2851121329 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f892c004bb0 con 0x7f893410faf0 2024-05-03T14:41:52.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:41:52.278+0000 7f8931600700 1 -- 172.21.15.60:0/2851121329 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892c004080 con 0x7f893410faf0 2024-05-03T14:41:52.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:41:52.278+0000 7f893b800700 1 -- 172.21.15.60:0/2851121329 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89341a8210 con 0x7f893410faf0 2024-05-03T14:41:52.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:41:52.279+0000 7f893b800700 1 -- 172.21.15.60:0/2851121329 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f893402e510 con 0x7f893410faf0 2024-05-03T14:41:52.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:41:52.281+0000 7f8931600700 1 -- 172.21.15.60:0/2851121329 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f892c01b030 con 0x7f893410faf0 2024-05-03T14:41:52.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:41:52.283+0000 7f8931600700 1 --2- 172.21.15.60:0/2851121329 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f891c0b2880 0x7f891c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:41:52.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:41:52.283+0000 7f8931600700 1 -- 172.21.15.60:0/2851121329 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5829+0+0 (secure 0 0 0) 0x7f892c0e1310 con 0x7f893410faf0 2024-05-03T14:41:52.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:41:52.283+0000 7f8938e00700 1 --2- 172.21.15.60:0/2851121329 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f891c0b2880 0x7f891c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:41:52.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:41:52.284+0000 7f8938e00700 1 --2- 172.21.15.60:0/2851121329 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f891c0b2880 0x7f891c0b4d40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f892800fda0 tx=0x7f8928019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:41:52.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:41:52.287+0000 7f8931600700 1 -- 172.21.15.60:0/2851121329 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f892c022020 con 0x7f893410faf0 2024-05-03T14:41:52.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:41:52.538+0000 7f893b800700 1 -- 172.21.15.60:0/2851121329 --> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f89340638e0 con 0x7f891c0b2880 2024-05-03T14:41:53.579 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:53 smithi060 ceph-mon[28873]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-05-03T14:41:53.580 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:53 smithi060 ceph-mon[28873]: from='client.24331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:41:53.580 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:53 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T14:41:53.580 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:53 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T14:41:53.580 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:53 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:41:53.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:53 smithi090 ceph-mon[33568]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-05-03T14:41:53.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:53 smithi090 ceph-mon[33568]: from='client.24331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:41:53.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:53 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T14:41:53.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:53 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T14:41:53.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:53 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:41:53.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:53 smithi149 ceph-mon[33332]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-05-03T14:41:53.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:53 smithi149 ceph-mon[33332]: from='client.24331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:41:53.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:53 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T14:41:53.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:53 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T14:41:53.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:53 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:41:55.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:55 smithi060 ceph-mon[28873]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-03T14:41:55.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:55 smithi090 ceph-mon[33568]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-03T14:41:55.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:55 smithi149 ceph-mon[33332]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-03T14:41:57.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:57 smithi060 ceph-mon[28873]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-05-03T14:41:57.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:57 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/3602239305' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d0c0a70d-aa76-47e4-91e8-c343c8c41197"}]: dispatch 2024-05-03T14:41:57.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:57 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/3602239305' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d0c0a70d-aa76-47e4-91e8-c343c8c41197"}]': finished 2024-05-03T14:41:57.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:57 smithi060 ceph-mon[28873]: osdmap e56: 10 total, 9 up, 10 in 2024-05-03T14:41:57.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:57 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T14:41:57.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:57 smithi090 ceph-mon[33568]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-05-03T14:41:57.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:57 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/3602239305' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d0c0a70d-aa76-47e4-91e8-c343c8c41197"}]: dispatch 2024-05-03T14:41:57.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:57 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/3602239305' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d0c0a70d-aa76-47e4-91e8-c343c8c41197"}]': finished 2024-05-03T14:41:57.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:57 smithi090 ceph-mon[33568]: osdmap e56: 10 total, 9 up, 10 in 2024-05-03T14:41:57.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:57 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T14:41:57.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:57 smithi149 ceph-mon[33332]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-05-03T14:41:57.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:57 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/3602239305' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d0c0a70d-aa76-47e4-91e8-c343c8c41197"}]: dispatch 2024-05-03T14:41:57.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:57 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/3602239305' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d0c0a70d-aa76-47e4-91e8-c343c8c41197"}]': finished 2024-05-03T14:41:57.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:57 smithi149 ceph-mon[33332]: osdmap e56: 10 total, 9 up, 10 in 2024-05-03T14:41:57.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:57 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T14:41:58.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:58 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/2983129446' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T14:41:58.624 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:58 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/2983129446' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T14:41:58.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:58 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/2983129446' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T14:41:59.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:41:59 smithi090 ceph-mon[33568]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T14:41:59.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:41:59 smithi149 ceph-mon[33332]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T14:41:59.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:41:59 smithi060 ceph-mon[28873]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T14:42:01.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:01 smithi060 ceph-mon[28873]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T14:42:01.624 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:01 smithi090 ceph-mon[33568]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T14:42:01.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:01 smithi149 ceph-mon[33332]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T14:42:03.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:03 smithi090 ceph-mon[33568]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-03T14:42:03.641 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:03 smithi060 ceph-mon[28873]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-03T14:42:03.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:03 smithi149 ceph-mon[33332]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-03T14:42:05.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:05 smithi090 ceph-mon[33568]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-03T14:42:05.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:05 smithi149 ceph-mon[33332]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-03T14:42:05.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:05 smithi060 ceph-mon[28873]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-03T14:42:06.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:06 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-03T14:42:06.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:06 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:42:06.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:06 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-03T14:42:06.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:06 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:42:06.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:06 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-03T14:42:06.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:06 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:42:07.430 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:07 smithi060 ceph-mon[28873]: Deploying daemon osd.9 on smithi060 2024-05-03T14:42:07.430 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:07 smithi060 ceph-mon[28873]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-03T14:42:07.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:07 smithi090 ceph-mon[33568]: Deploying daemon osd.9 on smithi060 2024-05-03T14:42:07.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:07 smithi090 ceph-mon[33568]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-03T14:42:07.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:07 smithi149 ceph-mon[33332]: Deploying daemon osd.9 on smithi060 2024-05-03T14:42:07.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:07 smithi149 ceph-mon[33332]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-03T14:42:09.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:09 smithi060 ceph-mon[28873]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-03T14:42:09.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:09 smithi090 ceph-mon[33568]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-03T14:42:09.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:09 smithi149 ceph-mon[33332]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-03T14:42:11.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:11 smithi060 ceph-mon[28873]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-03T14:42:11.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:11 smithi090 ceph-mon[33568]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-03T14:42:11.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:11 smithi149 ceph-mon[33332]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-03T14:42:13.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:13 smithi149 ceph-mon[33332]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-03T14:42:13.767 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:13 smithi060 ceph-mon[28873]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-03T14:42:13.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:13 smithi090 ceph-mon[33568]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-03T14:42:15.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:15 smithi149 ceph-mon[33332]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-03T14:42:15.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:15 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:15.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:15 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:15.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:15 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:42:15.735 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:15 smithi060 ceph-mon[28873]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-03T14:42:15.736 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:15 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:15.736 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:15 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:15.736 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:15 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:42:15.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:15 smithi090 ceph-mon[33568]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-03T14:42:15.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:15 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:15.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:15 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:15.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:15 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:42:17.661 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:17 smithi060 ceph-mon[28873]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-03T14:42:17.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:17 smithi149 ceph-mon[33332]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-03T14:42:17.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:17 smithi090 ceph-mon[33568]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-03T14:42:19.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:19 smithi149 ceph-mon[33332]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-03T14:42:19.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:19 smithi149 ceph-mon[33332]: from='osd.9 [v2:172.21.15.60:6826/2156130999,v1:172.21.15.60:6827/2156130999]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-03T14:42:19.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:19 smithi149 ceph-mon[33332]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-03T14:42:19.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:19 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:19.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:19 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:19.674 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:19 smithi060 ceph-mon[28873]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-03T14:42:19.674 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:19 smithi060 ceph-mon[28873]: from='osd.9 [v2:172.21.15.60:6826/2156130999,v1:172.21.15.60:6827/2156130999]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-03T14:42:19.674 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:19 smithi060 ceph-mon[28873]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-03T14:42:19.675 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:19 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:19.675 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:19 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:19.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:19 smithi090 ceph-mon[33568]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-03T14:42:19.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:19 smithi090 ceph-mon[33568]: from='osd.9 [v2:172.21.15.60:6826/2156130999,v1:172.21.15.60:6827/2156130999]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-03T14:42:19.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:19 smithi090 ceph-mon[33568]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-03T14:42:19.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:19 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:19.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:19 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:20.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:42:20.295+0000 7f8931600700 1 -- 172.21.15.60:0/2851121329 <== mgr.14152 v2:172.21.15.60:6800/3066142956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f89340638e0 con 0x7f891c0b2880 2024-05-03T14:42:20.298 INFO:teuthology.orchestra.run.smithi060.stdout:Created osd(s) 9 on host 'smithi060' 2024-05-03T14:42:20.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:42:20.300+0000 7f893b800700 1 -- 172.21.15.60:0/2851121329 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f891c0b2880 msgr2=0x7f891c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:42:20.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:42:20.300+0000 7f893b800700 1 --2- 172.21.15.60:0/2851121329 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f891c0b2880 0x7f891c0b4d40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f892800fda0 tx=0x7f8928019040 comp rx=0 tx=0).stop 2024-05-03T14:42:20.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:42:20.300+0000 7f893b800700 1 -- 172.21.15.60:0/2851121329 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f893410faf0 msgr2=0x7f89341a7480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:42:20.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:42:20.300+0000 7f893b800700 1 --2- 172.21.15.60:0/2851121329 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f893410faf0 0x7f89341a7480 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f892c009f60 tx=0x7f892c009f90 comp rx=0 tx=0).stop 2024-05-03T14:42:20.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:42:20.300+0000 7f893b800700 1 -- 172.21.15.60:0/2851121329 shutdown_connections 2024-05-03T14:42:20.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:42:20.300+0000 7f893b800700 1 --2- 172.21.15.60:0/2851121329 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f891c0b2880 0x7f891c0b4d40 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:42:20.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:42:20.300+0000 7f893b800700 1 --2- 172.21.15.60:0/2851121329 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f893410faf0 0x7f89341a7480 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:42:20.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:42:20.300+0000 7f893b800700 1 --2- 172.21.15.60:0/2851121329 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8934075f00 0x7f893419fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:42:20.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:42:20.300+0000 7f893b800700 1 --2- 172.21.15.60:0/2851121329 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f89340774d0 0x7f89341a0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:42:20.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:42:20.300+0000 7f893b800700 1 -- 172.21.15.60:0/2851121329 >> 172.21.15.60:0/2851121329 conn(0x7f89340fea70 msgr2=0x7f8934102e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:42:20.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:42:20.300+0000 7f893b800700 1 -- 172.21.15.60:0/2851121329 shutdown_connections 2024-05-03T14:42:20.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:42:20.300+0000 7f893b800700 1 -- 172.21.15.60:0/2851121329 wait complete. 2024-05-03T14:42:20.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:20 smithi060 ceph-mon[28873]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-03T14:42:20.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:20 smithi060 ceph-mon[28873]: from='osd.9 [v2:172.21.15.60:6826/2156130999,v1:172.21.15.60:6827/2156130999]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-05-03T14:42:20.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:20 smithi060 ceph-mon[28873]: osdmap e57: 10 total, 9 up, 10 in 2024-05-03T14:42:20.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:20 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T14:42:20.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:20 smithi060 ceph-mon[28873]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-05-03T14:42:20.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:20 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:20.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:20 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:20.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:20 smithi060 ceph-mon[28873]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-05-03T14:42:20.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:20 smithi060 ceph-mon[28873]: osdmap e58: 10 total, 9 up, 10 in 2024-05-03T14:42:20.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:20 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T14:42:20.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:20 smithi149 ceph-mon[33332]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-03T14:42:20.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:20 smithi149 ceph-mon[33332]: from='osd.9 [v2:172.21.15.60:6826/2156130999,v1:172.21.15.60:6827/2156130999]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-05-03T14:42:20.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:20 smithi149 ceph-mon[33332]: osdmap e57: 10 total, 9 up, 10 in 2024-05-03T14:42:20.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:20 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T14:42:20.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:20 smithi149 ceph-mon[33332]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-05-03T14:42:20.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:20 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:20.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:20 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:20.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:20 smithi149 ceph-mon[33332]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-05-03T14:42:20.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:20 smithi149 ceph-mon[33332]: osdmap e58: 10 total, 9 up, 10 in 2024-05-03T14:42:20.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:20 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T14:42:20.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:20 smithi090 ceph-mon[33568]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-03T14:42:20.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:20 smithi090 ceph-mon[33568]: from='osd.9 [v2:172.21.15.60:6826/2156130999,v1:172.21.15.60:6827/2156130999]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-05-03T14:42:20.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:20 smithi090 ceph-mon[33568]: osdmap e57: 10 total, 9 up, 10 in 2024-05-03T14:42:20.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:20 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T14:42:20.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:20 smithi090 ceph-mon[33568]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-05-03T14:42:20.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:20 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:20.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:20 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:20.876 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:20 smithi090 ceph-mon[33568]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-05-03T14:42:20.876 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:20 smithi090 ceph-mon[33568]: osdmap e58: 10 total, 9 up, 10 in 2024-05-03T14:42:20.876 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:20 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T14:42:21.117 DEBUG:teuthology.orchestra.run.smithi060:osd.9> sudo journalctl -f -n 0 -u ceph-285d64b2-095a-11ef-bc97-c7b262605968@osd.9.service 2024-05-03T14:42:21.122 INFO:tasks.cephadm:Deploying osd.10 on smithi090 with /dev/vg_nvme/lv_1... 2024-05-03T14:42:21.122 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-03T14:42:21.367 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.b/config 2024-05-03T14:42:21.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:21 smithi149 ceph-mon[33332]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-03T14:42:21.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:21 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T14:42:21.690 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:21 smithi060 ceph-mon[28873]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-03T14:42:21.690 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:21 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T14:42:21.777 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:21 smithi090 ceph-mon[33568]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-03T14:42:21.777 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:21 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T14:42:22.585 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:22 smithi060 ceph-mon[28873]: purged_snaps scrub starts 2024-05-03T14:42:22.585 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:22 smithi060 ceph-mon[28873]: purged_snaps scrub ok 2024-05-03T14:42:22.585 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:22 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T14:42:22.585 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:22 smithi060 ceph-mon[28873]: osd.9 [v2:172.21.15.60:6826/2156130999,v1:172.21.15.60:6827/2156130999] boot 2024-05-03T14:42:22.585 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:22 smithi060 ceph-mon[28873]: osdmap e59: 10 total, 10 up, 10 in 2024-05-03T14:42:22.585 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:22 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T14:42:22.801 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:22 smithi090 ceph-mon[33568]: purged_snaps scrub starts 2024-05-03T14:42:22.802 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:22 smithi090 ceph-mon[33568]: purged_snaps scrub ok 2024-05-03T14:42:22.802 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:22 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T14:42:22.802 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:22 smithi090 ceph-mon[33568]: osd.9 [v2:172.21.15.60:6826/2156130999,v1:172.21.15.60:6827/2156130999] boot 2024-05-03T14:42:22.802 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:22 smithi090 ceph-mon[33568]: osdmap e59: 10 total, 10 up, 10 in 2024-05-03T14:42:22.802 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:22 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T14:42:22.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:22 smithi149 ceph-mon[33332]: purged_snaps scrub starts 2024-05-03T14:42:22.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:22 smithi149 ceph-mon[33332]: purged_snaps scrub ok 2024-05-03T14:42:22.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:22 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T14:42:22.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:22 smithi149 ceph-mon[33332]: osd.9 [v2:172.21.15.60:6826/2156130999,v1:172.21.15.60:6827/2156130999] boot 2024-05-03T14:42:22.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:22 smithi149 ceph-mon[33332]: osdmap e59: 10 total, 10 up, 10 in 2024-05-03T14:42:22.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:22 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T14:42:23.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:23 smithi060 ceph-mon[28873]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-03T14:42:23.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:23 smithi060 ceph-mon[28873]: osdmap e60: 10 total, 10 up, 10 in 2024-05-03T14:42:23.595 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-03T14:42:23.595 INFO:teuthology.orchestra.run.smithi090.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-03T14:42:23.596 INFO:teuthology.orchestra.run.smithi090.stderr: stderr: 10+0 records in 2024-05-03T14:42:23.596 INFO:teuthology.orchestra.run.smithi090.stderr:10+0 records out 2024-05-03T14:42:23.596 INFO:teuthology.orchestra.run.smithi090.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.011542 s, 908 MB/s 2024-05-03T14:42:23.596 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping successful for: 2024-05-03T14:42:23.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:23 smithi090 ceph-mon[33568]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-03T14:42:23.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:23 smithi090 ceph-mon[33568]: osdmap e60: 10 total, 10 up, 10 in 2024-05-03T14:42:23.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:23 smithi149 ceph-mon[33332]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-03T14:42:23.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:23 smithi149 ceph-mon[33332]: osdmap e60: 10 total, 10 up, 10 in 2024-05-03T14:42:24.281 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi090:vg_nvme/lv_1 2024-05-03T14:42:24.517 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.b/config 2024-05-03T14:42:25.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:25 smithi090 ceph-mon[33568]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-03T14:42:25.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:25 smithi060 ceph-mon[28873]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-03T14:42:25.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:25 smithi149 ceph-mon[33332]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-03T14:42:26.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:42:26.176+0000 7fbca5176700 1 -- 172.21.15.90:0/4042908594 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbca0102400 msgr2=0x7fbca0102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:42:26.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:42:26.176+0000 7fbca5176700 1 --2- 172.21.15.90:0/4042908594 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbca0102400 0x7fbca0102820 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbc900099c0 tx=0x7fbc90009cd0 comp rx=0 tx=0).stop 2024-05-03T14:42:26.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:42:26.176+0000 7fbca5176700 1 -- 172.21.15.90:0/4042908594 shutdown_connections 2024-05-03T14:42:26.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:42:26.176+0000 7fbca5176700 1 --2- 172.21.15.90:0/4042908594 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbca0103fa0 0x7fbca010a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:42:26.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:42:26.176+0000 7fbca5176700 1 --2- 172.21.15.90:0/4042908594 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbca0102400 0x7fbca0102820 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:42:26.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:42:26.176+0000 7fbca5176700 1 --2- 172.21.15.90:0/4042908594 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbca0103600 0x7fbca0103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:42:26.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:42:26.176+0000 7fbca5176700 1 -- 172.21.15.90:0/4042908594 >> 172.21.15.90:0/4042908594 conn(0x7fbca00fd9a0 msgr2=0x7fbca00ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:42:26.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:42:26.176+0000 7fbca5176700 1 -- 172.21.15.90:0/4042908594 shutdown_connections 2024-05-03T14:42:26.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:42:26.176+0000 7fbca5176700 1 -- 172.21.15.90:0/4042908594 wait complete. 2024-05-03T14:42:26.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:42:26.177+0000 7fbca5176700 1 Processor -- start 2024-05-03T14:42:26.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:42:26.177+0000 7fbca5176700 1 -- start start 2024-05-03T14:42:26.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:42:26.177+0000 7fbca5176700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbca0102400 0x7fbca006c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:42:26.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:42:26.177+0000 7fbca5176700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbca0103600 0x7fbca019e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:42:26.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:42:26.177+0000 7fbca5176700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbca0103fa0 0x7fbca01a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:42:26.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:42:26.177+0000 7fbca5176700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fbca010e250 con 0x7fbca0103600 2024-05-03T14:42:26.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:42:26.177+0000 7fbca5176700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fbca010e0d0 con 0x7fbca0103fa0 2024-05-03T14:42:26.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:42:26.177+0000 7fbca5176700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbca010e3d0 con 0x7fbca0102400 2024-05-03T14:42:26.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:42:26.177+0000 7fbc9f59e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbca0103fa0 0x7fbca01a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:42:26.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:42:26.177+0000 7fbc9f59e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbca0103fa0 0x7fbca01a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36414/0 (socket says 172.21.15.90:36414) 2024-05-03T14:42:26.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:42:26.177+0000 7fbc9f59e700 1 -- 172.21.15.90:0/3093147288 learned_addr learned my addr 172.21.15.90:0/3093147288 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-03T14:42:26.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:42:26.177+0000 7fbc9ed9d700 1 --2- 172.21.15.90:0/3093147288 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbca0102400 0x7fbca006c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:42:26.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:42:26.178+0000 7fbc9f59e700 1 -- 172.21.15.90:0/3093147288 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbca0102400 msgr2=0x7fbca006c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:42:26.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:42:26.178+0000 7fbc9f59e700 1 --2- 172.21.15.90:0/3093147288 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbca0102400 0x7fbca006c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:42:26.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:42:26.178+0000 7fbc9f59e700 1 -- 172.21.15.90:0/3093147288 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbca0103600 msgr2=0x7fbca019e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:42:26.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:42:26.178+0000 7fbc9f59e700 1 --2- 172.21.15.90:0/3093147288 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbca0103600 0x7fbca019e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:42:26.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:42:26.178+0000 7fbc9f59e700 1 -- 172.21.15.90:0/3093147288 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbca01a1600 con 0x7fbca0103fa0 2024-05-03T14:42:26.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:42:26.178+0000 7fbc9f59e700 1 --2- 172.21.15.90:0/3093147288 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbca0103fa0 0x7fbca01a0d30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbc9800b970 tx=0x7fbc9800bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:42:26.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:42:26.178+0000 7fbc87fff700 1 -- 172.21.15.90:0/3093147288 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc98019070 con 0x7fbca0103fa0 2024-05-03T14:42:26.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:42:26.178+0000 7fbc87fff700 1 -- 172.21.15.90:0/3093147288 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7fbc9800ce30 con 0x7fbca0103fa0 2024-05-03T14:42:26.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:42:26.178+0000 7fbc87fff700 1 -- 172.21.15.90:0/3093147288 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc980075d0 con 0x7fbca0103fa0 2024-05-03T14:42:26.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:42:26.178+0000 7fbca5176700 1 -- 172.21.15.90:0/3093147288 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbca01a18f0 con 0x7fbca0103fa0 2024-05-03T14:42:26.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:42:26.178+0000 7fbca5176700 1 -- 172.21.15.90:0/3093147288 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbca01a1e60 con 0x7fbca0103fa0 2024-05-03T14:42:26.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:42:26.179+0000 7fbca5176700 1 -- 172.21.15.90:0/3093147288 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbca006a190 con 0x7fbca0103fa0 2024-05-03T14:42:26.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:42:26.180+0000 7fbc87fff700 1 -- 172.21.15.90:0/3093147288 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fbc9801c070 con 0x7fbca0103fa0 2024-05-03T14:42:26.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:42:26.181+0000 7fbc87fff700 1 --2- 172.21.15.90:0/3093147288 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fbc880b2880 0x7fbc880b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:42:26.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:42:26.181+0000 7fbc87fff700 1 -- 172.21.15.90:0/3093147288 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6261+0+0 (secure 0 0 0) 0x7fbc980e10b0 con 0x7fbca0103fa0 2024-05-03T14:42:26.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:42:26.181+0000 7fbc9ed9d700 1 --2- 172.21.15.90:0/3093147288 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fbc880b2880 0x7fbc880b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:42:26.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:42:26.182+0000 7fbc9ed9d700 1 --2- 172.21.15.90:0/3093147288 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fbc880b2880 0x7fbc880b4d40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbc9000b580 tx=0x7fbc90005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:42:26.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:42:26.185+0000 7fbc87fff700 1 -- 172.21.15.90:0/3093147288 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fbc98051d70 con 0x7fbca0103fa0 2024-05-03T14:42:26.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:42:26.411+0000 7fbca5176700 1 -- 172.21.15.90:0/3093147288 --> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fbca00638e0 con 0x7fbc880b2880 2024-05-03T14:42:27.287 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:27 smithi090 ceph-mon[33568]: Detected new or changed devices on smithi060 2024-05-03T14:42:27.287 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:27 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:27.287 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:27 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:27.287 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:27 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:42:27.287 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:27 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:42:27.288 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:27 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:42:27.288 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:27 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:42:27.288 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:27 smithi090 ceph-mon[33568]: Adjusting osd_memory_target on smithi060 to 4273M 2024-05-03T14:42:27.288 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:27 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:27.288 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:27 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:42:27.288 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:27 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:42:27.288 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:27 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:27.288 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:27 smithi090 ceph-mon[33568]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-03T14:42:27.288 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:27 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:42:27.288 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:27 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:42:27.288 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:27 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:42:27.288 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:27 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:27.288 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:27 smithi090 ceph-mon[33568]: from='client.24362 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:42:27.288 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:27 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T14:42:27.289 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:27 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T14:42:27.289 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:27 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:42:27.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:27 smithi060 ceph-mon[28873]: Detected new or changed devices on smithi060 2024-05-03T14:42:27.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:27 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:27.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:27 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:27.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:27 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:42:27.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:27 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:42:27.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:27 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:42:27.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:27 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:42:27.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:27 smithi060 ceph-mon[28873]: Adjusting osd_memory_target on smithi060 to 4273M 2024-05-03T14:42:27.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:27 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:27.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:27 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:42:27.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:27 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:42:27.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:27 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:27.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:27 smithi060 ceph-mon[28873]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-03T14:42:27.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:27 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:42:27.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:27 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:42:27.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:27 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:42:27.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:27 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:27.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:27 smithi060 ceph-mon[28873]: from='client.24362 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:42:27.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:27 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T14:42:27.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:27 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T14:42:27.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:27 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:42:27.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:27 smithi149 ceph-mon[33332]: Detected new or changed devices on smithi060 2024-05-03T14:42:27.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:27 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:27.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:27 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:27.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:27 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:42:27.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:27 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:42:27.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:27 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:42:27.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:27 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:42:27.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:27 smithi149 ceph-mon[33332]: Adjusting osd_memory_target on smithi060 to 4273M 2024-05-03T14:42:27.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:27 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:27.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:27 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:42:27.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:27 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:42:27.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:27 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:27.425 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:27 smithi149 ceph-mon[33332]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-03T14:42:27.425 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:27 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:42:27.425 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:27 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:42:27.425 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:27 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:42:27.425 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:27 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:27.425 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:27 smithi149 ceph-mon[33332]: from='client.24362 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:42:27.425 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:27 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T14:42:27.425 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:27 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T14:42:27.425 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:27 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:42:29.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:29 smithi060 ceph-mon[28873]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-03T14:42:29.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:29 smithi090 ceph-mon[33568]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-03T14:42:29.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:29 smithi149 ceph-mon[33332]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-03T14:42:30.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:30 smithi060 ceph-mon[28873]: from='client.? 172.21.15.90:0/4242446625' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cdc3d7b8-33c7-4cc4-be4e-6887d15a3d8c"}]: dispatch 2024-05-03T14:42:30.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:30 smithi060 ceph-mon[28873]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cdc3d7b8-33c7-4cc4-be4e-6887d15a3d8c"}]: dispatch 2024-05-03T14:42:30.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:30 smithi060 ceph-mon[28873]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cdc3d7b8-33c7-4cc4-be4e-6887d15a3d8c"}]': finished 2024-05-03T14:42:30.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:30 smithi060 ceph-mon[28873]: osdmap e61: 11 total, 10 up, 11 in 2024-05-03T14:42:30.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:30 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T14:42:30.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:30 smithi060 ceph-mon[28873]: from='client.? 172.21.15.90:0/23695289' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T14:42:30.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:30 smithi090 ceph-mon[33568]: from='client.? 172.21.15.90:0/4242446625' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cdc3d7b8-33c7-4cc4-be4e-6887d15a3d8c"}]: dispatch 2024-05-03T14:42:30.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:30 smithi090 ceph-mon[33568]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cdc3d7b8-33c7-4cc4-be4e-6887d15a3d8c"}]: dispatch 2024-05-03T14:42:30.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:30 smithi090 ceph-mon[33568]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cdc3d7b8-33c7-4cc4-be4e-6887d15a3d8c"}]': finished 2024-05-03T14:42:30.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:30 smithi090 ceph-mon[33568]: osdmap e61: 11 total, 10 up, 11 in 2024-05-03T14:42:30.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:30 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T14:42:30.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:30 smithi090 ceph-mon[33568]: from='client.? 172.21.15.90:0/23695289' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T14:42:30.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:30 smithi149 ceph-mon[33332]: from='client.? 172.21.15.90:0/4242446625' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cdc3d7b8-33c7-4cc4-be4e-6887d15a3d8c"}]: dispatch 2024-05-03T14:42:30.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:30 smithi149 ceph-mon[33332]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cdc3d7b8-33c7-4cc4-be4e-6887d15a3d8c"}]: dispatch 2024-05-03T14:42:30.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:30 smithi149 ceph-mon[33332]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cdc3d7b8-33c7-4cc4-be4e-6887d15a3d8c"}]': finished 2024-05-03T14:42:30.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:30 smithi149 ceph-mon[33332]: osdmap e61: 11 total, 10 up, 11 in 2024-05-03T14:42:30.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:30 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T14:42:30.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:30 smithi149 ceph-mon[33332]: from='client.? 172.21.15.90:0/23695289' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T14:42:31.624 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:31 smithi090 ceph-mon[33568]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-03T14:42:31.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:31 smithi149 ceph-mon[33332]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-03T14:42:31.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:31 smithi060 ceph-mon[28873]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-03T14:42:33.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:33 smithi090 ceph-mon[33568]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-03T14:42:33.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:33 smithi149 ceph-mon[33332]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-03T14:42:33.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:33 smithi060 ceph-mon[28873]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-03T14:42:35.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:35 smithi090 ceph-mon[33568]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-03T14:42:35.639 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:35 smithi060 ceph-mon[28873]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-03T14:42:35.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:35 smithi149 ceph-mon[33332]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-03T14:42:37.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:37 smithi060 ceph-mon[28873]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-03T14:42:37.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:37 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-03T14:42:37.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:37 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:42:37.614 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:37 smithi090 ceph-mon[33568]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-03T14:42:37.614 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:37 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-03T14:42:37.614 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:37 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:42:37.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:37 smithi149 ceph-mon[33332]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-03T14:42:37.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:37 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-03T14:42:37.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:37 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:42:38.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:38 smithi090 ceph-mon[33568]: Deploying daemon osd.10 on smithi090 2024-05-03T14:42:38.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:38 smithi149 ceph-mon[33332]: Deploying daemon osd.10 on smithi090 2024-05-03T14:42:38.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:38 smithi060 ceph-mon[28873]: Deploying daemon osd.10 on smithi090 2024-05-03T14:42:39.525 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:39 smithi090 ceph-mon[33568]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-03T14:42:39.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:39 smithi149 ceph-mon[33332]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-03T14:42:39.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:39 smithi060 ceph-mon[28873]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-03T14:42:41.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:41 smithi149 ceph-mon[33332]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-03T14:42:41.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:41 smithi090 ceph-mon[33568]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-03T14:42:41.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:41 smithi060 ceph-mon[28873]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-03T14:42:43.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:43 smithi149 ceph-mon[33332]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-03T14:42:43.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:43 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:43.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:43 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:42:43.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:43 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:44.038 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:43 smithi090 ceph-mon[33568]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-03T14:42:44.038 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:43 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:44.038 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:43 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:42:44.038 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:43 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:44.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:43 smithi060 ceph-mon[28873]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-03T14:42:44.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:43 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:44.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:43 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:42:44.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:43 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:45.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:45 smithi090 ceph-mon[33568]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-03T14:42:45.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:45 smithi149 ceph-mon[33332]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-03T14:42:46.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:45 smithi060 ceph-mon[28873]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-03T14:42:46.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:42:46.306+0000 7fbc87fff700 1 -- 172.21.15.90:0/3093147288 <== mgr.14152 v2:172.21.15.60:6800/3066142956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fbca00638e0 con 0x7fbc880b2880 2024-05-03T14:42:46.308 INFO:teuthology.orchestra.run.smithi090.stdout:Created osd(s) 10 on host 'smithi090' 2024-05-03T14:42:46.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:42:46.309+0000 7fbca5176700 1 -- 172.21.15.90:0/3093147288 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fbc880b2880 msgr2=0x7fbc880b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:42:46.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:42:46.309+0000 7fbca5176700 1 --2- 172.21.15.90:0/3093147288 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fbc880b2880 0x7fbc880b4d40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbc9000b580 tx=0x7fbc90005fb0 comp rx=0 tx=0).stop 2024-05-03T14:42:46.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:42:46.309+0000 7fbca5176700 1 -- 172.21.15.90:0/3093147288 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbca0103fa0 msgr2=0x7fbca01a0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:42:46.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:42:46.309+0000 7fbca5176700 1 --2- 172.21.15.90:0/3093147288 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbca0103fa0 0x7fbca01a0d30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbc9800b970 tx=0x7fbc9800bc80 comp rx=0 tx=0).stop 2024-05-03T14:42:46.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:42:46.309+0000 7fbca5176700 1 -- 172.21.15.90:0/3093147288 shutdown_connections 2024-05-03T14:42:46.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:42:46.309+0000 7fbca5176700 1 --2- 172.21.15.90:0/3093147288 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fbc880b2880 0x7fbc880b4d40 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:42:46.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:42:46.309+0000 7fbca5176700 1 --2- 172.21.15.90:0/3093147288 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbca0103fa0 0x7fbca01a0d30 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:42:46.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:42:46.309+0000 7fbca5176700 1 --2- 172.21.15.90:0/3093147288 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbca0102400 0x7fbca006c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:42:46.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:42:46.309+0000 7fbca5176700 1 --2- 172.21.15.90:0/3093147288 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbca0103600 0x7fbca019e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:42:46.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:42:46.309+0000 7fbca5176700 1 -- 172.21.15.90:0/3093147288 >> 172.21.15.90:0/3093147288 conn(0x7fbca00fd9a0 msgr2=0x7fbca0106900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:42:46.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:42:46.309+0000 7fbca5176700 1 -- 172.21.15.90:0/3093147288 shutdown_connections 2024-05-03T14:42:46.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-03T14:42:46.309+0000 7fbca5176700 1 -- 172.21.15.90:0/3093147288 wait complete. 2024-05-03T14:42:46.965 DEBUG:teuthology.orchestra.run.smithi090:osd.10> sudo journalctl -f -n 0 -u ceph-285d64b2-095a-11ef-bc97-c7b262605968@osd.10.service 2024-05-03T14:42:46.968 INFO:tasks.cephadm:Deploying osd.11 on smithi149 with /dev/vg_nvme/lv_1... 2024-05-03T14:42:46.968 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-03T14:42:47.217 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T14:42:47.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:47 smithi060 ceph-mon[28873]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-03T14:42:47.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:47 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:47.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:47 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:47.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:47 smithi060 ceph-mon[28873]: from='osd.10 [v2:172.21.15.90:6824/2492593590,v1:172.21.15.90:6825/2492593590]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-03T14:42:47.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:47 smithi060 ceph-mon[28873]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-03T14:42:47.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:47 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:47.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:47 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:47.624 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:47 smithi090 ceph-mon[33568]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-03T14:42:47.624 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:47 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:47.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:47 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:47.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:47 smithi090 ceph-mon[33568]: from='osd.10 [v2:172.21.15.90:6824/2492593590,v1:172.21.15.90:6825/2492593590]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-03T14:42:47.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:47 smithi090 ceph-mon[33568]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-03T14:42:47.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:47 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:47.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:47 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:47.632 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:47 smithi149 ceph-mon[33332]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-03T14:42:47.632 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:47 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:47.632 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:47 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:47.633 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:47 smithi149 ceph-mon[33332]: from='osd.10 [v2:172.21.15.90:6824/2492593590,v1:172.21.15.90:6825/2492593590]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-03T14:42:47.633 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:47 smithi149 ceph-mon[33332]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-03T14:42:47.633 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:47 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:47.633 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:47 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:48.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:48 smithi090 ceph-mon[33568]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-03T14:42:48.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:48 smithi090 ceph-mon[33568]: from='osd.10 [v2:172.21.15.90:6824/2492593590,v1:172.21.15.90:6825/2492593590]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-05-03T14:42:48.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:48 smithi090 ceph-mon[33568]: osdmap e62: 11 total, 10 up, 11 in 2024-05-03T14:42:48.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:48 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T14:42:48.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:48 smithi090 ceph-mon[33568]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-05-03T14:42:48.376 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 14:42:48 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T14:42:48.311+0000 7f7bec46f700 -1 osd.10 0 waiting for initial osdmap 2024-05-03T14:42:48.376 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 14:42:48 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T14:42:48.324+0000 7f7be8262700 -1 osd.10 63 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-03T14:42:48.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:48 smithi060 ceph-mon[28873]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-03T14:42:48.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:48 smithi060 ceph-mon[28873]: from='osd.10 [v2:172.21.15.90:6824/2492593590,v1:172.21.15.90:6825/2492593590]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-05-03T14:42:48.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:48 smithi060 ceph-mon[28873]: osdmap e62: 11 total, 10 up, 11 in 2024-05-03T14:42:48.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:48 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T14:42:48.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:48 smithi060 ceph-mon[28873]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-05-03T14:42:48.662 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:48 smithi149 ceph-mon[33332]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-03T14:42:48.662 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:48 smithi149 ceph-mon[33332]: from='osd.10 [v2:172.21.15.90:6824/2492593590,v1:172.21.15.90:6825/2492593590]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-05-03T14:42:48.663 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:48 smithi149 ceph-mon[33332]: osdmap e62: 11 total, 10 up, 11 in 2024-05-03T14:42:48.663 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:48 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T14:42:48.663 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:48 smithi149 ceph-mon[33332]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-05-03T14:42:49.422 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-03T14:42:49.422 INFO:teuthology.orchestra.run.smithi149.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-03T14:42:49.422 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10+0 records in 2024-05-03T14:42:49.423 INFO:teuthology.orchestra.run.smithi149.stderr:10+0 records out 2024-05-03T14:42:49.423 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114001 s, 920 MB/s 2024-05-03T14:42:49.423 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping successful for: 2024-05-03T14:42:49.482 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:49 smithi090 ceph-mon[33568]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-03T14:42:49.483 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:49 smithi090 ceph-mon[33568]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-05-03T14:42:49.483 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:49 smithi090 ceph-mon[33568]: osdmap e63: 11 total, 10 up, 11 in 2024-05-03T14:42:49.483 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:49 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T14:42:49.483 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:49 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T14:42:49.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:49 smithi060 ceph-mon[28873]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-03T14:42:49.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:49 smithi060 ceph-mon[28873]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-05-03T14:42:49.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:49 smithi060 ceph-mon[28873]: osdmap e63: 11 total, 10 up, 11 in 2024-05-03T14:42:49.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:49 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T14:42:49.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:49 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T14:42:49.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:49 smithi149 ceph-mon[33332]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-03T14:42:49.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:49 smithi149 ceph-mon[33332]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-05-03T14:42:49.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:49 smithi149 ceph-mon[33332]: osdmap e63: 11 total, 10 up, 11 in 2024-05-03T14:42:49.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:49 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T14:42:49.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:49 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T14:42:50.058 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi149:vg_nvme/lv_1 2024-05-03T14:42:50.302 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T14:42:50.619 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:50 smithi090 ceph-mon[33568]: purged_snaps scrub starts 2024-05-03T14:42:50.620 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:50 smithi090 ceph-mon[33568]: purged_snaps scrub ok 2024-05-03T14:42:50.620 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:50 smithi090 ceph-mon[33568]: osd.10 [v2:172.21.15.90:6824/2492593590,v1:172.21.15.90:6825/2492593590] boot 2024-05-03T14:42:50.620 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:50 smithi090 ceph-mon[33568]: osdmap e64: 11 total, 11 up, 11 in 2024-05-03T14:42:50.620 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:50 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T14:42:50.620 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:50 smithi090 ceph-mon[33568]: osdmap e65: 11 total, 11 up, 11 in 2024-05-03T14:42:50.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:50 smithi149 ceph-mon[33332]: purged_snaps scrub starts 2024-05-03T14:42:50.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:50 smithi149 ceph-mon[33332]: purged_snaps scrub ok 2024-05-03T14:42:50.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:50 smithi149 ceph-mon[33332]: osd.10 [v2:172.21.15.90:6824/2492593590,v1:172.21.15.90:6825/2492593590] boot 2024-05-03T14:42:50.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:50 smithi149 ceph-mon[33332]: osdmap e64: 11 total, 11 up, 11 in 2024-05-03T14:42:50.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:50 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T14:42:50.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:50 smithi149 ceph-mon[33332]: osdmap e65: 11 total, 11 up, 11 in 2024-05-03T14:42:50.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:50 smithi060 ceph-mon[28873]: purged_snaps scrub starts 2024-05-03T14:42:50.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:50 smithi060 ceph-mon[28873]: purged_snaps scrub ok 2024-05-03T14:42:50.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:50 smithi060 ceph-mon[28873]: osd.10 [v2:172.21.15.90:6824/2492593590,v1:172.21.15.90:6825/2492593590] boot 2024-05-03T14:42:50.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:50 smithi060 ceph-mon[28873]: osdmap e64: 11 total, 11 up, 11 in 2024-05-03T14:42:50.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:50 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T14:42:50.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:50 smithi060 ceph-mon[28873]: osdmap e65: 11 total, 11 up, 11 in 2024-05-03T14:42:51.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:51 smithi149 ceph-mon[33332]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-03T14:42:51.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:51 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:51.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:51 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:51.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:51 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:42:51.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:51 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:42:51.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:51 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:42:51.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:51 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:42:51.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:51 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:51.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:51 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:42:51.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:51 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:42:51.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:51 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:51.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:51 smithi149 ceph-mon[33332]: osdmap e66: 11 total, 11 up, 11 in 2024-05-03T14:42:51.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:51 smithi090 ceph-mon[33568]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-03T14:42:51.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:51 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:51.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:51 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:51.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:51 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:42:51.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:51 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:42:51.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:51 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:42:51.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:51 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:42:51.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:51 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:51.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:51 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:42:51.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:51 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:42:51.626 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:51 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:51.626 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:51 smithi090 ceph-mon[33568]: osdmap e66: 11 total, 11 up, 11 in 2024-05-03T14:42:51.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:51 smithi060 ceph-mon[28873]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-03T14:42:51.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:51 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:51.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:51 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:51.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:51 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:42:51.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:51 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:42:51.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:51 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:42:51.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:51 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:42:51.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:51 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:51.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:51 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:42:51.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:51 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:42:51.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:51 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:51.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:51 smithi060 ceph-mon[28873]: osdmap e66: 11 total, 11 up, 11 in 2024-05-03T14:42:52.042 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:42:52.039+0000 7f2f73df3700 1 -- 172.21.15.149:0/2121024648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f6c102dd0 msgr2=0x7f2f6c103230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:42:52.042 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:42:52.039+0000 7f2f73df3700 1 --2- 172.21.15.149:0/2121024648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f6c102dd0 0x7f2f6c103230 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c009990 tx=0x7f2f5c009ca0 comp rx=0 tx=0).stop 2024-05-03T14:42:52.042 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:42:52.039+0000 7f2f73df3700 1 -- 172.21.15.149:0/2121024648 shutdown_connections 2024-05-03T14:42:52.042 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:42:52.039+0000 7f2f73df3700 1 --2- 172.21.15.149:0/2121024648 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f6c103770 0x7f2f6c10a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:42:52.042 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:42:52.039+0000 7f2f73df3700 1 --2- 172.21.15.149:0/2121024648 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f6c101bd0 0x7f2f6c101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:42:52.042 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:42:52.039+0000 7f2f73df3700 1 --2- 172.21.15.149:0/2121024648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f6c102dd0 0x7f2f6c103230 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:42:52.042 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:42:52.039+0000 7f2f73df3700 1 -- 172.21.15.149:0/2121024648 >> 172.21.15.149:0/2121024648 conn(0x7f2f6c0fd170 msgr2=0x7f2f6c0ff5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:42:52.042 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:42:52.039+0000 7f2f73df3700 1 -- 172.21.15.149:0/2121024648 shutdown_connections 2024-05-03T14:42:52.043 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:42:52.039+0000 7f2f73df3700 1 -- 172.21.15.149:0/2121024648 wait complete. 2024-05-03T14:42:52.044 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:42:52.039+0000 7f2f73df3700 1 Processor -- start 2024-05-03T14:42:52.044 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:42:52.039+0000 7f2f73df3700 1 -- start start 2024-05-03T14:42:52.044 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:42:52.040+0000 7f2f73df3700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f6c101bd0 0x7f2f6c199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:42:52.044 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:42:52.040+0000 7f2f73df3700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f6c102dd0 0x7f2f6c199d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:42:52.044 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:42:52.040+0000 7f2f73df3700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f6c103770 0x7f2f6c1a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:42:52.045 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:42:52.040+0000 7f2f73df3700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6c10e1d0 con 0x7f2f6c102dd0 2024-05-03T14:42:52.045 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:42:52.040+0000 7f2f73df3700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6c10e050 con 0x7f2f6c103770 2024-05-03T14:42:52.045 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:42:52.040+0000 7f2f73df3700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6c10e350 con 0x7f2f6c101bd0 2024-05-03T14:42:52.045 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:42:52.040+0000 7f2f71b8f700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f6c101bd0 0x7f2f6c199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:42:52.045 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:42:52.040+0000 7f2f71b8f700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f6c101bd0 0x7f2f6c199840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:34630/0 (socket says 172.21.15.149:34630) 2024-05-03T14:42:52.045 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:42:52.040+0000 7f2f71b8f700 1 -- 172.21.15.149:0/3735198484 learned_addr learned my addr 172.21.15.149:0/3735198484 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-05-03T14:42:52.045 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:42:52.040+0000 7f2f72390700 1 --2- 172.21.15.149:0/3735198484 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f6c103770 0x7f2f6c1a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:42:52.045 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:42:52.040+0000 7f2f71b8f700 1 -- 172.21.15.149:0/3735198484 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f6c103770 msgr2=0x7f2f6c1a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:42:52.045 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:42:52.040+0000 7f2f71b8f700 1 --2- 172.21.15.149:0/3735198484 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f6c103770 0x7f2f6c1a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:42:52.045 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:42:52.040+0000 7f2f71b8f700 1 -- 172.21.15.149:0/3735198484 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f6c102dd0 msgr2=0x7f2f6c199d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:42:52.045 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:42:52.040+0000 7f2f71b8f700 1 --2- 172.21.15.149:0/3735198484 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f6c102dd0 0x7f2f6c199d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:42:52.045 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:42:52.040+0000 7f2f71b8f700 1 -- 172.21.15.149:0/3735198484 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f6c1a14a0 con 0x7f2f6c101bd0 2024-05-03T14:42:52.045 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:42:52.040+0000 7f2f71b8f700 1 --2- 172.21.15.149:0/3735198484 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f6c101bd0 0x7f2f6c199840 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2f6800c9a0 tx=0x7f2f6800ccb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:42:52.046 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:42:52.040+0000 7f2f62ffd700 1 -- 172.21.15.149:0/3735198484 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f68004220 con 0x7f2f6c101bd0 2024-05-03T14:42:52.046 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:42:52.040+0000 7f2f73df3700 1 -- 172.21.15.149:0/3735198484 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f6c1a1790 con 0x7f2f6c101bd0 2024-05-03T14:42:52.046 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:42:52.040+0000 7f2f62ffd700 1 -- 172.21.15.149:0/3735198484 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f2f68007d60 con 0x7f2f6c101bd0 2024-05-03T14:42:52.046 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:42:52.040+0000 7f2f62ffd700 1 -- 172.21.15.149:0/3735198484 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f6800f450 con 0x7f2f6c101bd0 2024-05-03T14:42:52.046 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:42:52.041+0000 7f2f73df3700 1 -- 172.21.15.149:0/3735198484 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f6c1a1cd0 con 0x7f2f6c101bd0 2024-05-03T14:42:52.046 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:42:52.041+0000 7f2f73df3700 1 -- 172.21.15.149:0/3735198484 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f6c06a190 con 0x7f2f6c101bd0 2024-05-03T14:42:52.047 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:42:52.043+0000 7f2f62ffd700 1 -- 172.21.15.149:0/3735198484 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f2f68007860 con 0x7f2f6c101bd0 2024-05-03T14:42:52.047 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:42:52.043+0000 7f2f62ffd700 1 --2- 172.21.15.149:0/3735198484 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f2f580b2880 0x7f2f580b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:42:52.047 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:42:52.044+0000 7f2f62ffd700 1 -- 172.21.15.149:0/3735198484 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6694+0+0 (secure 0 0 0) 0x7f2f680e1b40 con 0x7f2f6c101bd0 2024-05-03T14:42:52.047 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:42:52.044+0000 7f2f7138e700 1 --2- 172.21.15.149:0/3735198484 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f2f580b2880 0x7f2f580b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:42:52.048 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:42:52.045+0000 7f2f7138e700 1 --2- 172.21.15.149:0/3735198484 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f2f580b2880 0x7f2f580b4d40 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c005f50 tx=0x7f2f5c005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:42:52.050 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:42:52.047+0000 7f2f62ffd700 1 -- 172.21.15.149:0/3735198484 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f2f68014030 con 0x7f2f6c101bd0 2024-05-03T14:42:52.261 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:42:52.258+0000 7f2f73df3700 1 -- 172.21.15.149:0/3735198484 --> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f2f6c0ff210 con 0x7f2f580b2880 2024-05-03T14:42:52.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:52 smithi090 ceph-mon[33568]: Detected new or changed devices on smithi090 2024-05-03T14:42:52.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:52 smithi090 ceph-mon[33568]: Adjusting osd_memory_target on smithi090 to 4272M 2024-05-03T14:42:52.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:52 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:42:52.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:52 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:42:52.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:52 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:42:52.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:52 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:52.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:52 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T14:42:52.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:52 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T14:42:52.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:52 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:42:52.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:52 smithi149 ceph-mon[33332]: Detected new or changed devices on smithi090 2024-05-03T14:42:52.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:52 smithi149 ceph-mon[33332]: Adjusting osd_memory_target on smithi090 to 4272M 2024-05-03T14:42:52.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:52 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:42:52.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:52 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:42:52.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:52 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:42:52.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:52 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:52.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:52 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T14:42:52.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:52 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T14:42:52.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:52 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:42:52.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:52 smithi060 ceph-mon[28873]: Detected new or changed devices on smithi090 2024-05-03T14:42:52.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:52 smithi060 ceph-mon[28873]: Adjusting osd_memory_target on smithi090 to 4272M 2024-05-03T14:42:52.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:52 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:42:52.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:52 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:42:52.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:52 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:42:52.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:52 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:42:52.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:52 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T14:42:52.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:52 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T14:42:52.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:52 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:42:53.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:53 smithi149 ceph-mon[33332]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:42:53.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:53 smithi149 ceph-mon[33332]: pgmap v260: 1 pgs: 1 peering; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-03T14:42:53.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:53 smithi060 ceph-mon[28873]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:42:53.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:53 smithi060 ceph-mon[28873]: pgmap v260: 1 pgs: 1 peering; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-03T14:42:53.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:53 smithi090 ceph-mon[33568]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:42:53.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:53 smithi090 ceph-mon[33568]: pgmap v260: 1 pgs: 1 peering; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-03T14:42:55.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:55 smithi149 ceph-mon[33332]: pgmap v261: 1 pgs: 1 peering; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-03T14:42:55.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:55 smithi149 ceph-mon[33332]: from='client.? 172.21.15.149:0/1416949804' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d17ec1a-bdd5-41f1-9b59-c2b6a527bfbb"}]: dispatch 2024-05-03T14:42:55.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:55 smithi149 ceph-mon[33332]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d17ec1a-bdd5-41f1-9b59-c2b6a527bfbb"}]: dispatch 2024-05-03T14:42:55.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:55 smithi149 ceph-mon[33332]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8d17ec1a-bdd5-41f1-9b59-c2b6a527bfbb"}]': finished 2024-05-03T14:42:55.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:55 smithi149 ceph-mon[33332]: osdmap e67: 12 total, 11 up, 12 in 2024-05-03T14:42:55.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:55 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T14:42:55.689 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:55 smithi090 ceph-mon[33568]: pgmap v261: 1 pgs: 1 peering; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-03T14:42:55.689 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:55 smithi090 ceph-mon[33568]: from='client.? 172.21.15.149:0/1416949804' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d17ec1a-bdd5-41f1-9b59-c2b6a527bfbb"}]: dispatch 2024-05-03T14:42:55.689 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:55 smithi090 ceph-mon[33568]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d17ec1a-bdd5-41f1-9b59-c2b6a527bfbb"}]: dispatch 2024-05-03T14:42:55.689 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:55 smithi090 ceph-mon[33568]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8d17ec1a-bdd5-41f1-9b59-c2b6a527bfbb"}]': finished 2024-05-03T14:42:55.689 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:55 smithi090 ceph-mon[33568]: osdmap e67: 12 total, 11 up, 12 in 2024-05-03T14:42:55.689 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:55 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T14:42:55.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:55 smithi060 ceph-mon[28873]: pgmap v261: 1 pgs: 1 peering; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-03T14:42:55.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:55 smithi060 ceph-mon[28873]: from='client.? 172.21.15.149:0/1416949804' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d17ec1a-bdd5-41f1-9b59-c2b6a527bfbb"}]: dispatch 2024-05-03T14:42:55.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:55 smithi060 ceph-mon[28873]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d17ec1a-bdd5-41f1-9b59-c2b6a527bfbb"}]: dispatch 2024-05-03T14:42:55.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:55 smithi060 ceph-mon[28873]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8d17ec1a-bdd5-41f1-9b59-c2b6a527bfbb"}]': finished 2024-05-03T14:42:55.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:55 smithi060 ceph-mon[28873]: osdmap e67: 12 total, 11 up, 12 in 2024-05-03T14:42:55.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:55 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T14:42:56.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:56 smithi060 ceph-mon[28873]: from='client.? 172.21.15.149:0/438439893' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T14:42:56.866 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:56 smithi090 ceph-mon[33568]: from='client.? 172.21.15.149:0/438439893' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T14:42:56.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:56 smithi149 ceph-mon[33332]: from='client.? 172.21.15.149:0/438439893' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T14:42:57.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:57 smithi060 ceph-mon[28873]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-03T14:42:57.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:57 smithi090 ceph-mon[33568]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-03T14:42:57.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:57 smithi149 ceph-mon[33332]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-03T14:42:59.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:42:59 smithi060 ceph-mon[28873]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-03T14:42:59.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:42:59 smithi090 ceph-mon[33568]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-03T14:42:59.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:42:59 smithi149 ceph-mon[33332]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-03T14:43:01.542 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:01 smithi149 ceph-mon[33332]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-03T14:43:01.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:01 smithi060 ceph-mon[28873]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-03T14:43:01.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:01 smithi090 ceph-mon[33568]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-03T14:43:03.601 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:03 smithi149 ceph-mon[33332]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T14:43:03.601 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:03 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-03T14:43:03.601 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:03 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:43:03.601 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:03 smithi149 ceph-mon[33332]: Deploying daemon osd.11 on smithi149 2024-05-03T14:43:03.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:03 smithi060 ceph-mon[28873]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T14:43:03.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:03 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-03T14:43:03.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:03 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:43:03.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:03 smithi060 ceph-mon[28873]: Deploying daemon osd.11 on smithi149 2024-05-03T14:43:03.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:03 smithi090 ceph-mon[33568]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T14:43:03.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:03 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-03T14:43:03.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:03 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:43:03.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:03 smithi090 ceph-mon[33568]: Deploying daemon osd.11 on smithi149 2024-05-03T14:43:05.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:05 smithi149 ceph-mon[33332]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T14:43:05.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:05 smithi060 ceph-mon[28873]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T14:43:05.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:05 smithi090 ceph-mon[33568]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T14:43:07.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:07 smithi060 ceph-mon[28873]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-05-03T14:43:07.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:07 smithi090 ceph-mon[33568]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-05-03T14:43:07.917 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:07 smithi149 ceph-mon[33332]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-05-03T14:43:09.530 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:09 smithi149 ceph-mon[33332]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-03T14:43:09.530 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:09 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:43:09.530 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:09 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:09.530 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:09 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:09.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:09 smithi060 ceph-mon[28873]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-03T14:43:09.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:09 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:43:09.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:09 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:09.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:09 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:09.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:09 smithi090 ceph-mon[33568]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-03T14:43:09.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:09 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:43:09.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:09 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:09.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:09 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:11.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:11 smithi149 ceph-mon[33332]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-03T14:43:11.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:11 smithi060 ceph-mon[28873]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-03T14:43:11.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:11 smithi090 ceph-mon[33568]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-03T14:43:12.658 INFO:teuthology.orchestra.run.smithi149.stdout:Created osd(s) 11 on host 'smithi149' 2024-05-03T14:43:12.659 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:12.656+0000 7f2f62ffd700 1 -- 172.21.15.149:0/3735198484 <== mgr.14152 v2:172.21.15.60:6800/3066142956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f2f6c0ff210 con 0x7f2f580b2880 2024-05-03T14:43:12.661 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:12.659+0000 7f2f73df3700 1 -- 172.21.15.149:0/3735198484 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f2f580b2880 msgr2=0x7f2f580b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:43:12.661 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:12.659+0000 7f2f73df3700 1 --2- 172.21.15.149:0/3735198484 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f2f580b2880 0x7f2f580b4d40 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c005f50 tx=0x7f2f5c005e80 comp rx=0 tx=0).stop 2024-05-03T14:43:12.661 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:12.659+0000 7f2f73df3700 1 -- 172.21.15.149:0/3735198484 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f6c101bd0 msgr2=0x7f2f6c199840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:43:12.661 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:12.659+0000 7f2f73df3700 1 --2- 172.21.15.149:0/3735198484 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f6c101bd0 0x7f2f6c199840 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2f6800c9a0 tx=0x7f2f6800ccb0 comp rx=0 tx=0).stop 2024-05-03T14:43:12.661 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:12.659+0000 7f2f73df3700 1 -- 172.21.15.149:0/3735198484 shutdown_connections 2024-05-03T14:43:12.661 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:12.659+0000 7f2f73df3700 1 --2- 172.21.15.149:0/3735198484 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f2f580b2880 0x7f2f580b4d40 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:12.661 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:12.659+0000 7f2f73df3700 1 --2- 172.21.15.149:0/3735198484 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f6c103770 0x7f2f6c1a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:12.661 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:12.659+0000 7f2f73df3700 1 --2- 172.21.15.149:0/3735198484 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f6c101bd0 0x7f2f6c199840 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:12.661 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:12.659+0000 7f2f73df3700 1 --2- 172.21.15.149:0/3735198484 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f6c102dd0 0x7f2f6c199d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:12.661 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:12.659+0000 7f2f73df3700 1 -- 172.21.15.149:0/3735198484 >> 172.21.15.149:0/3735198484 conn(0x7f2f6c0fd170 msgr2=0x7f2f6c0feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:43:12.661 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:12.659+0000 7f2f73df3700 1 -- 172.21.15.149:0/3735198484 shutdown_connections 2024-05-03T14:43:12.662 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:12.659+0000 7f2f73df3700 1 -- 172.21.15.149:0/3735198484 wait complete. 2024-05-03T14:43:13.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:12 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:13.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:12 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:13.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:12 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:43:13.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:12 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:43:13.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:12 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:13.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:12 smithi090 ceph-mon[33568]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-03T14:43:13.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:12 smithi090 ceph-mon[33568]: from='osd.11 [v2:172.21.15.149:6824/1063245126,v1:172.21.15.149:6825/1063245126]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-03T14:43:13.126 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:12 smithi090 ceph-mon[33568]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-03T14:43:13.126 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:12 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:13.126 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:12 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:13.126 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:12 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:43:13.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:12 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:13.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:12 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:13.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:12 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:43:13.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:12 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:43:13.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:12 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:13.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:12 smithi149 ceph-mon[33332]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-03T14:43:13.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:12 smithi149 ceph-mon[33332]: from='osd.11 [v2:172.21.15.149:6824/1063245126,v1:172.21.15.149:6825/1063245126]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-03T14:43:13.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:12 smithi149 ceph-mon[33332]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-03T14:43:13.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:12 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:13.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:12 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:13.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:12 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:43:13.329 DEBUG:teuthology.orchestra.run.smithi149:osd.11> sudo journalctl -f -n 0 -u ceph-285d64b2-095a-11ef-bc97-c7b262605968@osd.11.service 2024-05-03T14:43:13.331 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-05-03T14:43:13.331 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph osd stat -f json 2024-05-03T14:43:13.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:12 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:13.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:12 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:13.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:12 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:43:13.340 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:12 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:43:13.340 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:12 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:13.340 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:12 smithi060 ceph-mon[28873]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-03T14:43:13.340 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:12 smithi060 ceph-mon[28873]: from='osd.11 [v2:172.21.15.149:6824/1063245126,v1:172.21.15.149:6825/1063245126]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-03T14:43:13.340 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:12 smithi060 ceph-mon[28873]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-03T14:43:13.340 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:12 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:13.340 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:12 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:13.340 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:12 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:43:13.815 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:43:14.143 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:13 smithi060 ceph-mon[28873]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-03T14:43:14.143 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:13 smithi060 ceph-mon[28873]: from='osd.11 [v2:172.21.15.149:6824/1063245126,v1:172.21.15.149:6825/1063245126]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-03T14:43:14.143 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:13 smithi060 ceph-mon[28873]: osdmap e68: 12 total, 11 up, 12 in 2024-05-03T14:43:14.143 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:13 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T14:43:14.143 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:13 smithi060 ceph-mon[28873]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-03T14:43:14.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:13 smithi149 ceph-mon[33332]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-03T14:43:14.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:13 smithi149 ceph-mon[33332]: from='osd.11 [v2:172.21.15.149:6824/1063245126,v1:172.21.15.149:6825/1063245126]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-03T14:43:14.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:13 smithi149 ceph-mon[33332]: osdmap e68: 12 total, 11 up, 12 in 2024-05-03T14:43:14.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:13 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T14:43:14.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:13 smithi149 ceph-mon[33332]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-03T14:43:14.174 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 14:43:13 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T14:43:13.878+0000 7f0a18219700 -1 osd.11 0 waiting for initial osdmap 2024-05-03T14:43:14.174 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 14:43:13 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T14:43:13.890+0000 7f0a1400c700 -1 osd.11 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-03T14:43:14.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:13 smithi090 ceph-mon[33568]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-03T14:43:14.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:13 smithi090 ceph-mon[33568]: from='osd.11 [v2:172.21.15.149:6824/1063245126,v1:172.21.15.149:6825/1063245126]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-03T14:43:14.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:13 smithi090 ceph-mon[33568]: osdmap e68: 12 total, 11 up, 12 in 2024-05-03T14:43:14.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:13 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T14:43:14.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:13 smithi090 ceph-mon[33568]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-03T14:43:15.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:14 smithi149 ceph-mon[33332]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-05-03T14:43:15.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:14 smithi149 ceph-mon[33332]: osdmap e69: 12 total, 11 up, 12 in 2024-05-03T14:43:15.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:14 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T14:43:15.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:14 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T14:43:15.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:14 smithi149 ceph-mon[33332]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-03T14:43:15.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:14 smithi060 ceph-mon[28873]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-05-03T14:43:15.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:14 smithi060 ceph-mon[28873]: osdmap e69: 12 total, 11 up, 12 in 2024-05-03T14:43:15.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:14 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T14:43:15.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:14 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T14:43:15.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:14 smithi060 ceph-mon[28873]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-03T14:43:15.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:14 smithi090 ceph-mon[33568]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-05-03T14:43:15.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:14 smithi090 ceph-mon[33568]: osdmap e69: 12 total, 11 up, 12 in 2024-05-03T14:43:15.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:14 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T14:43:15.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:14 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T14:43:15.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:14 smithi090 ceph-mon[33568]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-03T14:43:15.973 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:15 smithi060 ceph-mon[28873]: purged_snaps scrub starts 2024-05-03T14:43:15.973 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:15 smithi060 ceph-mon[28873]: purged_snaps scrub ok 2024-05-03T14:43:15.973 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:15 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T14:43:15.973 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:15 smithi060 ceph-mon[28873]: osd.11 [v2:172.21.15.149:6824/1063245126,v1:172.21.15.149:6825/1063245126] boot 2024-05-03T14:43:15.973 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:15 smithi060 ceph-mon[28873]: osdmap e70: 12 total, 12 up, 12 in 2024-05-03T14:43:15.974 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:15 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T14:43:16.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:15 smithi149 ceph-mon[33332]: purged_snaps scrub starts 2024-05-03T14:43:16.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:15 smithi149 ceph-mon[33332]: purged_snaps scrub ok 2024-05-03T14:43:16.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:15 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T14:43:16.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:15 smithi149 ceph-mon[33332]: osd.11 [v2:172.21.15.149:6824/1063245126,v1:172.21.15.149:6825/1063245126] boot 2024-05-03T14:43:16.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:15 smithi149 ceph-mon[33332]: osdmap e70: 12 total, 12 up, 12 in 2024-05-03T14:43:16.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:15 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T14:43:16.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:16.311+0000 7fd001c00700 1 -- 172.21.15.60:0/2750366536 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcffc103ec0 msgr2=0x7fcffc1062b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:43:16.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:16.311+0000 7fd001c00700 1 --2- 172.21.15.60:0/2750366536 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcffc103ec0 0x7fcffc1062b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcfec009990 tx=0x7fcfec009ca0 comp rx=0 tx=0).stop 2024-05-03T14:43:16.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:16.311+0000 7fd001c00700 1 -- 172.21.15.60:0/2750366536 shutdown_connections 2024-05-03T14:43:16.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:16.311+0000 7fd001c00700 1 --2- 172.21.15.60:0/2750366536 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcffc109120 0x7fcffc10b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:16.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:16.311+0000 7fd001c00700 1 --2- 172.21.15.60:0/2750366536 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcffc103ec0 0x7fcffc1062b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:16.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:16.311+0000 7fd001c00700 1 --2- 172.21.15.60:0/2750366536 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcffc1067f0 0x7fcffc108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:16.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:16.311+0000 7fd001c00700 1 -- 172.21.15.60:0/2750366536 >> 172.21.15.60:0/2750366536 conn(0x7fcffc0fd8d0 msgr2=0x7fcffc0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:43:16.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:16.311+0000 7fd001c00700 1 -- 172.21.15.60:0/2750366536 shutdown_connections 2024-05-03T14:43:16.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:16.311+0000 7fd001c00700 1 -- 172.21.15.60:0/2750366536 wait complete. 2024-05-03T14:43:16.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:16.312+0000 7fd001c00700 1 Processor -- start 2024-05-03T14:43:16.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:16.312+0000 7fd001c00700 1 -- start start 2024-05-03T14:43:16.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:16.313+0000 7fd001c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcffc103ec0 0x7fcffc19b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:43:16.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:16.313+0000 7fd001c00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcffc1067f0 0x7fcffc19be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:43:16.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:16.313+0000 7fd001c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcffc109120 0x7fcffc1a2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:43:16.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:16.313+0000 7fd001c00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fcffc10f190 con 0x7fcffc1067f0 2024-05-03T14:43:16.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:16.313+0000 7fd001c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fcffc10f010 con 0x7fcffc103ec0 2024-05-03T14:43:16.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:16.313+0000 7fd001c00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcffc10f310 con 0x7fcffc109120 2024-05-03T14:43:16.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:16.313+0000 7fcffb400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcffc1067f0 0x7fcffc19be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:43:16.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:16.313+0000 7fcffb400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcffc1067f0 0x7fcffc19be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36832/0 (socket says 172.21.15.60:36832) 2024-05-03T14:43:16.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:16.313+0000 7fcffb400700 1 -- 172.21.15.60:0/2694313775 learned_addr learned my addr 172.21.15.60:0/2694313775 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:43:16.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:16.313+0000 7fd001200700 1 --2- 172.21.15.60:0/2694313775 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcffc109120 0x7fcffc1a2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:43:16.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:16.313+0000 7fcffbe00700 1 --2- 172.21.15.60:0/2694313775 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcffc103ec0 0x7fcffc19b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:43:16.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:16.313+0000 7fcffb400700 1 -- 172.21.15.60:0/2694313775 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcffc109120 msgr2=0x7fcffc1a2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:43:16.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:16.313+0000 7fcffb400700 1 --2- 172.21.15.60:0/2694313775 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcffc109120 0x7fcffc1a2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:16.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:16.313+0000 7fcffb400700 1 -- 172.21.15.60:0/2694313775 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcffc103ec0 msgr2=0x7fcffc19b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:43:16.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:16.313+0000 7fcffb400700 1 --2- 172.21.15.60:0/2694313775 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcffc103ec0 0x7fcffc19b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:16.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:16.314+0000 7fcffb400700 1 -- 172.21.15.60:0/2694313775 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcffc1a35b0 con 0x7fcffc1067f0 2024-05-03T14:43:16.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:16.314+0000 7fcffbe00700 1 --2- 172.21.15.60:0/2694313775 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcffc103ec0 0x7fcffc19b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:43:16.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:16.314+0000 7fcffb400700 1 --2- 172.21.15.60:0/2694313775 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcffc1067f0 0x7fcffc19be90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcfe800e9d0 tx=0x7fcfe800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:43:16.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:16.314+0000 7fcff8c00700 1 -- 172.21.15.60:0/2694313775 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe800cbd0 con 0x7fcffc1067f0 2024-05-03T14:43:16.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:16.314+0000 7fcff8c00700 1 -- 172.21.15.60:0/2694313775 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7fcfe800ee70 con 0x7fcffc1067f0 2024-05-03T14:43:16.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:16.314+0000 7fcff8c00700 1 -- 172.21.15.60:0/2694313775 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe8018710 con 0x7fcffc1067f0 2024-05-03T14:43:16.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:16.314+0000 7fd001c00700 1 -- 172.21.15.60:0/2694313775 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcffc1a38a0 con 0x7fcffc1067f0 2024-05-03T14:43:16.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:16.314+0000 7fd001c00700 1 -- 172.21.15.60:0/2694313775 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcffc1a3d90 con 0x7fcffc1067f0 2024-05-03T14:43:16.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:16.315+0000 7fd001c00700 1 -- 172.21.15.60:0/2694313775 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcffc06a190 con 0x7fcffc1067f0 2024-05-03T14:43:16.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:16.319+0000 7fcff8c00700 1 -- 172.21.15.60:0/2694313775 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fcfe80188b0 con 0x7fcffc1067f0 2024-05-03T14:43:16.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:16.320+0000 7fcff8c00700 1 --2- 172.21.15.60:0/2694313775 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fcfe00b2900 0x7fcfe00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:43:16.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:16.320+0000 7fcff8c00700 1 -- 172.21.15.60:0/2694313775 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fcfe80e21c0 con 0x7fcffc1067f0 2024-05-03T14:43:16.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:16.320+0000 7fcffbe00700 1 --2- 172.21.15.60:0/2694313775 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fcfe00b2900 0x7fcfe00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:43:16.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:16.321+0000 7fcffbe00700 1 --2- 172.21.15.60:0/2694313775 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fcfe00b2900 0x7fcfe00b4dc0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcfec00b2e0 tx=0x7fcfec01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:43:16.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:16.323+0000 7fcff8c00700 1 -- 172.21.15.60:0/2694313775 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fcfe80aa2e0 con 0x7fcffc1067f0 2024-05-03T14:43:16.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:15 smithi090 ceph-mon[33568]: purged_snaps scrub starts 2024-05-03T14:43:16.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:15 smithi090 ceph-mon[33568]: purged_snaps scrub ok 2024-05-03T14:43:16.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:15 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T14:43:16.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:15 smithi090 ceph-mon[33568]: osd.11 [v2:172.21.15.149:6824/1063245126,v1:172.21.15.149:6825/1063245126] boot 2024-05-03T14:43:16.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:15 smithi090 ceph-mon[33568]: osdmap e70: 12 total, 12 up, 12 in 2024-05-03T14:43:16.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:15 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T14:43:16.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:16.572+0000 7fd001c00700 1 -- 172.21.15.60:0/2694313775 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fcffc02e510 con 0x7fcffc1067f0 2024-05-03T14:43:16.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:16.573+0000 7fcff8c00700 1 -- 172.21.15.60:0/2694313775 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v71) ==== 74+0+133 (secure 0 0 0) 0x7fcfe80b2360 con 0x7fcffc1067f0 2024-05-03T14:43:16.576 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:43:16.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:16.578+0000 7fd001c00700 1 -- 172.21.15.60:0/2694313775 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fcfe00b2900 msgr2=0x7fcfe00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:43:16.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:16.578+0000 7fd001c00700 1 --2- 172.21.15.60:0/2694313775 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fcfe00b2900 0x7fcfe00b4dc0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcfec00b2e0 tx=0x7fcfec01a040 comp rx=0 tx=0).stop 2024-05-03T14:43:16.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:16.578+0000 7fd001c00700 1 -- 172.21.15.60:0/2694313775 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcffc1067f0 msgr2=0x7fcffc19be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:43:16.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:16.578+0000 7fd001c00700 1 --2- 172.21.15.60:0/2694313775 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcffc1067f0 0x7fcffc19be90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcfe800e9d0 tx=0x7fcfe800ece0 comp rx=0 tx=0).stop 2024-05-03T14:43:16.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:16.578+0000 7fd001c00700 1 -- 172.21.15.60:0/2694313775 shutdown_connections 2024-05-03T14:43:16.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:16.578+0000 7fd001c00700 1 --2- 172.21.15.60:0/2694313775 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fcfe00b2900 0x7fcfe00b4dc0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:16.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:16.578+0000 7fd001c00700 1 --2- 172.21.15.60:0/2694313775 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcffc109120 0x7fcffc1a2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:16.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:16.578+0000 7fd001c00700 1 --2- 172.21.15.60:0/2694313775 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcffc103ec0 0x7fcffc19b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:16.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:16.578+0000 7fd001c00700 1 --2- 172.21.15.60:0/2694313775 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcffc1067f0 0x7fcffc19be90 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:16.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:16.578+0000 7fd001c00700 1 -- 172.21.15.60:0/2694313775 >> 172.21.15.60:0/2694313775 conn(0x7fcffc0fd8d0 msgr2=0x7fcffc103630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:43:16.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:16.578+0000 7fd001c00700 1 -- 172.21.15.60:0/2694313775 shutdown_connections 2024-05-03T14:43:16.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:16.578+0000 7fd001c00700 1 -- 172.21.15.60:0/2694313775 wait complete. 2024-05-03T14:43:17.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:16 smithi149 ceph-mon[33332]: osdmap e71: 12 total, 12 up, 12 in 2024-05-03T14:43:17.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:16 smithi149 ceph-mon[33332]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:17.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:16 smithi149 ceph-mon[33332]: Detected new or changed devices on smithi149 2024-05-03T14:43:17.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:16 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:17.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:16 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:17.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:16 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:43:17.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:16 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:43:17.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:16 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:43:17.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:16 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:43:17.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:16 smithi149 ceph-mon[33332]: Adjusting osd_memory_target on smithi149 to 1454M 2024-05-03T14:43:17.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:16 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:17.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:16 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:43:17.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:16 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:43:17.175 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:16 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:17.175 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:16 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/2694313775' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-03T14:43:17.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:16 smithi060 ceph-mon[28873]: osdmap e71: 12 total, 12 up, 12 in 2024-05-03T14:43:17.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:16 smithi060 ceph-mon[28873]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:17.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:16 smithi060 ceph-mon[28873]: Detected new or changed devices on smithi149 2024-05-03T14:43:17.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:16 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:17.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:16 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:17.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:16 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:43:17.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:16 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:43:17.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:16 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:43:17.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:16 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:43:17.340 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:16 smithi060 ceph-mon[28873]: Adjusting osd_memory_target on smithi149 to 1454M 2024-05-03T14:43:17.340 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:16 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:17.340 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:16 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:43:17.340 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:16 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:43:17.340 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:16 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:17.340 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:16 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/2694313775' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-03T14:43:17.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:16 smithi090 ceph-mon[33568]: osdmap e71: 12 total, 12 up, 12 in 2024-05-03T14:43:17.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:16 smithi090 ceph-mon[33568]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:17.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:16 smithi090 ceph-mon[33568]: Detected new or changed devices on smithi149 2024-05-03T14:43:17.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:16 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:17.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:16 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:17.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:16 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:43:17.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:16 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:43:17.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:16 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:43:17.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:16 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:43:17.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:16 smithi090 ceph-mon[33568]: Adjusting osd_memory_target on smithi149 to 1454M 2024-05-03T14:43:17.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:16 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:17.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:16 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:43:17.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:16 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:43:17.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:16 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:17.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:16 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/2694313775' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-03T14:43:17.471 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":71,"num_osds":12,"num_up_osds":12,"osd_up_since":1714747394,"num_in_osds":12,"osd_in_since":1714747375,"num_remapped_pgs":0} 2024-05-03T14:43:17.472 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-03T14:43:17.898 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:43:19.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:19 smithi060 ceph-mon[28873]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:19.624 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:19 smithi090 ceph-mon[33568]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:19.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:19 smithi149 ceph-mon[33332]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:20.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:20.142+0000 7fb640a00700 1 -- 172.21.15.60:0/2169257415 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb63c1055d0 msgr2=0x7fb63c105a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:43:20.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:20.142+0000 7fb640a00700 1 --2- 172.21.15.60:0/2169257415 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb63c1055d0 0x7fb63c105a50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb628009990 tx=0x7fb628009ca0 comp rx=0 tx=0).stop 2024-05-03T14:43:20.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:20.142+0000 7fb640a00700 1 -- 172.21.15.60:0/2169257415 shutdown_connections 2024-05-03T14:43:20.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:20.142+0000 7fb640a00700 1 --2- 172.21.15.60:0/2169257415 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb63c105f90 0x7fb63c1124a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:20.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:20.142+0000 7fb640a00700 1 --2- 172.21.15.60:0/2169257415 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb63c10b660 0x7fb63c10ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:20.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:20.142+0000 7fb640a00700 1 --2- 172.21.15.60:0/2169257415 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb63c1055d0 0x7fb63c105a50 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:20.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:20.142+0000 7fb640a00700 1 -- 172.21.15.60:0/2169257415 >> 172.21.15.60:0/2169257415 conn(0x7fb63c1010c0 msgr2=0x7fb63c1034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:43:20.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:20.143+0000 7fb640a00700 1 -- 172.21.15.60:0/2169257415 shutdown_connections 2024-05-03T14:43:20.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:20.143+0000 7fb640a00700 1 -- 172.21.15.60:0/2169257415 wait complete. 2024-05-03T14:43:20.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:20.144+0000 7fb640a00700 1 Processor -- start 2024-05-03T14:43:20.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:20.144+0000 7fb640a00700 1 -- start start 2024-05-03T14:43:20.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:20.144+0000 7fb640a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb63c1055d0 0x7fb63c19bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:43:20.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:20.144+0000 7fb640a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb63c105f90 0x7fb63c19c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:43:20.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:20.144+0000 7fb640a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb63c10b660 0x7fb63c1a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:43:20.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:20.144+0000 7fb640a00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fb63c115a30 con 0x7fb63c1055d0 2024-05-03T14:43:20.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:20.144+0000 7fb640a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb63c1158b0 con 0x7fb63c105f90 2024-05-03T14:43:20.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:20.144+0000 7fb640a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb63c115bb0 con 0x7fb63c10b660 2024-05-03T14:43:20.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:20.145+0000 7fb63a800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb63c1055d0 0x7fb63c19bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:43:20.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:20.145+0000 7fb63a800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb63c1055d0 0x7fb63c19bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36844/0 (socket says 172.21.15.60:36844) 2024-05-03T14:43:20.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:20.145+0000 7fb63a800700 1 -- 172.21.15.60:0/1933763143 learned_addr learned my addr 172.21.15.60:0/1933763143 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:43:20.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:20.145+0000 7fb639e00700 1 --2- 172.21.15.60:0/1933763143 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb63c105f90 0x7fb63c19c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:43:20.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:20.145+0000 7fb63b200700 1 --2- 172.21.15.60:0/1933763143 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb63c10b660 0x7fb63c1a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:43:20.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:20.145+0000 7fb63a800700 1 -- 172.21.15.60:0/1933763143 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb63c10b660 msgr2=0x7fb63c1a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:43:20.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:20.145+0000 7fb63a800700 1 --2- 172.21.15.60:0/1933763143 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb63c10b660 0x7fb63c1a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:20.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:20.145+0000 7fb63a800700 1 -- 172.21.15.60:0/1933763143 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb63c105f90 msgr2=0x7fb63c19c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:43:20.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:20.145+0000 7fb63a800700 1 --2- 172.21.15.60:0/1933763143 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb63c105f90 0x7fb63c19c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:20.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:20.145+0000 7fb63a800700 1 -- 172.21.15.60:0/1933763143 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb63c1a0da0 con 0x7fb63c1055d0 2024-05-03T14:43:20.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:20.146+0000 7fb63a800700 1 --2- 172.21.15.60:0/1933763143 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb63c1055d0 0x7fb63c19bde0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb62c00ea50 tx=0x7fb62c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:43:20.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:20.146+0000 7fb633400700 1 -- 172.21.15.60:0/1933763143 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62c00cbd0 con 0x7fb63c1055d0 2024-05-03T14:43:20.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:20.146+0000 7fb633400700 1 -- 172.21.15.60:0/1933763143 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7fb62c00cd70 con 0x7fb63c1055d0 2024-05-03T14:43:20.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:20.146+0000 7fb640a00700 1 -- 172.21.15.60:0/1933763143 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb63c1a1090 con 0x7fb63c1055d0 2024-05-03T14:43:20.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:20.146+0000 7fb633400700 1 -- 172.21.15.60:0/1933763143 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62c018820 con 0x7fb63c1055d0 2024-05-03T14:43:20.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:20.146+0000 7fb640a00700 1 -- 172.21.15.60:0/1933763143 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb63c1a8910 con 0x7fb63c1055d0 2024-05-03T14:43:20.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:20.147+0000 7fb640a00700 1 -- 172.21.15.60:0/1933763143 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb63c06a190 con 0x7fb63c1055d0 2024-05-03T14:43:20.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:20.150+0000 7fb633400700 1 -- 172.21.15.60:0/1933763143 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fb62c01f020 con 0x7fb63c1055d0 2024-05-03T14:43:20.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:20.151+0000 7fb633400700 1 --2- 172.21.15.60:0/1933763143 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fb6200b2900 0x7fb6200b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:43:20.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:20.152+0000 7fb633400700 1 -- 172.21.15.60:0/1933763143 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fb62c0e2600 con 0x7fb63c1055d0 2024-05-03T14:43:20.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:20.152+0000 7fb639e00700 1 --2- 172.21.15.60:0/1933763143 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fb6200b2900 0x7fb6200b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:43:20.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:20.152+0000 7fb639e00700 1 --2- 172.21.15.60:0/1933763143 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fb6200b2900 0x7fb6200b4dc0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb6280098e0 tx=0x7fb628005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:43:20.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:20.153+0000 7fb633400700 1 -- 172.21.15.60:0/1933763143 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fb62c026070 con 0x7fb63c1055d0 2024-05-03T14:43:20.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:20.399+0000 7fb640a00700 1 -- 172.21.15.60:0/1933763143 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb63c1a8b40 con 0x7fb63c1055d0 2024-05-03T14:43:20.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:20.402+0000 7fb633400700 1 -- 172.21.15.60:0/1933763143 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+18428 (secure 0 0 0) 0x7fb62c0aa720 con 0x7fb63c1055d0 2024-05-03T14:43:20.405 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:43:20.405 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":71,"fsid":"285d64b2-095a-11ef-bc97-c7b262605968","created":"2024-05-03T14:34:53.560460+0000","modified":"2024-05-03T14:43:15.891979+0000","last_up_change":"2024-05-03T14:43:14.877043+0000","last_in_change":"2024-05-03T14:42:55.085959+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-03T14:38:54.333714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f0797730-c4c1-47ea-9dce-9270d5ebb4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2952930162},{"type":"v1","addr":"172.21.15.60:6803","nonce":2952930162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2952930162},{"type":"v1","addr":"172.21.15.60:6805","nonce":2952930162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":2952930162},{"type":"v1","addr":"172.21.15.60:6809","nonce":2952930162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2952930162},{"type":"v1","addr":"172.21.15.60:6807","nonce":2952930162}]},"public_addr":"172.21.15.60:6803/2952930162","cluster_addr":"172.21.15.60:6805/2952930162","heartbeat_back_addr":"172.21.15.60:6809/2952930162","heartbeat_front_addr":"172.21.15.60:6807/2952930162","state":["exists","up"]},{"osd":1,"uuid":"96d0ad08-cda4-4824-bad4-e64973508c0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":720063242},{"type":"v1","addr":"172.21.15.90:6801","nonce":720063242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":720063242},{"type":"v1","addr":"172.21.15.90:6803","nonce":720063242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":720063242},{"type":"v1","addr":"172.21.15.90:6807","nonce":720063242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":720063242},{"type":"v1","addr":"172.21.15.90:6805","nonce":720063242}]},"public_addr":"172.21.15.90:6801/720063242","cluster_addr":"172.21.15.90:6803/720063242","heartbeat_back_addr":"172.21.15.90:6807/720063242","heartbeat_front_addr":"172.21.15.90:6805/720063242","state":["exists","up"]},{"osd":2,"uuid":"ce95fb7c-1611-4dd1-aee7-97c420d6a466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1359662077},{"type":"v1","addr":"172.21.15.149:6801","nonce":1359662077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1359662077},{"type":"v1","addr":"172.21.15.149:6803","nonce":1359662077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1359662077},{"type":"v1","addr":"172.21.15.149:6807","nonce":1359662077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1359662077},{"type":"v1","addr":"172.21.15.149:6805","nonce":1359662077}]},"public_addr":"172.21.15.149:6801/1359662077","cluster_addr":"172.21.15.149:6803/1359662077","heartbeat_back_addr":"172.21.15.149:6807/1359662077","heartbeat_front_addr":"172.21.15.149:6805/1359662077","state":["exists","up"]},{"osd":3,"uuid":"ef31136a-9b36-4b87-8389-f53d311bc5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3226133334},{"type":"v1","addr":"172.21.15.60:6811","nonce":3226133334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3226133334},{"type":"v1","addr":"172.21.15.60:6813","nonce":3226133334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3226133334},{"type":"v1","addr":"172.21.15.60:6817","nonce":3226133334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3226133334},{"type":"v1","addr":"172.21.15.60:6815","nonce":3226133334}]},"public_addr":"172.21.15.60:6811/3226133334","cluster_addr":"172.21.15.60:6813/3226133334","heartbeat_back_addr":"172.21.15.60:6817/3226133334","heartbeat_front_addr":"172.21.15.60:6815/3226133334","state":["exists","up"]},{"osd":4,"uuid":"73db7408-a2ff-47e9-b13d-d31688a93610","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2201391883},{"type":"v1","addr":"172.21.15.90:6809","nonce":2201391883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":2201391883},{"type":"v1","addr":"172.21.15.90:6811","nonce":2201391883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":2201391883},{"type":"v1","addr":"172.21.15.90:6815","nonce":2201391883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":2201391883},{"type":"v1","addr":"172.21.15.90:6813","nonce":2201391883}]},"public_addr":"172.21.15.90:6809/2201391883","cluster_addr":"172.21.15.90:6811/2201391883","heartbeat_back_addr":"172.21.15.90:6815/2201391883","heartbeat_front_addr":"172.21.15.90:6813/2201391883","state":["exists","up"]},{"osd":5,"uuid":"aefb78a2-522a-4b35-92b8-e5c04fa47b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2139732836},{"type":"v1","addr":"172.21.15.149:6809","nonce":2139732836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":2139732836},{"type":"v1","addr":"172.21.15.149:6811","nonce":2139732836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":2139732836},{"type":"v1","addr":"172.21.15.149:6815","nonce":2139732836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":2139732836},{"type":"v1","addr":"172.21.15.149:6813","nonce":2139732836}]},"public_addr":"172.21.15.149:6809/2139732836","cluster_addr":"172.21.15.149:6811/2139732836","heartbeat_back_addr":"172.21.15.149:6815/2139732836","heartbeat_front_addr":"172.21.15.149:6813/2139732836","state":["exists","up"]},{"osd":6,"uuid":"7c887819-6182-4636-8cfd-d845c30bd13c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3738521002},{"type":"v1","addr":"172.21.15.60:6819","nonce":3738521002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3738521002},{"type":"v1","addr":"172.21.15.60:6821","nonce":3738521002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3738521002},{"type":"v1","addr":"172.21.15.60:6825","nonce":3738521002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3738521002},{"type":"v1","addr":"172.21.15.60:6823","nonce":3738521002}]},"public_addr":"172.21.15.60:6819/3738521002","cluster_addr":"172.21.15.60:6821/3738521002","heartbeat_back_addr":"172.21.15.60:6825/3738521002","heartbeat_front_addr":"172.21.15.60:6823/3738521002","state":["exists","up"]},{"osd":7,"uuid":"ebce6acf-8873-4b35-bd5f-80de2163a307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":1686298661},{"type":"v1","addr":"172.21.15.90:6817","nonce":1686298661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":1686298661},{"type":"v1","addr":"172.21.15.90:6819","nonce":1686298661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":1686298661},{"type":"v1","addr":"172.21.15.90:6823","nonce":1686298661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1686298661},{"type":"v1","addr":"172.21.15.90:6821","nonce":1686298661}]},"public_addr":"172.21.15.90:6817/1686298661","cluster_addr":"172.21.15.90:6819/1686298661","heartbeat_back_addr":"172.21.15.90:6823/1686298661","heartbeat_front_addr":"172.21.15.90:6821/1686298661","state":["exists","up"]},{"osd":8,"uuid":"4ae04b98-e9b2-4956-83d9-17e5ac3c963f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":2608348248},{"type":"v1","addr":"172.21.15.149:6817","nonce":2608348248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2608348248},{"type":"v1","addr":"172.21.15.149:6819","nonce":2608348248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2608348248},{"type":"v1","addr":"172.21.15.149:6823","nonce":2608348248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2608348248},{"type":"v1","addr":"172.21.15.149:6821","nonce":2608348248}]},"public_addr":"172.21.15.149:6817/2608348248","cluster_addr":"172.21.15.149:6819/2608348248","heartbeat_back_addr":"172.21.15.149:6823/2608348248","heartbeat_front_addr":"172.21.15.149:6821/2608348248","state":["exists","up"]},{"osd":9,"uuid":"d0c0a70d-aa76-47e4-91e8-c343c8c41197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":2156130999},{"type":"v1","addr":"172.21.15.60:6827","nonce":2156130999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":2156130999},{"type":"v1","addr":"172.21.15.60:6829","nonce":2156130999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":2156130999},{"type":"v1","addr":"172.21.15.60:6833","nonce":2156130999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":2156130999},{"type":"v1","addr":"172.21.15.60:6831","nonce":2156130999}]},"public_addr":"172.21.15.60:6827/2156130999","cluster_addr":"172.21.15.60:6829/2156130999","heartbeat_back_addr":"172.21.15.60:6833/2156130999","heartbeat_front_addr":"172.21.15.60:6831/2156130999","state":["exists","up"]},{"osd":10,"uuid":"cdc3d7b8-33c7-4cc4-be4e-6887d15a3d8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":2492593590},{"type":"v1","addr":"172.21.15.90:6825","nonce":2492593590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":2492593590},{"type":"v1","addr":"172.21.15.90:6827","nonce":2492593590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":2492593590},{"type":"v1","addr":"172.21.15.90:6831","nonce":2492593590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":2492593590},{"type":"v1","addr":"172.21.15.90:6829","nonce":2492593590}]},"public_addr":"172.21.15.90:6825/2492593590","cluster_addr":"172.21.15.90:6827/2492593590","heartbeat_back_addr":"172.21.15.90:6831/2492593590","heartbeat_front_addr":"172.21.15.90:6829/2492593590","state":["exists","up"]},{"osd":11,"uuid":"8d17ec1a-bdd5-41f1-9b59-c2b6a527bfbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":1063245126},{"type":"v1","addr":"172.21.15.149:6825","nonce":1063245126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":1063245126},{"type":"v1","addr":"172.21.15.149:6827","nonce":1063245126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":1063245126},{"type":"v1","addr":"172.21.15.149:6831","nonce":1063245126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":1063245126},{"type":"v1","addr":"172.21.15.149:6829","nonce":1063245126}]},"public_addr":"172.21.15.149:6825/1063245126","cluster_addr":"172.21.15.149:6827/1063245126","heartbeat_back_addr":"172.21.15.149:6831/1063245126","heartbeat_front_addr":"172.21.15.149:6829/1063245126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:38:02.056563+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:38:27.140677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:38:51.287070+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:39:25.484410+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:39:51.483133+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:40:17.200423+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:40:51.979804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:41:18.742542+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:41:44.922541+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:42:19.778451+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:42:47.438440+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:43:13.429419+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6801/424391417":"2024-05-04T14:36:04.154333+0000","172.21.15.60:6800/424391417":"2024-05-04T14:36:04.154333+0000","172.21.15.60:0/1426532561":"2024-05-04T14:36:04.154333+0000","172.21.15.60:0/3230079926":"2024-05-04T14:35:28.066961+0000","172.21.15.60:0/2334895610":"2024-05-04T14:35:28.066961+0000","172.21.15.60:6800/888447185":"2024-05-04T14:35:28.066961+0000","172.21.15.60:0/3996899937":"2024-05-04T14:35:28.066961+0000","172.21.15.60:0/2779644018":"2024-05-04T14:36:04.154333+0000","172.21.15.60:0/71440838":"2024-05-04T14:36:04.154333+0000","172.21.15.60:6801/888447185":"2024-05-04T14:35:28.066961+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-03T14:43:20.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:20.406+0000 7fb640a00700 1 -- 172.21.15.60:0/1933763143 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fb6200b2900 msgr2=0x7fb6200b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:43:20.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:20.406+0000 7fb640a00700 1 --2- 172.21.15.60:0/1933763143 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fb6200b2900 0x7fb6200b4dc0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb6280098e0 tx=0x7fb628005960 comp rx=0 tx=0).stop 2024-05-03T14:43:20.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:20.406+0000 7fb640a00700 1 -- 172.21.15.60:0/1933763143 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb63c1055d0 msgr2=0x7fb63c19bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:43:20.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:20.406+0000 7fb640a00700 1 --2- 172.21.15.60:0/1933763143 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb63c1055d0 0x7fb63c19bde0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb62c00ea50 tx=0x7fb62c00ed60 comp rx=0 tx=0).stop 2024-05-03T14:43:20.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:20.406+0000 7fb640a00700 1 -- 172.21.15.60:0/1933763143 shutdown_connections 2024-05-03T14:43:20.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:20.406+0000 7fb640a00700 1 --2- 172.21.15.60:0/1933763143 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fb6200b2900 0x7fb6200b4dc0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:20.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:20.406+0000 7fb640a00700 1 --2- 172.21.15.60:0/1933763143 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb63c10b660 0x7fb63c1a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:20.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:20.406+0000 7fb640a00700 1 --2- 172.21.15.60:0/1933763143 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb63c1055d0 0x7fb63c19bde0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:20.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:20.406+0000 7fb640a00700 1 --2- 172.21.15.60:0/1933763143 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb63c105f90 0x7fb63c19c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:20.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:20.406+0000 7fb640a00700 1 -- 172.21.15.60:0/1933763143 >> 172.21.15.60:0/1933763143 conn(0x7fb63c1010c0 msgr2=0x7fb63c1025a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:43:20.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:20.406+0000 7fb640a00700 1 -- 172.21.15.60:0/1933763143 shutdown_connections 2024-05-03T14:43:20.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:20.406+0000 7fb640a00700 1 -- 172.21.15.60:0/1933763143 wait complete. 2024-05-03T14:43:21.282 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-03T14:38:54.333714+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-03T14:43:21.282 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-03T14:43:21.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:21 smithi060 ceph-mon[28873]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:21.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:21 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/1933763143' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T14:43:21.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:21 smithi090 ceph-mon[33568]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:21.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:21 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/1933763143' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T14:43:21.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:21 smithi149 ceph-mon[33332]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:21.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:21 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/1933763143' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T14:43:21.746 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:43:23.532 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:23 smithi060 ceph-mon[28873]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:23.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:23 smithi090 ceph-mon[33568]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:23.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:23 smithi149 ceph-mon[33332]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:24.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:24.049+0000 7f5305400700 1 -- 172.21.15.60:0/2235264427 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f530006bb40 msgr2=0x7f5300110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:43:24.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:24.049+0000 7f5305400700 1 --2- 172.21.15.60:0/2235264427 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f530006bb40 0x7f5300110330 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f52f800b0a0 tx=0x7f52f800b3b0 comp rx=0 tx=0).stop 2024-05-03T14:43:24.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:24.049+0000 7f5305400700 1 -- 172.21.15.60:0/2235264427 shutdown_connections 2024-05-03T14:43:24.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:24.049+0000 7f5305400700 1 --2- 172.21.15.60:0/2235264427 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f530006bb40 0x7f5300110330 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:24.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:24.049+0000 7f5305400700 1 --2- 172.21.15.60:0/2235264427 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5300109510 0x7f53001098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:24.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:24.049+0000 7f5305400700 1 --2- 172.21.15.60:0/2235264427 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f530006b180 0x7f530006b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:24.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:24.049+0000 7f5305400700 1 -- 172.21.15.60:0/2235264427 >> 172.21.15.60:0/2235264427 conn(0x7f5300079350 msgr2=0x7f5300079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:43:24.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:24.049+0000 7f5305400700 1 -- 172.21.15.60:0/2235264427 shutdown_connections 2024-05-03T14:43:24.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:24.049+0000 7f5305400700 1 -- 172.21.15.60:0/2235264427 wait complete. 2024-05-03T14:43:24.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:24.051+0000 7f5305400700 1 Processor -- start 2024-05-03T14:43:24.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:24.051+0000 7f5305400700 1 -- start start 2024-05-03T14:43:24.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:24.051+0000 7f5305400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f530006b180 0x7f530019f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:43:24.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:24.051+0000 7f5305400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f530006bb40 0x7f530019fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:43:24.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:24.051+0000 7f5305400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5300109510 0x7f5300199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:43:24.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:24.051+0000 7f5305400700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f53001138b0 con 0x7f530006bb40 2024-05-03T14:43:24.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:24.051+0000 7f5305400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5300113730 con 0x7f530006b180 2024-05-03T14:43:24.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:24.051+0000 7f5305400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5300113a30 con 0x7f5300109510 2024-05-03T14:43:24.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:24.052+0000 7f52fde00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f530006bb40 0x7f530019fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:43:24.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:24.052+0000 7f52fde00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f530006bb40 0x7f530019fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45876/0 (socket says 172.21.15.60:45876) 2024-05-03T14:43:24.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:24.052+0000 7f52fde00700 1 -- 172.21.15.60:0/129243477 learned_addr learned my addr 172.21.15.60:0/129243477 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:43:24.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:24.052+0000 7f52ff200700 1 --2- 172.21.15.60:0/129243477 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5300109510 0x7f5300199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:43:24.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:24.052+0000 7f52fe800700 1 --2- 172.21.15.60:0/129243477 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f530006b180 0x7f530019f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:43:24.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:24.052+0000 7f52ff200700 1 -- 172.21.15.60:0/129243477 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f530006b180 msgr2=0x7f530019f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:43:24.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:24.052+0000 7f52ff200700 1 --2- 172.21.15.60:0/129243477 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f530006b180 0x7f530019f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:24.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:24.052+0000 7f52ff200700 1 -- 172.21.15.60:0/129243477 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f530006bb40 msgr2=0x7f530019fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:43:24.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:24.052+0000 7f52ff200700 1 --2- 172.21.15.60:0/129243477 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f530006bb40 0x7f530019fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:24.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:24.052+0000 7f52ff200700 1 -- 172.21.15.60:0/129243477 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f530019a230 con 0x7f5300109510 2024-05-03T14:43:24.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:24.052+0000 7f52fde00700 1 --2- 172.21.15.60:0/129243477 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f530006bb40 0x7f530019fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:43:24.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:24.053+0000 7f52ff200700 1 --2- 172.21.15.60:0/129243477 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5300109510 0x7f5300199ad0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f52f800b0a0 tx=0x7f52f8004d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:43:24.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:24.053+0000 7f52f7400700 1 -- 172.21.15.60:0/129243477 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f8021070 con 0x7f5300109510 2024-05-03T14:43:24.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:24.053+0000 7f52f7400700 1 -- 172.21.15.60:0/129243477 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f52f8007b80 con 0x7f5300109510 2024-05-03T14:43:24.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:24.053+0000 7f5305400700 1 -- 172.21.15.60:0/129243477 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f530019a4c0 con 0x7f5300109510 2024-05-03T14:43:24.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:24.053+0000 7f52f7400700 1 -- 172.21.15.60:0/129243477 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f8005040 con 0x7f5300109510 2024-05-03T14:43:24.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:24.053+0000 7f5305400700 1 -- 172.21.15.60:0/129243477 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53001a67e0 con 0x7f5300109510 2024-05-03T14:43:24.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:24.055+0000 7f5305400700 1 -- 172.21.15.60:0/129243477 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f530006a190 con 0x7f5300109510 2024-05-03T14:43:24.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:24.057+0000 7f52f7400700 1 -- 172.21.15.60:0/129243477 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f52f8024070 con 0x7f5300109510 2024-05-03T14:43:24.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:24.058+0000 7f52f7400700 1 --2- 172.21.15.60:0/129243477 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f52e40b2900 0x7f52e40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:43:24.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:24.059+0000 7f52f7400700 1 -- 172.21.15.60:0/129243477 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f52f80e30d0 con 0x7f5300109510 2024-05-03T14:43:24.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:24.059+0000 7f52fe800700 1 --2- 172.21.15.60:0/129243477 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f52e40b2900 0x7f52e40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:43:24.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:24.060+0000 7f52fe800700 1 --2- 172.21.15.60:0/129243477 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f52e40b2900 0x7f52e40b4dc0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f52f0009be0 tx=0x7f52f0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:43:24.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:24.061+0000 7f52f7400700 1 -- 172.21.15.60:0/129243477 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f52f80ab280 con 0x7f5300109510 2024-05-03T14:43:24.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:24.298+0000 7f5305400700 1 -- 172.21.15.60:0/129243477 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f530002e510 con 0x7f5300109510 2024-05-03T14:43:24.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:24.299+0000 7f52f7400700 1 -- 172.21.15.60:0/129243477 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) ==== 93+0+10 (secure 0 0 0) 0x7f52f80aab80 con 0x7f5300109510 2024-05-03T14:43:24.303 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-05-03T14:43:24.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:24.302+0000 7f5305400700 1 -- 172.21.15.60:0/129243477 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f52e40b2900 msgr2=0x7f52e40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:43:24.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:24.302+0000 7f5305400700 1 --2- 172.21.15.60:0/129243477 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f52e40b2900 0x7f52e40b4dc0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f52f0009be0 tx=0x7f52f0009260 comp rx=0 tx=0).stop 2024-05-03T14:43:24.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:24.302+0000 7f5305400700 1 -- 172.21.15.60:0/129243477 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5300109510 msgr2=0x7f5300199ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:43:24.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:24.302+0000 7f5305400700 1 --2- 172.21.15.60:0/129243477 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5300109510 0x7f5300199ad0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f52f800b0a0 tx=0x7f52f8004d00 comp rx=0 tx=0).stop 2024-05-03T14:43:24.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:24.302+0000 7f5305400700 1 -- 172.21.15.60:0/129243477 shutdown_connections 2024-05-03T14:43:24.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:24.302+0000 7f5305400700 1 --2- 172.21.15.60:0/129243477 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f52e40b2900 0x7f52e40b4dc0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:24.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:24.302+0000 7f5305400700 1 --2- 172.21.15.60:0/129243477 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5300109510 0x7f5300199ad0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:24.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:24.302+0000 7f5305400700 1 --2- 172.21.15.60:0/129243477 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f530006b180 0x7f530019f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:24.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:24.302+0000 7f5305400700 1 --2- 172.21.15.60:0/129243477 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f530006bb40 0x7f530019fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:24.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:24.302+0000 7f5305400700 1 -- 172.21.15.60:0/129243477 >> 172.21.15.60:0/129243477 conn(0x7f5300079350 msgr2=0x7f5300100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:43:24.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:24.302+0000 7f5305400700 1 -- 172.21.15.60:0/129243477 shutdown_connections 2024-05-03T14:43:24.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:24.303+0000 7f5305400700 1 -- 172.21.15.60:0/129243477 wait complete. 2024-05-03T14:43:24.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:24 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/129243477' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-03T14:43:24.624 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:24 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/129243477' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-03T14:43:24.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:24 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/129243477' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-03T14:43:25.112 INFO:tasks.cephadm:Adding mds.a on smithi060 2024-05-03T14:43:25.112 INFO:tasks.cephadm:Adding mds.d on smithi060 2024-05-03T14:43:25.112 INFO:tasks.cephadm:Adding mds.g on smithi060 2024-05-03T14:43:25.112 INFO:tasks.cephadm:Adding mds.j on smithi060 2024-05-03T14:43:25.112 INFO:tasks.cephadm:Adding mds.b on smithi090 2024-05-03T14:43:25.112 INFO:tasks.cephadm:Adding mds.e on smithi090 2024-05-03T14:43:25.112 INFO:tasks.cephadm:Adding mds.h on smithi090 2024-05-03T14:43:25.112 INFO:tasks.cephadm:Adding mds.k on smithi090 2024-05-03T14:43:25.112 INFO:tasks.cephadm:Adding mds.c on smithi149 2024-05-03T14:43:25.112 INFO:tasks.cephadm:Adding mds.f on smithi149 2024-05-03T14:43:25.112 INFO:tasks.cephadm:Adding mds.i on smithi149 2024-05-03T14:43:25.112 INFO:tasks.cephadm:Adding mds.l on smithi149 2024-05-03T14:43:25.113 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph orch apply mds all '12;smithi060=a;smithi060=d;smithi060=g;smithi060=j;smithi090=b;smithi090=e;smithi090=h;smithi090=k;smithi149=c;smithi149=f;smithi149=i;smithi149=l' 2024-05-03T14:43:25.366 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T14:43:25.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:25 smithi149 ceph-mon[33332]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:25.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:25 smithi090 ceph-mon[33568]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:25.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:25 smithi060 ceph-mon[28873]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:26.979 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:26.975+0000 7f3c30811700 1 -- 172.21.15.149:0/2839829773 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c2c103620 msgr2=0x7f3c2c103a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:43:26.979 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:26.975+0000 7f3c30811700 1 --2- 172.21.15.149:0/2839829773 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c2c103620 0x7f3c2c103a80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3c14009990 tx=0x7f3c14009ca0 comp rx=0 tx=0).stop 2024-05-03T14:43:26.979 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:26.975+0000 7f3c30811700 1 -- 172.21.15.149:0/2839829773 shutdown_connections 2024-05-03T14:43:26.979 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:26.975+0000 7f3c30811700 1 --2- 172.21.15.149:0/2839829773 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3c2c103fc0 0x7f3c2c10a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:26.980 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:26.975+0000 7f3c30811700 1 --2- 172.21.15.149:0/2839829773 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c2c102420 0x7f3c2c102840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:26.980 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:26.975+0000 7f3c30811700 1 --2- 172.21.15.149:0/2839829773 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c2c103620 0x7f3c2c103a80 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:26.980 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:26.975+0000 7f3c30811700 1 -- 172.21.15.149:0/2839829773 >> 172.21.15.149:0/2839829773 conn(0x7f3c2c0fda00 msgr2=0x7f3c2c0ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:43:26.980 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:26.975+0000 7f3c30811700 1 -- 172.21.15.149:0/2839829773 shutdown_connections 2024-05-03T14:43:26.980 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:26.975+0000 7f3c30811700 1 -- 172.21.15.149:0/2839829773 wait complete. 2024-05-03T14:43:26.980 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:26.976+0000 7f3c30811700 1 Processor -- start 2024-05-03T14:43:26.980 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:26.976+0000 7f3c30811700 1 -- start start 2024-05-03T14:43:26.980 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:26.976+0000 7f3c30811700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3c2c102420 0x7f3c2c199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:43:26.980 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:26.976+0000 7f3c30811700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c2c103620 0x7f3c2c199dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:43:26.980 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:26.976+0000 7f3c30811700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c2c103fc0 0x7f3c2c1a0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:43:26.980 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:26.976+0000 7f3c30811700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f3c2c10e240 con 0x7f3c2c103fc0 2024-05-03T14:43:26.980 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:26.976+0000 7f3c30811700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3c2c10e0c0 con 0x7f3c2c102420 2024-05-03T14:43:26.980 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:26.976+0000 7f3c30811700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3c2c10e3c0 con 0x7f3c2c103620 2024-05-03T14:43:26.980 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:26.976+0000 7f3c23fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c2c103620 0x7f3c2c199dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:43:26.981 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:26.976+0000 7f3c23fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c2c103620 0x7f3c2c199dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:55088/0 (socket says 172.21.15.149:55088) 2024-05-03T14:43:26.981 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:26.976+0000 7f3c23fff700 1 -- 172.21.15.149:0/2304424203 learned_addr learned my addr 172.21.15.149:0/2304424203 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-05-03T14:43:26.981 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:26.976+0000 7f3c2a59c700 1 --2- 172.21.15.149:0/2304424203 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3c2c102420 0x7f3c2c199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:43:26.981 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:26.976+0000 7f3c23fff700 1 -- 172.21.15.149:0/2304424203 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3c2c102420 msgr2=0x7f3c2c199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:43:26.981 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:26.976+0000 7f3c23fff700 1 --2- 172.21.15.149:0/2304424203 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3c2c102420 0x7f3c2c199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:26.981 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:26.976+0000 7f3c23fff700 1 -- 172.21.15.149:0/2304424203 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c2c103fc0 msgr2=0x7f3c2c1a0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:43:26.981 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:26.976+0000 7f3c23fff700 1 --2- 172.21.15.149:0/2304424203 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c2c103fc0 0x7f3c2c1a0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:26.981 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:26.976+0000 7f3c23fff700 1 -- 172.21.15.149:0/2304424203 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c2c104930 con 0x7f3c2c103620 2024-05-03T14:43:26.981 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:26.976+0000 7f3c2a59c700 1 --2- 172.21.15.149:0/2304424203 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3c2c102420 0x7f3c2c199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T14:43:26.981 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:26.977+0000 7f3c23fff700 1 --2- 172.21.15.149:0/2304424203 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c2c103620 0x7f3c2c199dd0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3c1400f670 tx=0x7f3c1400f750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:43:26.981 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:26.977+0000 7f3c237fe700 1 -- 172.21.15.149:0/2304424203 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c14027070 con 0x7f3c2c103620 2024-05-03T14:43:26.981 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:26.977+0000 7f3c237fe700 1 -- 172.21.15.149:0/2304424203 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f3c1400fcf0 con 0x7f3c2c103620 2024-05-03T14:43:26.981 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:26.977+0000 7f3c30811700 1 -- 172.21.15.149:0/2304424203 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c2c104bc0 con 0x7f3c2c103620 2024-05-03T14:43:26.981 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:26.977+0000 7f3c237fe700 1 -- 172.21.15.149:0/2304424203 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c14017810 con 0x7f3c2c103620 2024-05-03T14:43:26.982 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:26.977+0000 7f3c30811700 1 -- 172.21.15.149:0/2304424203 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c2c105170 con 0x7f3c2c103620 2024-05-03T14:43:26.982 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:26.978+0000 7f3c30811700 1 -- 172.21.15.149:0/2304424203 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c2c06a190 con 0x7f3c2c103620 2024-05-03T14:43:26.983 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:26.979+0000 7f3c237fe700 1 -- 172.21.15.149:0/2304424203 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f3c1402a070 con 0x7f3c2c103620 2024-05-03T14:43:26.983 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:26.980+0000 7f3c237fe700 1 --2- 172.21.15.149:0/2304424203 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f3c180b2900 0x7f3c180b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:43:26.983 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:26.980+0000 7f3c237fe700 1 -- 172.21.15.149:0/2304424203 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f3c140e2ec0 con 0x7f3c2c103620 2024-05-03T14:43:26.983 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:26.981+0000 7f3c2a59c700 1 --2- 172.21.15.149:0/2304424203 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f3c180b2900 0x7f3c180b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:43:26.983 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:26.981+0000 7f3c2a59c700 1 --2- 172.21.15.149:0/2304424203 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f3c180b2900 0x7f3c180b4dc0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c103480 tx=0x7f3c1c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:43:26.985 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:26.984+0000 7f3c237fe700 1 -- 172.21.15.149:0/2304424203 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f3c14013070 con 0x7f3c2c103620 2024-05-03T14:43:27.198 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:27.196+0000 7f3c30811700 1 -- 172.21.15.149:0/2304424203 --> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi060=a;smithi060=d;smithi060=g;smithi060=j;smithi090=b;smithi090=e;smithi090=h;smithi090=k;smithi149=c;smithi149=f;smithi149=i;smithi149=l", "target": ["mon-mgr", ""]}) -- 0x7f3c2c06c420 con 0x7f3c180b2900 2024-05-03T14:43:27.216 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:27.214+0000 7f3c237fe700 1 -- 172.21.15.149:0/2304424203 <== mgr.14152 v2:172.21.15.60:6800/3066142956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f3c2c06c420 con 0x7f3c180b2900 2024-05-03T14:43:27.216 INFO:teuthology.orchestra.run.smithi149.stdout:Scheduled mds.all update... 2024-05-03T14:43:27.219 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:27.217+0000 7f3c30811700 1 -- 172.21.15.149:0/2304424203 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f3c180b2900 msgr2=0x7f3c180b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:43:27.219 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:27.217+0000 7f3c30811700 1 --2- 172.21.15.149:0/2304424203 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f3c180b2900 0x7f3c180b4dc0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c103480 tx=0x7f3c1c008040 comp rx=0 tx=0).stop 2024-05-03T14:43:27.219 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:27.217+0000 7f3c30811700 1 -- 172.21.15.149:0/2304424203 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c2c103620 msgr2=0x7f3c2c199dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:43:27.219 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:27.217+0000 7f3c30811700 1 --2- 172.21.15.149:0/2304424203 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c2c103620 0x7f3c2c199dd0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3c1400f670 tx=0x7f3c1400f750 comp rx=0 tx=0).stop 2024-05-03T14:43:27.219 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:27.217+0000 7f3c30811700 1 -- 172.21.15.149:0/2304424203 shutdown_connections 2024-05-03T14:43:27.219 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:27.217+0000 7f3c30811700 1 --2- 172.21.15.149:0/2304424203 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f3c180b2900 0x7f3c180b4dc0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:27.219 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:27.217+0000 7f3c30811700 1 --2- 172.21.15.149:0/2304424203 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c2c103fc0 0x7f3c2c1a0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:27.219 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:27.217+0000 7f3c30811700 1 --2- 172.21.15.149:0/2304424203 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3c2c102420 0x7f3c2c199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:27.219 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:27.217+0000 7f3c30811700 1 --2- 172.21.15.149:0/2304424203 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c2c103620 0x7f3c2c199dd0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:27.219 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:27.217+0000 7f3c30811700 1 -- 172.21.15.149:0/2304424203 >> 172.21.15.149:0/2304424203 conn(0x7f3c2c0fda00 msgr2=0x7f3c2c06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:43:27.220 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:27.217+0000 7f3c30811700 1 -- 172.21.15.149:0/2304424203 shutdown_connections 2024-05-03T14:43:27.220 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-03T14:43:27.217+0000 7f3c30811700 1 -- 172.21.15.149:0/2304424203 wait complete. 2024-05-03T14:43:27.624 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:27 smithi090 ceph-mon[33568]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:27.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:27 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:27.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:27 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:43:27.670 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:27 smithi149 ceph-mon[33332]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:27.670 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:27 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:27.670 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:27 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:43:27.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:27 smithi060 ceph-mon[28873]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:27.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:27 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:27.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:27 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:43:28.099 DEBUG:teuthology.orchestra.run.smithi060:mds.a> sudo journalctl -f -n 0 -u ceph-285d64b2-095a-11ef-bc97-c7b262605968@mds.a.service 2024-05-03T14:43:28.108 DEBUG:teuthology.orchestra.run.smithi060:mds.d> sudo journalctl -f -n 0 -u ceph-285d64b2-095a-11ef-bc97-c7b262605968@mds.d.service 2024-05-03T14:43:28.155 DEBUG:teuthology.orchestra.run.smithi060:mds.g> sudo journalctl -f -n 0 -u ceph-285d64b2-095a-11ef-bc97-c7b262605968@mds.g.service 2024-05-03T14:43:28.201 DEBUG:teuthology.orchestra.run.smithi060:mds.j> sudo journalctl -f -n 0 -u ceph-285d64b2-095a-11ef-bc97-c7b262605968@mds.j.service 2024-05-03T14:43:28.248 DEBUG:teuthology.orchestra.run.smithi090:mds.b> sudo journalctl -f -n 0 -u ceph-285d64b2-095a-11ef-bc97-c7b262605968@mds.b.service 2024-05-03T14:43:28.250 DEBUG:teuthology.orchestra.run.smithi090:mds.e> sudo journalctl -f -n 0 -u ceph-285d64b2-095a-11ef-bc97-c7b262605968@mds.e.service 2024-05-03T14:43:28.293 DEBUG:teuthology.orchestra.run.smithi090:mds.h> sudo journalctl -f -n 0 -u ceph-285d64b2-095a-11ef-bc97-c7b262605968@mds.h.service 2024-05-03T14:43:28.336 DEBUG:teuthology.orchestra.run.smithi090:mds.k> sudo journalctl -f -n 0 -u ceph-285d64b2-095a-11ef-bc97-c7b262605968@mds.k.service 2024-05-03T14:43:28.379 DEBUG:teuthology.orchestra.run.smithi149:mds.c> sudo journalctl -f -n 0 -u ceph-285d64b2-095a-11ef-bc97-c7b262605968@mds.c.service 2024-05-03T14:43:28.381 DEBUG:teuthology.orchestra.run.smithi149:mds.f> sudo journalctl -f -n 0 -u ceph-285d64b2-095a-11ef-bc97-c7b262605968@mds.f.service 2024-05-03T14:43:28.424 DEBUG:teuthology.orchestra.run.smithi149:mds.i> sudo journalctl -f -n 0 -u ceph-285d64b2-095a-11ef-bc97-c7b262605968@mds.i.service 2024-05-03T14:43:28.467 DEBUG:teuthology.orchestra.run.smithi149:mds.l> sudo journalctl -f -n 0 -u ceph-285d64b2-095a-11ef-bc97-c7b262605968@mds.l.service 2024-05-03T14:43:28.509 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-05-03T14:43:28.510 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T14:43:28.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:28 smithi090 ceph-mon[33568]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi060=a;smithi060=d;smithi060=g;smithi060=j;smithi090=b;smithi090=e;smithi090=h;smithi090=k;smithi149=c;smithi149=f;smithi149=i;smithi149=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:43:28.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:28 smithi090 ceph-mon[33568]: Saving service mds.all spec with placement smithi060=a;smithi060=d;smithi060=g;smithi060=j;smithi090=b;smithi090=e;smithi090=h;smithi090=k;smithi149=c;smithi149=f;smithi149=i;smithi149=l;count:12 2024-05-03T14:43:28.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:28 smithi149 ceph-mon[33332]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi060=a;smithi060=d;smithi060=g;smithi060=j;smithi090=b;smithi090=e;smithi090=h;smithi090=k;smithi149=c;smithi149=f;smithi149=i;smithi149=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:43:28.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:28 smithi149 ceph-mon[33332]: Saving service mds.all spec with placement smithi060=a;smithi060=d;smithi060=g;smithi060=j;smithi090=b;smithi090=e;smithi090=h;smithi090=k;smithi149=c;smithi149=f;smithi149=i;smithi149=l;count:12 2024-05-03T14:43:28.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:28 smithi060 ceph-mon[28873]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi060=a;smithi060=d;smithi060=g;smithi060=j;smithi090=b;smithi090=e;smithi090=h;smithi090=k;smithi149=c;smithi149=f;smithi149=i;smithi149=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:43:28.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:28 smithi060 ceph-mon[28873]: Saving service mds.all spec with placement smithi060=a;smithi060=d;smithi060=g;smithi060=j;smithi090=b;smithi090=e;smithi090=h;smithi090=k;smithi149=c;smithi149=f;smithi149=i;smithi149=l;count:12 2024-05-03T14:43:28.976 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:43:29.412 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:29 smithi149 ceph-mon[33332]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:29.412 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:29 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:43:29.412 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:29 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:43:29.412 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:29 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:29.412 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:29 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T14:43:29.412 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:29 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T14:43:29.412 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:29 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:43:29.412 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:29 smithi149 ceph-mon[33332]: Deploying daemon mds.l on smithi149 2024-05-03T14:43:29.491 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:29 smithi060 ceph-mon[28873]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:29.492 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:29 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:43:29.492 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:29 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:43:29.492 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:29 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:29.492 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:29 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T14:43:29.492 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:29 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T14:43:29.492 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:29 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:43:29.492 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:29 smithi060 ceph-mon[28873]: Deploying daemon mds.l on smithi149 2024-05-03T14:43:29.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:29 smithi090 ceph-mon[33568]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:29.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:29 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:43:29.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:29 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:43:29.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:29 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:29.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:29 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T14:43:29.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:29 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T14:43:29.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:29 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:43:29.876 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:29 smithi090 ceph-mon[33568]: Deploying daemon mds.l on smithi149 2024-05-03T14:43:30.924 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 03 14:43:30 smithi149 systemd[1]: Starting Ceph mds.l for 285d64b2-095a-11ef-bc97-c7b262605968... 2024-05-03T14:43:31.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:31.226+0000 7f27d0600700 1 -- 172.21.15.60:0/3876895647 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f27c810fb60 msgr2=0x7f27c810ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:43:31.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:31.226+0000 7f27d0600700 1 --2- 172.21.15.60:0/3876895647 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f27c810fb60 0x7f27c810ff40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f27bc009990 tx=0x7f27bc009ca0 comp rx=0 tx=0).stop 2024-05-03T14:43:31.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:31.226+0000 7f27d0600700 1 -- 172.21.15.60:0/3876895647 shutdown_connections 2024-05-03T14:43:31.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:31.226+0000 7f27d0600700 1 --2- 172.21.15.60:0/3876895647 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27c810a490 0x7f27c81169a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:31.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:31.226+0000 7f27d0600700 1 --2- 172.21.15.60:0/3876895647 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f27c810fb60 0x7f27c810ff40 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:31.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:31.226+0000 7f27d0600700 1 --2- 172.21.15.60:0/3876895647 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27c8109ad0 0x7f27c8109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:31.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:31.226+0000 7f27d0600700 1 -- 172.21.15.60:0/3876895647 >> 172.21.15.60:0/3876895647 conn(0x7f27c8076c70 msgr2=0x7f27c81082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:43:31.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:31.226+0000 7f27d0600700 1 -- 172.21.15.60:0/3876895647 shutdown_connections 2024-05-03T14:43:31.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:31.226+0000 7f27d0600700 1 -- 172.21.15.60:0/3876895647 wait complete. 2024-05-03T14:43:31.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:31.228+0000 7f27d0600700 1 Processor -- start 2024-05-03T14:43:31.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:31.228+0000 7f27d0600700 1 -- start start 2024-05-03T14:43:31.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:31.228+0000 7f27d0600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f27c8109ad0 0x7f27c81a0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:43:31.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:31.228+0000 7f27d0600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27c810a490 0x7f27c81a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:43:31.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:31.228+0000 7f27d0600700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27c810fb60 0x7f27c81a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:43:31.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:31.228+0000 7f27d0600700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f27c8119f90 con 0x7f27c810fb60 2024-05-03T14:43:31.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:31.228+0000 7f27d0600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f27c8119e10 con 0x7f27c810a490 2024-05-03T14:43:31.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:31.228+0000 7f27d0600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f27c811a110 con 0x7f27c8109ad0 2024-05-03T14:43:31.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:31.229+0000 7f27cd200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27c810a490 0x7f27c81a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:43:31.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:31.229+0000 7f27ce600700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27c810fb60 0x7f27c81a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:43:31.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:31.229+0000 7f27cd200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27c810a490 0x7f27c81a08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.60:49222/0 (socket says 172.21.15.60:49222) 2024-05-03T14:43:31.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:31.229+0000 7f27cdc00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f27c8109ad0 0x7f27c81a0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:43:31.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:31.229+0000 7f27cd200700 1 -- 172.21.15.60:0/3104263883 learned_addr learned my addr 172.21.15.60:0/3104263883 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:43:31.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:31.229+0000 7f27cd200700 1 -- 172.21.15.60:0/3104263883 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f27c8109ad0 msgr2=0x7f27c81a0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:43:31.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:31.229+0000 7f27cd200700 1 --2- 172.21.15.60:0/3104263883 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f27c8109ad0 0x7f27c81a0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:31.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:31.229+0000 7f27cd200700 1 -- 172.21.15.60:0/3104263883 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27c810fb60 msgr2=0x7f27c81a4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:43:31.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:31.229+0000 7f27cd200700 1 --2- 172.21.15.60:0/3104263883 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27c810fb60 0x7f27c81a4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:31.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:31.229+0000 7f27cd200700 1 -- 172.21.15.60:0/3104263883 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27c81a5370 con 0x7f27c810a490 2024-05-03T14:43:31.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:31.229+0000 7f27cdc00700 1 --2- 172.21.15.60:0/3104263883 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f27c8109ad0 0x7f27c81a0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:43:31.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:31.229+0000 7f27ce600700 1 --2- 172.21.15.60:0/3104263883 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27c810fb60 0x7f27c81a4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:43:31.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:31.230+0000 7f27cd200700 1 --2- 172.21.15.60:0/3104263883 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27c810a490 0x7f27c81a08a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f27b400eb30 tx=0x7f27b400ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:43:31.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:31.230+0000 7f27c6000700 1 -- 172.21.15.60:0/3104263883 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b400cbd0 con 0x7f27c810a490 2024-05-03T14:43:31.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:31.230+0000 7f27c6000700 1 -- 172.21.15.60:0/3104263883 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f27b4004520 con 0x7f27c810a490 2024-05-03T14:43:31.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:31.230+0000 7f27d0600700 1 -- 172.21.15.60:0/3104263883 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27c81a5660 con 0x7f27c810a490 2024-05-03T14:43:31.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:31.230+0000 7f27c6000700 1 -- 172.21.15.60:0/3104263883 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b4010430 con 0x7f27c810a490 2024-05-03T14:43:31.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:31.230+0000 7f27d0600700 1 -- 172.21.15.60:0/3104263883 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27c81acee0 con 0x7f27c810a490 2024-05-03T14:43:31.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:31.231+0000 7f27d0600700 1 -- 172.21.15.60:0/3104263883 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27c8112060 con 0x7f27c810a490 2024-05-03T14:43:31.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:31.234+0000 7f27c6000700 1 -- 172.21.15.60:0/3104263883 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f27b4003f90 con 0x7f27c810a490 2024-05-03T14:43:31.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:31.235+0000 7f27c6000700 1 --2- 172.21.15.60:0/3104263883 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f27b00b2880 0x7f27b00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:43:31.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:31.235+0000 7f27c6000700 1 -- 172.21.15.60:0/3104263883 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f27b40e2160 con 0x7f27c810a490 2024-05-03T14:43:31.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:31.235+0000 7f27cdc00700 1 --2- 172.21.15.60:0/3104263883 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f27b00b2880 0x7f27b00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:43:31.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:31.236+0000 7f27cdc00700 1 --2- 172.21.15.60:0/3104263883 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f27b00b2880 0x7f27b00b4d40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f27bc006010 tx=0x7f27bc00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:43:31.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:31.237+0000 7f27c6000700 1 -- 172.21.15.60:0/3104263883 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f27b40aa280 con 0x7f27c810a490 2024-05-03T14:43:31.279 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 03 14:43:30 smithi149 podman[63580]: 2024-05-03 14:43:30.884160094 +0000 UTC m=+0.018316229 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T14:43:31.280 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 03 14:43:31 smithi149 podman[63580]: 2024-05-03 14:43:31.041608646 +0000 UTC m=+0.175764809 container create 9c403211bfc412d732fbb3308df962f8583628549f26d91e7ac17a99462f0ce6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-l, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, ceph=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD) 2024-05-03T14:43:31.280 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 03 14:43:31 smithi149 podman[63580]: 2024-05-03 14:43:31.278106473 +0000 UTC m=+0.412262597 container init 9c403211bfc412d732fbb3308df962f8583628549f26d91e7ac17a99462f0ce6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-l, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-05-03T14:43:31.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:31.552+0000 7f27d0600700 1 -- 172.21.15.60:0/3104263883 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f27c8111a40 con 0x7f27c810a490 2024-05-03T14:43:31.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:31.553+0000 7f27c6000700 1 -- 172.21.15.60:0/3104263883 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f27b40a9b80 con 0x7f27c810a490 2024-05-03T14:43:31.555 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:43:31.555 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/3454021291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3454021291},{"type":"v1","addr":"172.21.15.149:6833","nonce":3454021291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-03T14:43:31.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:31.556+0000 7f27d0600700 1 -- 172.21.15.60:0/3104263883 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f27b00b2880 msgr2=0x7f27b00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:43:31.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:31.556+0000 7f27d0600700 1 --2- 172.21.15.60:0/3104263883 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f27b00b2880 0x7f27b00b4d40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f27bc006010 tx=0x7f27bc00b260 comp rx=0 tx=0).stop 2024-05-03T14:43:31.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:31.556+0000 7f27d0600700 1 -- 172.21.15.60:0/3104263883 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27c810a490 msgr2=0x7f27c81a08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:43:31.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:31.556+0000 7f27d0600700 1 --2- 172.21.15.60:0/3104263883 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27c810a490 0x7f27c81a08a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f27b400eb30 tx=0x7f27b400ee40 comp rx=0 tx=0).stop 2024-05-03T14:43:31.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:31.556+0000 7f27d0600700 1 -- 172.21.15.60:0/3104263883 shutdown_connections 2024-05-03T14:43:31.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:31.556+0000 7f27d0600700 1 --2- 172.21.15.60:0/3104263883 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f27b00b2880 0x7f27b00b4d40 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:31.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:31.556+0000 7f27d0600700 1 --2- 172.21.15.60:0/3104263883 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27c810fb60 0x7f27c81a4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:31.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:31.556+0000 7f27d0600700 1 --2- 172.21.15.60:0/3104263883 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f27c8109ad0 0x7f27c81a0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:31.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:31.556+0000 7f27d0600700 1 --2- 172.21.15.60:0/3104263883 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27c810a490 0x7f27c81a08a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:31.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:31.556+0000 7f27d0600700 1 -- 172.21.15.60:0/3104263883 >> 172.21.15.60:0/3104263883 conn(0x7f27c8076c70 msgr2=0x7f27c80771f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:43:31.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:31.556+0000 7f27d0600700 1 -- 172.21.15.60:0/3104263883 shutdown_connections 2024-05-03T14:43:31.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:31.557+0000 7f27d0600700 1 -- 172.21.15.60:0/3104263883 wait complete. 2024-05-03T14:43:31.560 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 2 2024-05-03T14:43:31.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:31 smithi149 ceph-mon[33332]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:31.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:31 smithi149 ceph-mon[33332]: mds.? [v2:172.21.15.149:6832/3454021291,v1:172.21.15.149:6833/3454021291] up:boot 2024-05-03T14:43:31.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:31 smithi149 ceph-mon[33332]: fsmap 1 up:standby 2024-05-03T14:43:31.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:31 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-03T14:43:31.674 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 03 14:43:31 smithi149 podman[63580]: 2024-05-03 14:43:31.280775671 +0000 UTC m=+0.414931797 container start 9c403211bfc412d732fbb3308df962f8583628549f26d91e7ac17a99462f0ce6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-l, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-05-03T14:43:31.674 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 03 14:43:31 smithi149 ceph-mds[63599]: -- 172.21.15.149:0/1597187477 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562c64251a40 con 0x562c64224800 2024-05-03T14:43:31.674 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 03 14:43:31 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-l[63595]: starting mds.l at 2024-05-03T14:43:31.674 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 03 14:43:31 smithi149 bash[63580]: 9c403211bfc412d732fbb3308df962f8583628549f26d91e7ac17a99462f0ce6 2024-05-03T14:43:31.674 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 03 14:43:31 smithi149 systemd[1]: Started Ceph mds.l for 285d64b2-095a-11ef-bc97-c7b262605968. 2024-05-03T14:43:31.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:31 smithi060 ceph-mon[28873]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:31.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:31 smithi060 ceph-mon[28873]: mds.? [v2:172.21.15.149:6832/3454021291,v1:172.21.15.149:6833/3454021291] up:boot 2024-05-03T14:43:31.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:31 smithi060 ceph-mon[28873]: fsmap 1 up:standby 2024-05-03T14:43:31.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:31 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-03T14:43:31.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:31 smithi090 ceph-mon[33568]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:31.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:31 smithi090 ceph-mon[33568]: mds.? [v2:172.21.15.149:6832/3454021291,v1:172.21.15.149:6833/3454021291] up:boot 2024-05-03T14:43:31.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:31 smithi090 ceph-mon[33568]: fsmap 1 up:standby 2024-05-03T14:43:31.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:31 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-03T14:43:32.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:32 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:32.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:32 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:32.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:32 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:32.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:32 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T14:43:32.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:32 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T14:43:32.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:32 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:43:32.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:32 smithi060 ceph-mon[28873]: Deploying daemon mds.k on smithi090 2024-05-03T14:43:32.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:32 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/3104263883' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T14:43:32.846 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:32 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:32.846 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:32 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:32.846 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:32 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:32.846 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:32 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T14:43:32.846 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:32 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T14:43:32.846 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:32 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:43:32.846 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:32 smithi090 ceph-mon[33568]: Deploying daemon mds.k on smithi090 2024-05-03T14:43:32.846 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:32 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/3104263883' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T14:43:32.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:32 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:32.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:32 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:32.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:32 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:32.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:32 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T14:43:32.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:32 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T14:43:32.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:32 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:43:32.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:32 smithi149 ceph-mon[33332]: Deploying daemon mds.k on smithi090 2024-05-03T14:43:32.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:32 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/3104263883' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T14:43:33.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:33 smithi090 ceph-mon[33568]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:33.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:33 smithi060 ceph-mon[28873]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:33.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:33 smithi149 ceph-mon[33332]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:34.050 INFO:journalctl@ceph.mds.k.smithi090.stdout:May 03 14:43:33 smithi090 systemd[1]: Starting Ceph mds.k for 285d64b2-095a-11ef-bc97-c7b262605968... 2024-05-03T14:43:34.323 INFO:journalctl@ceph.mds.k.smithi090.stdout:May 03 14:43:34 smithi090 podman[62769]: 2024-05-03 14:43:33.951739967 +0000 UTC m=+0.018044768 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T14:43:34.323 INFO:journalctl@ceph.mds.k.smithi090.stdout:May 03 14:43:34 smithi090 podman[62769]: 2024-05-03 14:43:34.05728176 +0000 UTC m=+0.123586561 container create 4b2334b62b44c11dc84b7449582b73ba1ad90fa080c3428e9e1a110eebbf7613 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-k, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-05-03T14:43:34.323 INFO:journalctl@ceph.mds.k.smithi090.stdout:May 03 14:43:34 smithi090 podman[62769]: 2024-05-03 14:43:34.29413557 +0000 UTC m=+0.360440377 container init 4b2334b62b44c11dc84b7449582b73ba1ad90fa080c3428e9e1a110eebbf7613 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-k, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True) 2024-05-03T14:43:34.323 INFO:journalctl@ceph.mds.k.smithi090.stdout:May 03 14:43:34 smithi090 podman[62769]: 2024-05-03 14:43:34.297141716 +0000 UTC m=+0.363446517 container start 4b2334b62b44c11dc84b7449582b73ba1ad90fa080c3428e9e1a110eebbf7613 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-k, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True) 2024-05-03T14:43:34.372 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T14:43:34.625 INFO:journalctl@ceph.mds.k.smithi090.stdout:May 03 14:43:34 smithi090 ceph-mds[62789]: -- 172.21.15.90:0/236492913 <== mon.2 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556a9f7e1a40 con 0x556a9f7b5000 2024-05-03T14:43:34.625 INFO:journalctl@ceph.mds.k.smithi090.stdout:May 03 14:43:34 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-k[62785]: starting mds.k at 2024-05-03T14:43:34.625 INFO:journalctl@ceph.mds.k.smithi090.stdout:May 03 14:43:34 smithi090 bash[62769]: 4b2334b62b44c11dc84b7449582b73ba1ad90fa080c3428e9e1a110eebbf7613 2024-05-03T14:43:34.625 INFO:journalctl@ceph.mds.k.smithi090.stdout:May 03 14:43:34 smithi090 systemd[1]: Started Ceph mds.k for 285d64b2-095a-11ef-bc97-c7b262605968. 2024-05-03T14:43:34.833 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:43:35.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:35 smithi060 ceph-mon[28873]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:35.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:35 smithi060 ceph-mon[28873]: mds.? [v2:172.21.15.90:6832/3033424631,v1:172.21.15.90:6833/3033424631] up:boot 2024-05-03T14:43:35.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:35 smithi060 ceph-mon[28873]: fsmap 2 up:standby 2024-05-03T14:43:35.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:35 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-03T14:43:35.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:35 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:35.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:35 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:35.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:35 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:35.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:35 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T14:43:35.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:35 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T14:43:35.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:35 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:43:35.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:35 smithi060 ceph-mon[28873]: Deploying daemon mds.j on smithi060 2024-05-03T14:43:35.758 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:35 smithi090 ceph-mon[33568]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:35.759 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:35 smithi090 ceph-mon[33568]: mds.? [v2:172.21.15.90:6832/3033424631,v1:172.21.15.90:6833/3033424631] up:boot 2024-05-03T14:43:35.759 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:35 smithi090 ceph-mon[33568]: fsmap 2 up:standby 2024-05-03T14:43:35.759 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:35 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-03T14:43:35.759 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:35 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:35.759 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:35 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:35.759 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:35 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:35.759 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:35 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T14:43:35.759 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:35 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T14:43:35.759 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:35 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:43:35.759 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:35 smithi090 ceph-mon[33568]: Deploying daemon mds.j on smithi060 2024-05-03T14:43:35.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:35 smithi149 ceph-mon[33332]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:35.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:35 smithi149 ceph-mon[33332]: mds.? [v2:172.21.15.90:6832/3033424631,v1:172.21.15.90:6833/3033424631] up:boot 2024-05-03T14:43:35.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:35 smithi149 ceph-mon[33332]: fsmap 2 up:standby 2024-05-03T14:43:35.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:35 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-03T14:43:35.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:35 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:35.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:35 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:35.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:35 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:35.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:35 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T14:43:35.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:35 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T14:43:35.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:35 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:43:35.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:35 smithi149 ceph-mon[33332]: Deploying daemon mds.j on smithi060 2024-05-03T14:43:37.679 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:37 smithi060 ceph-mon[28873]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:37.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:37 smithi090 ceph-mon[33568]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:37.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:37 smithi149 ceph-mon[33332]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:38.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:38.147+0000 7ff385c00700 1 -- 172.21.15.60:0/540471308 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff380114670 msgr2=0x7ff380116a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:43:38.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:38.147+0000 7ff385c00700 1 --2- 172.21.15.60:0/540471308 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff380114670 0x7ff380116a60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff378009f90 tx=0x7ff37800b330 comp rx=0 tx=0).stop 2024-05-03T14:43:38.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:38.147+0000 7ff385c00700 1 -- 172.21.15.60:0/540471308 shutdown_connections 2024-05-03T14:43:38.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:38.147+0000 7ff385c00700 1 --2- 172.21.15.60:0/540471308 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff380114670 0x7ff380116a60 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:38.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:38.147+0000 7ff385c00700 1 --2- 172.21.15.60:0/540471308 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3800756d0 0x7ff380075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:38.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:38.147+0000 7ff385c00700 1 --2- 172.21.15.60:0/540471308 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff380075ff0 0x7ff380114130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:38.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:38.147+0000 7ff385c00700 1 -- 172.21.15.60:0/540471308 >> 172.21.15.60:0/540471308 conn(0x7ff38007a600 msgr2=0x7ff38007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:43:38.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:38.147+0000 7ff385c00700 1 -- 172.21.15.60:0/540471308 shutdown_connections 2024-05-03T14:43:38.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:38.147+0000 7ff385c00700 1 -- 172.21.15.60:0/540471308 wait complete. 2024-05-03T14:43:38.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:38.149+0000 7ff385c00700 1 Processor -- start 2024-05-03T14:43:38.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:38.149+0000 7ff385c00700 1 -- start start 2024-05-03T14:43:38.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:38.150+0000 7ff385c00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3800756d0 0x7ff3801a02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:43:38.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:38.150+0000 7ff385c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff380075ff0 0x7ff3801a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:43:38.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:38.150+0000 7ff385c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff380114670 0x7ff3801a4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:43:38.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:38.150+0000 7ff385c00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff380119f10 con 0x7ff3800756d0 2024-05-03T14:43:38.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:38.150+0000 7ff385c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff380119d90 con 0x7ff380075ff0 2024-05-03T14:43:38.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:38.150+0000 7ff385c00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff38011a090 con 0x7ff380114670 2024-05-03T14:43:38.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:38.150+0000 7ff376a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3800756d0 0x7ff3801a02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:43:38.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:38.150+0000 7ff376a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3800756d0 0x7ff3801a02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59168/0 (socket says 172.21.15.60:59168) 2024-05-03T14:43:38.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:38.150+0000 7ff376a00700 1 -- 172.21.15.60:0/2759256885 learned_addr learned my addr 172.21.15.60:0/2759256885 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:43:38.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:38.150+0000 7ff37ea00700 1 --2- 172.21.15.60:0/2759256885 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff380075ff0 0x7ff3801a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:43:38.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:38.150+0000 7ff37f400700 1 --2- 172.21.15.60:0/2759256885 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff380114670 0x7ff3801a4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:43:38.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:38.150+0000 7ff37f400700 1 -- 172.21.15.60:0/2759256885 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff380075ff0 msgr2=0x7ff3801a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:43:38.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:38.150+0000 7ff37f400700 1 --2- 172.21.15.60:0/2759256885 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff380075ff0 0x7ff3801a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:38.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:38.151+0000 7ff37f400700 1 -- 172.21.15.60:0/2759256885 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3800756d0 msgr2=0x7ff3801a02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:43:38.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:38.151+0000 7ff37f400700 1 --2- 172.21.15.60:0/2759256885 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3800756d0 0x7ff3801a02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:38.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:38.151+0000 7ff37f400700 1 -- 172.21.15.60:0/2759256885 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3801a52f0 con 0x7ff380114670 2024-05-03T14:43:38.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:38.151+0000 7ff37ea00700 1 --2- 172.21.15.60:0/2759256885 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff380075ff0 0x7ff3801a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:43:38.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:38.151+0000 7ff376a00700 1 --2- 172.21.15.60:0/2759256885 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3800756d0 0x7ff3801a02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:43:38.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:38.151+0000 7ff37f400700 1 --2- 172.21.15.60:0/2759256885 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff380114670 0x7ff3801a4bb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff37800b6c0 tx=0x7ff37800bc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:43:38.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:38.151+0000 7ff377e00700 1 -- 172.21.15.60:0/2759256885 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff378021070 con 0x7ff380114670 2024-05-03T14:43:38.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:38.151+0000 7ff385c00700 1 -- 172.21.15.60:0/2759256885 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3801a5580 con 0x7ff380114670 2024-05-03T14:43:38.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:38.152+0000 7ff377e00700 1 -- 172.21.15.60:0/2759256885 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7ff37800be10 con 0x7ff380114670 2024-05-03T14:43:38.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:38.152+0000 7ff377e00700 1 -- 172.21.15.60:0/2759256885 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff378005040 con 0x7ff380114670 2024-05-03T14:43:38.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:38.152+0000 7ff385c00700 1 -- 172.21.15.60:0/2759256885 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3801ace60 con 0x7ff380114670 2024-05-03T14:43:38.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:38.153+0000 7ff385c00700 1 -- 172.21.15.60:0/2759256885 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff38006a190 con 0x7ff380114670 2024-05-03T14:43:38.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:38.155+0000 7ff377e00700 1 -- 172.21.15.60:0/2759256885 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7ff378024070 con 0x7ff380114670 2024-05-03T14:43:38.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:38.157+0000 7ff377e00700 1 --2- 172.21.15.60:0/2759256885 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7ff3640b2880 0x7ff3640b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:43:38.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:38.157+0000 7ff377e00700 1 -- 172.21.15.60:0/2759256885 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7ff37800f020 con 0x7ff380114670 2024-05-03T14:43:38.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:38.157+0000 7ff376a00700 1 --2- 172.21.15.60:0/2759256885 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7ff3640b2880 0x7ff3640b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:43:38.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:38.158+0000 7ff376a00700 1 --2- 172.21.15.60:0/2759256885 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7ff3640b2880 0x7ff3640b4d40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff370005950 tx=0x7ff370009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:43:38.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:38.159+0000 7ff377e00700 1 -- 172.21.15.60:0/2759256885 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7ff37802b020 con 0x7ff380114670 2024-05-03T14:43:38.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:38.545+0000 7ff385c00700 1 -- 172.21.15.60:0/2759256885 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff3801ad030 con 0x7ff380114670 2024-05-03T14:43:38.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:38.546+0000 7ff377e00700 1 -- 172.21.15.60:0/2759256885 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7ff3780ab670 con 0x7ff380114670 2024-05-03T14:43:38.549 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:43:38.549 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/3033424631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":3033424631},{"type":"v1","addr":"172.21.15.90:6833","nonce":3033424631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/3454021291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3454021291},{"type":"v1","addr":"172.21.15.149:6833","nonce":3454021291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-03T14:43:38.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:38.549+0000 7ff385c00700 1 -- 172.21.15.60:0/2759256885 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7ff3640b2880 msgr2=0x7ff3640b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:43:38.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:38.549+0000 7ff385c00700 1 --2- 172.21.15.60:0/2759256885 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7ff3640b2880 0x7ff3640b4d40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff370005950 tx=0x7ff370009310 comp rx=0 tx=0).stop 2024-05-03T14:43:38.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:38.549+0000 7ff385c00700 1 -- 172.21.15.60:0/2759256885 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff380114670 msgr2=0x7ff3801a4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:43:38.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:38.549+0000 7ff385c00700 1 --2- 172.21.15.60:0/2759256885 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff380114670 0x7ff3801a4bb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff37800b6c0 tx=0x7ff37800bc20 comp rx=0 tx=0).stop 2024-05-03T14:43:38.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:38.549+0000 7ff385c00700 1 -- 172.21.15.60:0/2759256885 shutdown_connections 2024-05-03T14:43:38.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:38.549+0000 7ff385c00700 1 --2- 172.21.15.60:0/2759256885 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7ff3640b2880 0x7ff3640b4d40 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:38.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:38.549+0000 7ff385c00700 1 --2- 172.21.15.60:0/2759256885 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff380114670 0x7ff3801a4bb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:38.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:38.549+0000 7ff385c00700 1 --2- 172.21.15.60:0/2759256885 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3800756d0 0x7ff3801a02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:38.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:38.549+0000 7ff385c00700 1 --2- 172.21.15.60:0/2759256885 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff380075ff0 0x7ff3801a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:38.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:38.549+0000 7ff385c00700 1 -- 172.21.15.60:0/2759256885 >> 172.21.15.60:0/2759256885 conn(0x7ff38007a600 msgr2=0x7ff380113360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:43:38.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:38.549+0000 7ff385c00700 1 -- 172.21.15.60:0/2759256885 shutdown_connections 2024-05-03T14:43:38.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:38.550+0000 7ff385c00700 1 -- 172.21.15.60:0/2759256885 wait complete. 2024-05-03T14:43:38.553 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 3 2024-05-03T14:43:39.790 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:39 smithi060 ceph-mon[28873]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:39.790 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:39 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/2759256885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T14:43:39.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:39 smithi090 ceph-mon[33568]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:39.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:39 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/2759256885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T14:43:39.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:39 smithi149 ceph-mon[33332]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:39.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:39 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/2759256885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T14:43:40.088 INFO:journalctl@ceph.mds.j.smithi060.stdout:May 03 14:43:39 smithi060 podman[66596]: 2024-05-03 14:43:39.812275489 +0000 UTC m=+0.156126424 container create 6e88f6f17e7d7ac51bbf65716ec006ececd573e3590fe0e7f39b28f2027bc2a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-j, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-03T14:43:40.587 INFO:journalctl@ceph.mds.j.smithi060.stdout:May 03 14:43:40 smithi060 podman[66596]: 2024-05-03 14:43:40.111474385 +0000 UTC m=+0.455325363 container init 6e88f6f17e7d7ac51bbf65716ec006ececd573e3590fe0e7f39b28f2027bc2a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-j, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430) 2024-05-03T14:43:40.587 INFO:journalctl@ceph.mds.j.smithi060.stdout:May 03 14:43:40 smithi060 podman[66596]: 2024-05-03 14:43:40.122704311 +0000 UTC m=+0.466555253 container start 6e88f6f17e7d7ac51bbf65716ec006ececd573e3590fe0e7f39b28f2027bc2a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-j, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True) 2024-05-03T14:43:40.587 INFO:journalctl@ceph.mds.j.smithi060.stdout:May 03 14:43:40 smithi060 ceph-mds[66615]: -- 172.21.15.60:0/1870990877 <== mon.2 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ff9e3dda40 con 0x55ff9e3b1000 2024-05-03T14:43:40.587 INFO:journalctl@ceph.mds.j.smithi060.stdout:May 03 14:43:40 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-j[66611]: starting mds.j at 2024-05-03T14:43:40.587 INFO:journalctl@ceph.mds.j.smithi060.stdout:May 03 14:43:40 smithi060 bash[66596]: 6e88f6f17e7d7ac51bbf65716ec006ececd573e3590fe0e7f39b28f2027bc2a3 2024-05-03T14:43:40.587 INFO:journalctl@ceph.mds.j.smithi060.stdout:May 03 14:43:40 smithi060 systemd[1]: Started Ceph mds.j for 285d64b2-095a-11ef-bc97-c7b262605968. 2024-05-03T14:43:41.518 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T14:43:41.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:41 smithi149 ceph-mon[33332]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:41.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:41 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:41.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:41 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:41.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:41 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:41.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:41 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T14:43:41.675 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:41 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T14:43:41.675 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:41 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:43:41.675 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:41 smithi149 ceph-mon[33332]: Deploying daemon mds.d on smithi060 2024-05-03T14:43:41.675 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:41 smithi149 ceph-mon[33332]: mds.? [v2:172.21.15.60:6834/1849151522,v1:172.21.15.60:6835/1849151522] up:boot 2024-05-03T14:43:41.675 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:41 smithi149 ceph-mon[33332]: fsmap 3 up:standby 2024-05-03T14:43:41.675 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:41 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-03T14:43:41.745 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:41 smithi060 ceph-mon[28873]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:41.745 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:41 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:41.745 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:41 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:41.745 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:41 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:41.745 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:41 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T14:43:41.745 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:41 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T14:43:41.745 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:41 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:43:41.745 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:41 smithi060 ceph-mon[28873]: Deploying daemon mds.d on smithi060 2024-05-03T14:43:41.745 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:41 smithi060 ceph-mon[28873]: mds.? [v2:172.21.15.60:6834/1849151522,v1:172.21.15.60:6835/1849151522] up:boot 2024-05-03T14:43:41.745 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:41 smithi060 ceph-mon[28873]: fsmap 3 up:standby 2024-05-03T14:43:41.745 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:41 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-03T14:43:41.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:41 smithi090 ceph-mon[33568]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:41.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:41 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:41.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:41 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:41.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:41 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:41.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:41 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T14:43:41.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:41 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T14:43:41.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:41 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:43:41.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:41 smithi090 ceph-mon[33568]: Deploying daemon mds.d on smithi060 2024-05-03T14:43:41.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:41 smithi090 ceph-mon[33568]: mds.? [v2:172.21.15.60:6834/1849151522,v1:172.21.15.60:6835/1849151522] up:boot 2024-05-03T14:43:41.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:41 smithi090 ceph-mon[33568]: fsmap 3 up:standby 2024-05-03T14:43:41.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:41 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-03T14:43:42.923 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:43:43.655 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:43 smithi060 ceph-mon[28873]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:43.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:43 smithi149 ceph-mon[33332]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:43.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:43 smithi090 ceph-mon[33568]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:45.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:45.374+0000 7fd601a00700 1 -- 172.21.15.60:0/2215438855 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5fc06b180 msgr2=0x7fd5fc06b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:43:45.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:45.374+0000 7fd601a00700 1 --2- 172.21.15.60:0/2215438855 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5fc06b180 0x7fd5fc06b600 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd5e40098f0 tx=0x7fd5e4009c00 comp rx=0 tx=0).stop 2024-05-03T14:43:45.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:45.374+0000 7fd601a00700 1 -- 172.21.15.60:0/2215438855 shutdown_connections 2024-05-03T14:43:45.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:45.374+0000 7fd601a00700 1 --2- 172.21.15.60:0/2215438855 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd5fc06bb40 0x7fd5fc110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:45.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:45.374+0000 7fd601a00700 1 --2- 172.21.15.60:0/2215438855 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5fc109510 0x7fd5fc1098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:45.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:45.374+0000 7fd601a00700 1 --2- 172.21.15.60:0/2215438855 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5fc06b180 0x7fd5fc06b600 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:45.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:45.374+0000 7fd601a00700 1 -- 172.21.15.60:0/2215438855 >> 172.21.15.60:0/2215438855 conn(0x7fd5fc079350 msgr2=0x7fd5fc079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:43:45.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:45.374+0000 7fd601a00700 1 -- 172.21.15.60:0/2215438855 shutdown_connections 2024-05-03T14:43:45.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:45.374+0000 7fd601a00700 1 -- 172.21.15.60:0/2215438855 wait complete. 2024-05-03T14:43:45.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:45.375+0000 7fd601a00700 1 Processor -- start 2024-05-03T14:43:45.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:45.375+0000 7fd601a00700 1 -- start start 2024-05-03T14:43:45.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:45.376+0000 7fd601a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5fc06b180 0x7fd5fc102470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:43:45.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:45.376+0000 7fd601a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd5fc06bb40 0x7fd5fc1029b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:43:45.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:45.376+0000 7fd601a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5fc109510 0x7fd5fc102ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:43:45.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:45.376+0000 7fd601a00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc1138b0 con 0x7fd5fc109510 2024-05-03T14:43:45.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:45.376+0000 7fd601a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc113730 con 0x7fd5fc06bb40 2024-05-03T14:43:45.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:45.376+0000 7fd601a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc113a30 con 0x7fd5fc06b180 2024-05-03T14:43:45.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:45.376+0000 7fd5faa00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5fc06b180 0x7fd5fc102470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:43:45.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:45.376+0000 7fd5f2000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd5fc06bb40 0x7fd5fc1029b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:43:45.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:45.376+0000 7fd5fb400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5fc109510 0x7fd5fc102ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:43:45.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:45.376+0000 7fd5faa00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5fc06b180 0x7fd5fc102470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.60:37416/0 (socket says 172.21.15.60:37416) 2024-05-03T14:43:45.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:45.376+0000 7fd5faa00700 1 -- 172.21.15.60:0/1221605809 learned_addr learned my addr 172.21.15.60:0/1221605809 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:43:45.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:45.377+0000 7fd5f2000700 1 -- 172.21.15.60:0/1221605809 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5fc06b180 msgr2=0x7fd5fc102470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:43:45.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:45.377+0000 7fd5f2000700 1 --2- 172.21.15.60:0/1221605809 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5fc06b180 0x7fd5fc102470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:45.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:45.377+0000 7fd5f2000700 1 -- 172.21.15.60:0/1221605809 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5fc109510 msgr2=0x7fd5fc102ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:43:45.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:45.377+0000 7fd5f2000700 1 --2- 172.21.15.60:0/1221605809 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5fc109510 0x7fd5fc102ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:45.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:45.377+0000 7fd5f2000700 1 -- 172.21.15.60:0/1221605809 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5fc1a8250 con 0x7fd5fc06bb40 2024-05-03T14:43:45.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:45.377+0000 7fd5f2000700 1 --2- 172.21.15.60:0/1221605809 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd5fc06bb40 0x7fd5fc1029b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4000c00 tx=0x7fd5e4010f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:43:45.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:45.377+0000 7fd5f3e00700 1 -- 172.21.15.60:0/1221605809 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e4026070 con 0x7fd5fc06bb40 2024-05-03T14:43:45.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:45.377+0000 7fd601a00700 1 -- 172.21.15.60:0/1221605809 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5fc1a84e0 con 0x7fd5fc06bb40 2024-05-03T14:43:45.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:45.378+0000 7fd5f3e00700 1 -- 172.21.15.60:0/1221605809 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7fd5e400f960 con 0x7fd5fc06bb40 2024-05-03T14:43:45.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:45.378+0000 7fd5f3e00700 1 -- 172.21.15.60:0/1221605809 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e4003b40 con 0x7fd5fc06bb40 2024-05-03T14:43:45.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:45.378+0000 7fd601a00700 1 -- 172.21.15.60:0/1221605809 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5fc1a8990 con 0x7fd5fc06bb40 2024-05-03T14:43:45.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:45.379+0000 7fd601a00700 1 -- 172.21.15.60:0/1221605809 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5fc06a190 con 0x7fd5fc06bb40 2024-05-03T14:43:45.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:45.382+0000 7fd5f3e00700 1 -- 172.21.15.60:0/1221605809 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fd5e4029070 con 0x7fd5fc06bb40 2024-05-03T14:43:45.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:45.383+0000 7fd5f3e00700 1 --2- 172.21.15.60:0/1221605809 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fd5e00b2900 0x7fd5e00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:43:45.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:45.383+0000 7fd5f3e00700 1 -- 172.21.15.60:0/1221605809 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fd5e40e2780 con 0x7fd5fc06bb40 2024-05-03T14:43:45.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:45.384+0000 7fd5faa00700 1 --2- 172.21.15.60:0/1221605809 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fd5e00b2900 0x7fd5e00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:43:45.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:45.384+0000 7fd5faa00700 1 --2- 172.21.15.60:0/1221605809 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fd5e00b2900 0x7fd5e00b4dc0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec009ca0 tx=0x7fd5ec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:43:45.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:45.388+0000 7fd5f3e00700 1 -- 172.21.15.60:0/1221605809 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fd5e4013030 con 0x7fd5fc06bb40 2024-05-03T14:43:45.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:45 smithi060 ceph-mon[28873]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:45.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:45 smithi149 ceph-mon[33332]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:45.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:45.687+0000 7fd601a00700 1 -- 172.21.15.60:0/1221605809 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd5fc02e510 con 0x7fd5fc06bb40 2024-05-03T14:43:45.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:45.688+0000 7fd5f3e00700 1 -- 172.21.15.60:0/1221605809 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3868 (secure 0 0 0) 0x7fd5e40aa930 con 0x7fd5fc06bb40 2024-05-03T14:43:45.690 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:43:45.691 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1849151522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1849151522},{"type":"v1","addr":"172.21.15.60:6835","nonce":1849151522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/523710346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":523710346},{"type":"v1","addr":"172.21.15.60:6837","nonce":523710346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/3033424631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":3033424631},{"type":"v1","addr":"172.21.15.90:6833","nonce":3033424631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/3454021291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3454021291},{"type":"v1","addr":"172.21.15.149:6833","nonce":3454021291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-03T14:43:45.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:45.691+0000 7fd601a00700 1 -- 172.21.15.60:0/1221605809 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fd5e00b2900 msgr2=0x7fd5e00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:43:45.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:45.691+0000 7fd601a00700 1 --2- 172.21.15.60:0/1221605809 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fd5e00b2900 0x7fd5e00b4dc0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec009ca0 tx=0x7fd5ec009260 comp rx=0 tx=0).stop 2024-05-03T14:43:45.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:45.691+0000 7fd601a00700 1 -- 172.21.15.60:0/1221605809 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd5fc06bb40 msgr2=0x7fd5fc1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:43:45.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:45.692+0000 7fd601a00700 1 --2- 172.21.15.60:0/1221605809 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd5fc06bb40 0x7fd5fc1029b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4000c00 tx=0x7fd5e4010f80 comp rx=0 tx=0).stop 2024-05-03T14:43:45.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:45.692+0000 7fd601a00700 1 -- 172.21.15.60:0/1221605809 shutdown_connections 2024-05-03T14:43:45.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:45.692+0000 7fd601a00700 1 --2- 172.21.15.60:0/1221605809 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fd5e00b2900 0x7fd5e00b4dc0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:45.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:45.692+0000 7fd601a00700 1 --2- 172.21.15.60:0/1221605809 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5fc109510 0x7fd5fc102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:45.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:45.692+0000 7fd601a00700 1 --2- 172.21.15.60:0/1221605809 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5fc06b180 0x7fd5fc102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:45.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:45.692+0000 7fd601a00700 1 --2- 172.21.15.60:0/1221605809 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd5fc06bb40 0x7fd5fc1029b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:45.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:45.692+0000 7fd601a00700 1 -- 172.21.15.60:0/1221605809 >> 172.21.15.60:0/1221605809 conn(0x7fd5fc079350 msgr2=0x7fd5fc100f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:43:45.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:45.692+0000 7fd601a00700 1 -- 172.21.15.60:0/1221605809 shutdown_connections 2024-05-03T14:43:45.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:45.692+0000 7fd601a00700 1 -- 172.21.15.60:0/1221605809 wait complete. 2024-05-03T14:43:45.695 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 5 2024-05-03T14:43:45.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:45 smithi090 ceph-mon[33568]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:46.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:46 smithi090 ceph-mon[33568]: mds.? [v2:172.21.15.60:6836/523710346,v1:172.21.15.60:6837/523710346] up:boot 2024-05-03T14:43:46.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:46 smithi090 ceph-mon[33568]: fsmap 4 up:standby 2024-05-03T14:43:46.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:46 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-03T14:43:46.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:46 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:46.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:46 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:46.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:46 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:46.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:46 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T14:43:46.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:46 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T14:43:46.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:46 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:43:46.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:46 smithi090 ceph-mon[33568]: Deploying daemon mds.e on smithi090 2024-05-03T14:43:46.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:46 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/1221605809' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T14:43:46.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:46 smithi149 ceph-mon[33332]: mds.? [v2:172.21.15.60:6836/523710346,v1:172.21.15.60:6837/523710346] up:boot 2024-05-03T14:43:46.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:46 smithi149 ceph-mon[33332]: fsmap 4 up:standby 2024-05-03T14:43:46.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:46 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-03T14:43:46.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:46 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:46.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:46 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:46.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:46 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:46.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:46 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T14:43:46.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:46 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T14:43:46.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:46 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:43:46.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:46 smithi149 ceph-mon[33332]: Deploying daemon mds.e on smithi090 2024-05-03T14:43:46.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:46 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/1221605809' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T14:43:46.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:46 smithi060 ceph-mon[28873]: mds.? [v2:172.21.15.60:6836/523710346,v1:172.21.15.60:6837/523710346] up:boot 2024-05-03T14:43:46.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:46 smithi060 ceph-mon[28873]: fsmap 4 up:standby 2024-05-03T14:43:46.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:46 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-03T14:43:46.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:46 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:46.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:46 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:46.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:46 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:46.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:46 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T14:43:46.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:46 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T14:43:46.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:46 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:43:46.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:46 smithi060 ceph-mon[28873]: Deploying daemon mds.e on smithi090 2024-05-03T14:43:46.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:46 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/1221605809' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T14:43:47.785 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:47 smithi060 ceph-mon[28873]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:47.801 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:47 smithi090 ceph-mon[33568]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:47.802 INFO:journalctl@ceph.mds.e.smithi090.stdout:May 03 14:43:47 smithi090 systemd[1]: Starting Ceph mds.e for 285d64b2-095a-11ef-bc97-c7b262605968... 2024-05-03T14:43:47.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:47 smithi149 ceph-mon[33332]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:48.075 INFO:journalctl@ceph.mds.e.smithi090.stdout:May 03 14:43:47 smithi090 podman[63085]: 2024-05-03 14:43:47.703477473 +0000 UTC m=+0.018470593 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T14:43:48.075 INFO:journalctl@ceph.mds.e.smithi090.stdout:May 03 14:43:47 smithi090 podman[63085]: 2024-05-03 14:43:47.819237881 +0000 UTC m=+0.134230963 container create 5daafecc9028ebd3edb91faf819fa78f54abbd8bd8b2e390499e183bd6d277c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-05-03T14:43:48.076 INFO:journalctl@ceph.mds.e.smithi090.stdout:May 03 14:43:48 smithi090 podman[63085]: 2024-05-03 14:43:48.074161206 +0000 UTC m=+0.389154288 container init 5daafecc9028ebd3edb91faf819fa78f54abbd8bd8b2e390499e183bd6d277c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-e, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-05-03T14:43:48.375 INFO:journalctl@ceph.mds.e.smithi090.stdout:May 03 14:43:48 smithi090 podman[63085]: 2024-05-03 14:43:48.076759017 +0000 UTC m=+0.391752117 container start 5daafecc9028ebd3edb91faf819fa78f54abbd8bd8b2e390499e183bd6d277c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-e, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T14:43:48.375 INFO:journalctl@ceph.mds.e.smithi090.stdout:May 03 14:43:48 smithi090 ceph-mds[63104]: -- 172.21.15.90:0/389802687 <== mon.2 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555c997ada40 con 0x555c99780c00 2024-05-03T14:43:48.375 INFO:journalctl@ceph.mds.e.smithi090.stdout:May 03 14:43:48 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-e[63100]: starting mds.e at 2024-05-03T14:43:48.375 INFO:journalctl@ceph.mds.e.smithi090.stdout:May 03 14:43:48 smithi090 bash[63085]: 5daafecc9028ebd3edb91faf819fa78f54abbd8bd8b2e390499e183bd6d277c8 2024-05-03T14:43:48.375 INFO:journalctl@ceph.mds.e.smithi090.stdout:May 03 14:43:48 smithi090 systemd[1]: Started Ceph mds.e for 285d64b2-095a-11ef-bc97-c7b262605968. 2024-05-03T14:43:48.571 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T14:43:49.054 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:43:49.493 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:49 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:49.493 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:49 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:49.493 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:49 smithi060 ceph-mon[28873]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:49.493 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:49 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:49.493 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:49 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T14:43:49.493 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:49 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T14:43:49.493 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:49 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:43:49.493 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:49 smithi060 ceph-mon[28873]: Deploying daemon mds.f on smithi149 2024-05-03T14:43:49.494 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:49 smithi060 ceph-mon[28873]: mds.? [v2:172.21.15.90:6834/1057049585,v1:172.21.15.90:6835/1057049585] up:boot 2024-05-03T14:43:49.494 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:49 smithi060 ceph-mon[28873]: fsmap 5 up:standby 2024-05-03T14:43:49.494 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:49 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-03T14:43:49.579 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:49 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:49.579 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:49 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:49.579 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:49 smithi149 ceph-mon[33332]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:49.579 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:49 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:49.580 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:49 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T14:43:49.580 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:49 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T14:43:49.580 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:49 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:43:49.580 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:49 smithi149 ceph-mon[33332]: Deploying daemon mds.f on smithi149 2024-05-03T14:43:49.580 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:49 smithi149 ceph-mon[33332]: mds.? [v2:172.21.15.90:6834/1057049585,v1:172.21.15.90:6835/1057049585] up:boot 2024-05-03T14:43:49.580 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:49 smithi149 ceph-mon[33332]: fsmap 5 up:standby 2024-05-03T14:43:49.580 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:49 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-03T14:43:49.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:49 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:49.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:49 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:49.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:49 smithi090 ceph-mon[33568]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:49.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:49 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:49.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:49 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T14:43:49.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:49 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T14:43:49.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:49 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:43:49.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:49 smithi090 ceph-mon[33568]: Deploying daemon mds.f on smithi149 2024-05-03T14:43:49.626 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:49 smithi090 ceph-mon[33568]: mds.? [v2:172.21.15.90:6834/1057049585,v1:172.21.15.90:6835/1057049585] up:boot 2024-05-03T14:43:49.626 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:49 smithi090 ceph-mon[33568]: fsmap 5 up:standby 2024-05-03T14:43:49.626 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:49 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-03T14:43:50.674 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 03 14:43:50 smithi149 systemd[1]: Starting Ceph mds.f for 285d64b2-095a-11ef-bc97-c7b262605968... 2024-05-03T14:43:51.072 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 03 14:43:50 smithi149 podman[63904]: 2024-05-03 14:43:50.702531166 +0000 UTC m=+0.018076182 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T14:43:51.072 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 03 14:43:50 smithi149 podman[63904]: 2024-05-03 14:43:50.834213943 +0000 UTC m=+0.149758949 container create 937f42513b12102ff78784b5a177d7189f4ada6a0348d2713f9f4a38c8a244d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-f, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T14:43:51.072 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 03 14:43:51 smithi149 podman[63904]: 2024-05-03 14:43:51.070576696 +0000 UTC m=+0.386121698 container init 937f42513b12102ff78784b5a177d7189f4ada6a0348d2713f9f4a38c8a244d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-f, org.label-schema.build-date=20240430, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS) 2024-05-03T14:43:51.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:51.088+0000 7fd811600700 1 -- 172.21.15.60:0/3938355745 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd80c10e8b0 msgr2=0x7fd80c10ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:43:51.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:51.088+0000 7fd811600700 1 --2- 172.21.15.60:0/3938355745 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd80c10e8b0 0x7fd80c10ec90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc0098f0 tx=0x7fd7fc009c00 comp rx=0 tx=0).stop 2024-05-03T14:43:51.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:51.088+0000 7fd811600700 1 -- 172.21.15.60:0/3938355745 shutdown_connections 2024-05-03T14:43:51.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:51.088+0000 7fd811600700 1 --2- 172.21.15.60:0/3938355745 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd80c10a3a0 0x7fd80c116840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:51.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:51.088+0000 7fd811600700 1 --2- 172.21.15.60:0/3938355745 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd80c10e8b0 0x7fd80c10ec90 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:51.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:51.088+0000 7fd811600700 1 --2- 172.21.15.60:0/3938355745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd80c1099e0 0x7fd80c109e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:51.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:51.088+0000 7fd811600700 1 -- 172.21.15.60:0/3938355745 >> 172.21.15.60:0/3938355745 conn(0x7fd80c079230 msgr2=0x7fd80c1082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:43:51.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:51.088+0000 7fd811600700 1 -- 172.21.15.60:0/3938355745 shutdown_connections 2024-05-03T14:43:51.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:51.088+0000 7fd811600700 1 -- 172.21.15.60:0/3938355745 wait complete. 2024-05-03T14:43:51.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:51.090+0000 7fd811600700 1 Processor -- start 2024-05-03T14:43:51.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:51.090+0000 7fd811600700 1 -- start start 2024-05-03T14:43:51.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:51.090+0000 7fd811600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd80c1099e0 0x7fd80c1a0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:43:51.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:51.090+0000 7fd811600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd80c10a3a0 0x7fd80c1a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:43:51.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:51.090+0000 7fd811600700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd80c10e8b0 0x7fd80c1a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:43:51.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:51.091+0000 7fd811600700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd80c119e10 con 0x7fd80c10e8b0 2024-05-03T14:43:51.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:51.091+0000 7fd811600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd80c119c90 con 0x7fd80c10a3a0 2024-05-03T14:43:51.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:51.091+0000 7fd811600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd80c119f90 con 0x7fd80c1099e0 2024-05-03T14:43:51.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:51.091+0000 7fd80be00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd80c1099e0 0x7fd80c1a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:43:51.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:51.091+0000 7fd80be00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd80c1099e0 0x7fd80c1a0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.60:37422/0 (socket says 172.21.15.60:37422) 2024-05-03T14:43:51.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:51.091+0000 7fd80be00700 1 -- 172.21.15.60:0/2856470803 learned_addr learned my addr 172.21.15.60:0/2856470803 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:43:51.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:51.091+0000 7fd80b400700 1 --2- 172.21.15.60:0/2856470803 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd80c10a3a0 0x7fd80c1a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:43:51.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:51.091+0000 7fd810c00700 1 --2- 172.21.15.60:0/2856470803 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd80c10e8b0 0x7fd80c1a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:43:51.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:51.091+0000 7fd80be00700 1 -- 172.21.15.60:0/2856470803 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd80c10a3a0 msgr2=0x7fd80c1a07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:43:51.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:51.091+0000 7fd80be00700 1 --2- 172.21.15.60:0/2856470803 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd80c10a3a0 0x7fd80c1a07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:51.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:51.091+0000 7fd80be00700 1 -- 172.21.15.60:0/2856470803 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd80c10e8b0 msgr2=0x7fd80c1a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:43:51.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:51.091+0000 7fd80be00700 1 --2- 172.21.15.60:0/2856470803 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd80c10e8b0 0x7fd80c1a4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:51.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:51.091+0000 7fd80be00700 1 -- 172.21.15.60:0/2856470803 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd80c1a5240 con 0x7fd80c1099e0 2024-05-03T14:43:51.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:51.091+0000 7fd80b400700 1 --2- 172.21.15.60:0/2856470803 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd80c10a3a0 0x7fd80c1a07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:43:51.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:51.092+0000 7fd80be00700 1 --2- 172.21.15.60:0/2856470803 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd80c1099e0 0x7fd80c1a0280 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc009230 tx=0x7fd7fc010f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:43:51.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:51.092+0000 7fd808c00700 1 -- 172.21.15.60:0/2856470803 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc027070 con 0x7fd80c1099e0 2024-05-03T14:43:51.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:51.092+0000 7fd808c00700 1 -- 172.21.15.60:0/2856470803 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7fd7fc00bbe0 con 0x7fd80c1099e0 2024-05-03T14:43:51.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:51.092+0000 7fd811600700 1 -- 172.21.15.60:0/2856470803 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd80c1a54d0 con 0x7fd80c1099e0 2024-05-03T14:43:51.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:51.092+0000 7fd808c00700 1 -- 172.21.15.60:0/2856470803 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc004b30 con 0x7fd80c1099e0 2024-05-03T14:43:51.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:51.092+0000 7fd811600700 1 -- 172.21.15.60:0/2856470803 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd80c1acdb0 con 0x7fd80c1099e0 2024-05-03T14:43:51.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:51.093+0000 7fd811600700 1 -- 172.21.15.60:0/2856470803 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd80c06a190 con 0x7fd80c1099e0 2024-05-03T14:43:51.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:51.095+0000 7fd808c00700 1 -- 172.21.15.60:0/2856470803 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fd7fc02a070 con 0x7fd80c1099e0 2024-05-03T14:43:51.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:51.097+0000 7fd808c00700 1 --2- 172.21.15.60:0/2856470803 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fd7f00b2900 0x7fd7f00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:43:51.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:51.097+0000 7fd808c00700 1 -- 172.21.15.60:0/2856470803 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fd7fc0e2830 con 0x7fd80c1099e0 2024-05-03T14:43:51.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:51.097+0000 7fd80b400700 1 --2- 172.21.15.60:0/2856470803 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fd7f00b2900 0x7fd7f00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:43:51.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:51.098+0000 7fd80b400700 1 --2- 172.21.15.60:0/2856470803 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fd7f00b2900 0x7fd7f00b4dc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd7f8009750 tx=0x7fd7f8009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:43:51.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:51.099+0000 7fd808c00700 1 -- 172.21.15.60:0/2856470803 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fd7fc013030 con 0x7fd80c1099e0 2024-05-03T14:43:51.361 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 03 14:43:51 smithi149 podman[63904]: 2024-05-03 14:43:51.073200597 +0000 UTC m=+0.388745608 container start 937f42513b12102ff78784b5a177d7189f4ada6a0348d2713f9f4a38c8a244d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux ) 2024-05-03T14:43:51.361 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 03 14:43:51 smithi149 ceph-mds[63923]: -- 172.21.15.149:0/3032307754 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559071c63a40 con 0x559071c36800 2024-05-03T14:43:51.361 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 03 14:43:51 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-f[63919]: starting mds.f at 2024-05-03T14:43:51.361 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 03 14:43:51 smithi149 bash[63904]: 937f42513b12102ff78784b5a177d7189f4ada6a0348d2713f9f4a38c8a244d6 2024-05-03T14:43:51.361 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 03 14:43:51 smithi149 systemd[1]: Started Ceph mds.f for 285d64b2-095a-11ef-bc97-c7b262605968. 2024-05-03T14:43:51.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:51.403+0000 7fd811600700 1 -- 172.21.15.60:0/2856470803 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd80c1a1590 con 0x7fd80c1099e0 2024-05-03T14:43:51.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:51.404+0000 7fd808c00700 1 -- 172.21.15.60:0/2856470803 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5518 (secure 0 0 0) 0x7fd7fc0aa9e0 con 0x7fd80c1099e0 2024-05-03T14:43:51.407 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:43:51.407 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1849151522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1849151522},{"type":"v1","addr":"172.21.15.60:6835","nonce":1849151522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/523710346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":523710346},{"type":"v1","addr":"172.21.15.60:6837","nonce":523710346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/3033424631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":3033424631},{"type":"v1","addr":"172.21.15.90:6833","nonce":3033424631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/3454021291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3454021291},{"type":"v1","addr":"172.21.15.149:6833","nonce":3454021291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1057049585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1057049585},{"type":"v1","addr":"172.21.15.90:6835","nonce":1057049585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/882239924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":882239924},{"type":"v1","addr":"172.21.15.149:6835","nonce":882239924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-05-03T14:43:51.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:51.408+0000 7fd811600700 1 -- 172.21.15.60:0/2856470803 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fd7f00b2900 msgr2=0x7fd7f00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:43:51.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:51.408+0000 7fd811600700 1 --2- 172.21.15.60:0/2856470803 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fd7f00b2900 0x7fd7f00b4dc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd7f8009750 tx=0x7fd7f8009310 comp rx=0 tx=0).stop 2024-05-03T14:43:51.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:51.408+0000 7fd811600700 1 -- 172.21.15.60:0/2856470803 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd80c1099e0 msgr2=0x7fd80c1a0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:43:51.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:51.408+0000 7fd811600700 1 --2- 172.21.15.60:0/2856470803 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd80c1099e0 0x7fd80c1a0280 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc009230 tx=0x7fd7fc010f80 comp rx=0 tx=0).stop 2024-05-03T14:43:51.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:51.408+0000 7fd811600700 1 -- 172.21.15.60:0/2856470803 shutdown_connections 2024-05-03T14:43:51.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:51.408+0000 7fd811600700 1 --2- 172.21.15.60:0/2856470803 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fd7f00b2900 0x7fd7f00b4dc0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:51.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:51.408+0000 7fd811600700 1 --2- 172.21.15.60:0/2856470803 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd80c10e8b0 0x7fd80c1a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:51.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:51.408+0000 7fd811600700 1 --2- 172.21.15.60:0/2856470803 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd80c1099e0 0x7fd80c1a0280 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:51.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:51.408+0000 7fd811600700 1 --2- 172.21.15.60:0/2856470803 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd80c10a3a0 0x7fd80c1a07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:51.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:51.408+0000 7fd811600700 1 -- 172.21.15.60:0/2856470803 >> 172.21.15.60:0/2856470803 conn(0x7fd80c079230 msgr2=0x7fd80c0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:43:51.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:51.408+0000 7fd811600700 1 -- 172.21.15.60:0/2856470803 shutdown_connections 2024-05-03T14:43:51.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:51.408+0000 7fd811600700 1 -- 172.21.15.60:0/2856470803 wait complete. 2024-05-03T14:43:51.411 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 7 2024-05-03T14:43:51.624 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:51 smithi090 ceph-mon[33568]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:51.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:51 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:51.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:51 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:51.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:51 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:51.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:51 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T14:43:51.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:51 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T14:43:51.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:51 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:43:51.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:51 smithi149 ceph-mon[33332]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:51.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:51 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:51.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:51 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:51.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:51 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:51.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:51 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T14:43:51.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:51 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T14:43:51.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:51 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:43:51.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:51 smithi060 ceph-mon[28873]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:51.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:51 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:51.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:51 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:51.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:51 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:51.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:51 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T14:43:51.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:51 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T14:43:51.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:51 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:43:52.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:52 smithi060 ceph-mon[28873]: Deploying daemon mds.c on smithi149 2024-05-03T14:43:52.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:52 smithi060 ceph-mon[28873]: mds.? [v2:172.21.15.149:6834/882239924,v1:172.21.15.149:6835/882239924] up:boot 2024-05-03T14:43:52.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:52 smithi060 ceph-mon[28873]: fsmap 6 up:standby 2024-05-03T14:43:52.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:52 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-03T14:43:52.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:52 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/2856470803' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T14:43:52.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:52 smithi090 ceph-mon[33568]: Deploying daemon mds.c on smithi149 2024-05-03T14:43:52.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:52 smithi090 ceph-mon[33568]: mds.? [v2:172.21.15.149:6834/882239924,v1:172.21.15.149:6835/882239924] up:boot 2024-05-03T14:43:52.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:52 smithi090 ceph-mon[33568]: fsmap 6 up:standby 2024-05-03T14:43:52.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:52 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-03T14:43:52.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:52 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/2856470803' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T14:43:52.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:52 smithi149 ceph-mon[33332]: Deploying daemon mds.c on smithi149 2024-05-03T14:43:52.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:52 smithi149 ceph-mon[33332]: mds.? [v2:172.21.15.149:6834/882239924,v1:172.21.15.149:6835/882239924] up:boot 2024-05-03T14:43:52.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:52 smithi149 ceph-mon[33332]: fsmap 6 up:standby 2024-05-03T14:43:52.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:52 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-03T14:43:52.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:52 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/2856470803' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T14:43:53.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:53 smithi149 ceph-mon[33332]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:53.624 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:53 smithi090 ceph-mon[33568]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:53.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:53 smithi060 ceph-mon[28873]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:53.872 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 03 14:43:53 smithi149 systemd[1]: Starting Ceph mds.c for 285d64b2-095a-11ef-bc97-c7b262605968... 2024-05-03T14:43:54.173 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 03 14:43:53 smithi149 podman[64205]: 2024-05-03 14:43:53.773115119 +0000 UTC m=+0.018005687 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T14:43:54.173 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 03 14:43:53 smithi149 podman[64205]: 2024-05-03 14:43:53.882289217 +0000 UTC m=+0.127179760 container create 45b18e5ebe10a70f66c319db9da6a824a4f5ca851bf651c9f1c1f183a5c7a3b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-c, ceph=True, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-03T14:43:54.286 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T14:43:54.673 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 03 14:43:54 smithi149 podman[64205]: 2024-05-03 14:43:54.194096334 +0000 UTC m=+0.438986880 container init 45b18e5ebe10a70f66c319db9da6a824a4f5ca851bf651c9f1c1f183a5c7a3b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-c, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-03T14:43:54.674 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 03 14:43:54 smithi149 podman[64205]: 2024-05-03 14:43:54.196745566 +0000 UTC m=+0.441636115 container start 45b18e5ebe10a70f66c319db9da6a824a4f5ca851bf651c9f1c1f183a5c7a3b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-c, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T14:43:54.674 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 03 14:43:54 smithi149 ceph-mds[64224]: -- 172.21.15.149:0/897177422 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5648f5bf1a40 con 0x5648f5bc4c00 2024-05-03T14:43:54.674 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 03 14:43:54 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-c[64220]: starting mds.c at 2024-05-03T14:43:54.674 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 03 14:43:54 smithi149 bash[64205]: 45b18e5ebe10a70f66c319db9da6a824a4f5ca851bf651c9f1c1f183a5c7a3b7 2024-05-03T14:43:54.674 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 03 14:43:54 smithi149 systemd[1]: Started Ceph mds.c for 285d64b2-095a-11ef-bc97-c7b262605968. 2024-05-03T14:43:54.767 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:43:55.585 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:55 smithi090 ceph-mon[33568]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:55.585 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:55 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:55.585 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:55 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:55.585 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:55 smithi090 ceph-mon[33568]: mds.? [v2:172.21.15.149:6836/2611278690,v1:172.21.15.149:6837/2611278690] up:boot 2024-05-03T14:43:55.585 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:55 smithi090 ceph-mon[33568]: fsmap 7 up:standby 2024-05-03T14:43:55.585 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:55 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-03T14:43:55.585 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:55 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-03T14:43:55.585 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:55 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:55.585 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:55 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T14:43:55.585 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:55 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T14:43:55.586 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:55 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:43:55.586 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:55 smithi090 ceph-mon[33568]: Deploying daemon mds.h on smithi090 2024-05-03T14:43:55.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:55 smithi060 ceph-mon[28873]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:55.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:55 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:55.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:55 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:55.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:55 smithi060 ceph-mon[28873]: mds.? [v2:172.21.15.149:6836/2611278690,v1:172.21.15.149:6837/2611278690] up:boot 2024-05-03T14:43:55.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:55 smithi060 ceph-mon[28873]: fsmap 7 up:standby 2024-05-03T14:43:55.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:55 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-03T14:43:55.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:55 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-03T14:43:55.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:55 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:55.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:55 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T14:43:55.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:55 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T14:43:55.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:55 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:43:55.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:55 smithi060 ceph-mon[28873]: Deploying daemon mds.h on smithi090 2024-05-03T14:43:55.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:55 smithi149 ceph-mon[33332]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:55.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:55 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:55.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:55 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:55.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:55 smithi149 ceph-mon[33332]: mds.? [v2:172.21.15.149:6836/2611278690,v1:172.21.15.149:6837/2611278690] up:boot 2024-05-03T14:43:55.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:55 smithi149 ceph-mon[33332]: fsmap 7 up:standby 2024-05-03T14:43:55.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:55 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-03T14:43:55.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:55 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-03T14:43:55.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:55 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:55.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:55 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T14:43:55.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:55 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T14:43:55.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:55 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:43:55.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:55 smithi149 ceph-mon[33332]: Deploying daemon mds.h on smithi090 2024-05-03T14:43:57.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:57.081+0000 7f1a9be00700 1 -- 172.21.15.60:0/802947424 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1a9c104f20 msgr2=0x7f1a9c10d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:43:57.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:57.081+0000 7f1a9be00700 1 --2- 172.21.15.60:0/802947424 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1a9c104f20 0x7f1a9c10d410 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1a84009990 tx=0x7f1a84009ca0 comp rx=0 tx=0).stop 2024-05-03T14:43:57.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:57.081+0000 7f1a9be00700 1 -- 172.21.15.60:0/802947424 shutdown_connections 2024-05-03T14:43:57.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:57.081+0000 7f1a9be00700 1 --2- 172.21.15.60:0/802947424 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a9c10da30 0x7f1a9c10fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:57.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:57.081+0000 7f1a9be00700 1 --2- 172.21.15.60:0/802947424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a9c104600 0x7f1a9c1049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:57.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:57.081+0000 7f1a9be00700 1 --2- 172.21.15.60:0/802947424 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1a9c104f20 0x7f1a9c10d410 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:57.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:57.081+0000 7f1a9be00700 1 -- 172.21.15.60:0/802947424 >> 172.21.15.60:0/802947424 conn(0x7f1a9c0fde50 msgr2=0x7f1a9c100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:43:57.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:57.081+0000 7f1a9be00700 1 -- 172.21.15.60:0/802947424 shutdown_connections 2024-05-03T14:43:57.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:57.081+0000 7f1a9be00700 1 -- 172.21.15.60:0/802947424 wait complete. 2024-05-03T14:43:57.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:57.083+0000 7f1a9be00700 1 Processor -- start 2024-05-03T14:43:57.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:57.083+0000 7f1a9be00700 1 -- start start 2024-05-03T14:43:57.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:57.083+0000 7f1a9be00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a9c104600 0x7f1a9c102260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:43:57.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:57.083+0000 7f1a9be00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a9c104f20 0x7f1a9c1027a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:43:57.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:57.083+0000 7f1a9be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1a9c10da30 0x7f1a9c102ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:43:57.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:57.083+0000 7f1a9be00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f1a9c113900 con 0x7f1a9c104600 2024-05-03T14:43:57.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:57.083+0000 7f1a9be00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1a9c113780 con 0x7f1a9c10da30 2024-05-03T14:43:57.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:57.083+0000 7f1a9be00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1a9c113a80 con 0x7f1a9c104f20 2024-05-03T14:43:57.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:57.084+0000 7f1a9aa00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a9c104600 0x7f1a9c102260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:43:57.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:57.084+0000 7f1a9aa00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a9c104600 0x7f1a9c102260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56880/0 (socket says 172.21.15.60:56880) 2024-05-03T14:43:57.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:57.084+0000 7f1a9aa00700 1 -- 172.21.15.60:0/1038437254 learned_addr learned my addr 172.21.15.60:0/1038437254 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:43:57.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:57.084+0000 7f1a9b400700 1 --2- 172.21.15.60:0/1038437254 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1a9c10da30 0x7f1a9c102ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:43:57.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:57.084+0000 7f1a92000700 1 --2- 172.21.15.60:0/1038437254 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a9c104f20 0x7f1a9c1027a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:43:57.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:57.084+0000 7f1a9aa00700 1 -- 172.21.15.60:0/1038437254 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a9c104f20 msgr2=0x7f1a9c1027a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:43:57.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:57.084+0000 7f1a9aa00700 1 --2- 172.21.15.60:0/1038437254 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a9c104f20 0x7f1a9c1027a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:57.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:57.084+0000 7f1a9aa00700 1 -- 172.21.15.60:0/1038437254 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1a9c10da30 msgr2=0x7f1a9c102ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:43:57.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:57.084+0000 7f1a9aa00700 1 --2- 172.21.15.60:0/1038437254 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1a9c10da30 0x7f1a9c102ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:57.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:57.084+0000 7f1a9aa00700 1 -- 172.21.15.60:0/1038437254 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a9c1a82d0 con 0x7f1a9c104600 2024-05-03T14:43:57.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:57.084+0000 7f1a9b400700 1 --2- 172.21.15.60:0/1038437254 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1a9c10da30 0x7f1a9c102ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:43:57.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:57.085+0000 7f1a9aa00700 1 --2- 172.21.15.60:0/1038437254 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a9c104600 0x7f1a9c102260 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c00e9c0 tx=0x7f1a8c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:43:57.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:57.085+0000 7f1a93e00700 1 -- 172.21.15.60:0/1038437254 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c00cbd0 con 0x7f1a9c104600 2024-05-03T14:43:57.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:57.085+0000 7f1a93e00700 1 -- 172.21.15.60:0/1038437254 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f1a8c00ee60 con 0x7f1a9c104600 2024-05-03T14:43:57.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:57.085+0000 7f1a93e00700 1 -- 172.21.15.60:0/1038437254 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c010430 con 0x7f1a9c104600 2024-05-03T14:43:57.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:57.085+0000 7f1a9be00700 1 -- 172.21.15.60:0/1038437254 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a9c1a85c0 con 0x7f1a9c104600 2024-05-03T14:43:57.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:57.085+0000 7f1a9be00700 1 -- 172.21.15.60:0/1038437254 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a9c1a8a80 con 0x7f1a9c104600 2024-05-03T14:43:57.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:57.086+0000 7f1a9be00700 1 -- 172.21.15.60:0/1038437254 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a9c06a190 con 0x7f1a9c104600 2024-05-03T14:43:57.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:57.089+0000 7f1a93e00700 1 -- 172.21.15.60:0/1038437254 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f1a8c004750 con 0x7f1a9c104600 2024-05-03T14:43:57.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:57.090+0000 7f1a93e00700 1 --2- 172.21.15.60:0/1038437254 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f1a7c0b2880 0x7f1a7c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:43:57.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:57.090+0000 7f1a93e00700 1 -- 172.21.15.60:0/1038437254 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f1a8c0e20d0 con 0x7f1a9c104600 2024-05-03T14:43:57.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:57.090+0000 7f1a92000700 1 --2- 172.21.15.60:0/1038437254 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f1a7c0b2880 0x7f1a7c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:43:57.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:57.091+0000 7f1a92000700 1 --2- 172.21.15.60:0/1038437254 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f1a7c0b2880 0x7f1a7c0b4d40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c103ca0 tx=0x7f1a84005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:43:57.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:57.095+0000 7f1a93e00700 1 -- 172.21.15.60:0/1038437254 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f1a8c0aa1f0 con 0x7f1a9c104600 2024-05-03T14:43:57.353 INFO:journalctl@ceph.mds.h.smithi090.stdout:May 03 14:43:57 smithi090 systemd[1]: Started Ceph mds.h for 285d64b2-095a-11ef-bc97-c7b262605968. 2024-05-03T14:43:57.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:57.394+0000 7f1a9be00700 1 -- 172.21.15.60:0/1038437254 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1a9c103740 con 0x7f1a9c104600 2024-05-03T14:43:57.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:57.395+0000 7f1a93e00700 1 -- 172.21.15.60:0/1038437254 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7171 (secure 0 0 0) 0x7f1a8c0aae20 con 0x7f1a9c104600 2024-05-03T14:43:57.397 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:43:57.398 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1849151522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1849151522},{"type":"v1","addr":"172.21.15.60:6835","nonce":1849151522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/523710346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":523710346},{"type":"v1","addr":"172.21.15.60:6837","nonce":523710346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/3033424631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":3033424631},{"type":"v1","addr":"172.21.15.90:6833","nonce":3033424631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/3454021291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3454021291},{"type":"v1","addr":"172.21.15.149:6833","nonce":3454021291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1057049585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1057049585},{"type":"v1","addr":"172.21.15.90:6835","nonce":1057049585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/882239924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":882239924},{"type":"v1","addr":"172.21.15.149:6835","nonce":882239924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/2243702844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":2243702844},{"type":"v1","addr":"172.21.15.90:6837","nonce":2243702844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/2611278690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2611278690},{"type":"v1","addr":"172.21.15.149:6837","nonce":2611278690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-05-03T14:43:57.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:57.398+0000 7f1a9be00700 1 -- 172.21.15.60:0/1038437254 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f1a7c0b2880 msgr2=0x7f1a7c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:43:57.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:57.398+0000 7f1a9be00700 1 --2- 172.21.15.60:0/1038437254 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f1a7c0b2880 0x7f1a7c0b4d40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c103ca0 tx=0x7f1a84005fd0 comp rx=0 tx=0).stop 2024-05-03T14:43:57.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:57.398+0000 7f1a9be00700 1 -- 172.21.15.60:0/1038437254 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a9c104600 msgr2=0x7f1a9c102260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:43:57.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:57.398+0000 7f1a9be00700 1 --2- 172.21.15.60:0/1038437254 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a9c104600 0x7f1a9c102260 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c00e9c0 tx=0x7f1a8c00ecd0 comp rx=0 tx=0).stop 2024-05-03T14:43:57.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:57.398+0000 7f1a9be00700 1 -- 172.21.15.60:0/1038437254 shutdown_connections 2024-05-03T14:43:57.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:57.398+0000 7f1a9be00700 1 --2- 172.21.15.60:0/1038437254 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f1a7c0b2880 0x7f1a7c0b4d40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:57.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:57.398+0000 7f1a9be00700 1 --2- 172.21.15.60:0/1038437254 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1a9c10da30 0x7f1a9c102ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:57.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:57.398+0000 7f1a9be00700 1 --2- 172.21.15.60:0/1038437254 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a9c104600 0x7f1a9c102260 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:57.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:57.398+0000 7f1a9be00700 1 --2- 172.21.15.60:0/1038437254 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a9c104f20 0x7f1a9c1027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:43:57.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:57.398+0000 7f1a9be00700 1 -- 172.21.15.60:0/1038437254 >> 172.21.15.60:0/1038437254 conn(0x7f1a9c0fde50 msgr2=0x7f1a9c0feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:43:57.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:57.398+0000 7f1a9be00700 1 -- 172.21.15.60:0/1038437254 shutdown_connections 2024-05-03T14:43:57.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:43:57.399+0000 7f1a9be00700 1 -- 172.21.15.60:0/1038437254 wait complete. 2024-05-03T14:43:57.402 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 9 2024-05-03T14:43:57.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:57 smithi090 ceph-mon[33568]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:57.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:57 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:57.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:57 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:57.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:57 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:57.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:57 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T14:43:57.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:57 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T14:43:57.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:57 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:43:57.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:57 smithi149 ceph-mon[33332]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:57.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:57 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:57.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:57 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:57.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:57 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:57.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:57 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T14:43:57.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:57 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T14:43:57.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:57 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:43:57.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:57 smithi060 ceph-mon[28873]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:57.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:57 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:57.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:57 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:57.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:57 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:43:57.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:57 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T14:43:57.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:57 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T14:43:57.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:57 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:43:58.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:58 smithi060 ceph-mon[28873]: Deploying daemon mds.i on smithi149 2024-05-03T14:43:58.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:58 smithi060 ceph-mon[28873]: mds.? [v2:172.21.15.90:6836/2243702844,v1:172.21.15.90:6837/2243702844] up:boot 2024-05-03T14:43:58.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:58 smithi060 ceph-mon[28873]: fsmap 8 up:standby 2024-05-03T14:43:58.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:58 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-03T14:43:58.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:58 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-03T14:43:58.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:58 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/1038437254' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T14:43:58.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:58 smithi090 ceph-mon[33568]: Deploying daemon mds.i on smithi149 2024-05-03T14:43:58.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:58 smithi090 ceph-mon[33568]: mds.? [v2:172.21.15.90:6836/2243702844,v1:172.21.15.90:6837/2243702844] up:boot 2024-05-03T14:43:58.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:58 smithi090 ceph-mon[33568]: fsmap 8 up:standby 2024-05-03T14:43:58.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:58 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-03T14:43:58.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:58 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-03T14:43:58.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:58 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/1038437254' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T14:43:58.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:58 smithi149 ceph-mon[33332]: Deploying daemon mds.i on smithi149 2024-05-03T14:43:58.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:58 smithi149 ceph-mon[33332]: mds.? [v2:172.21.15.90:6836/2243702844,v1:172.21.15.90:6837/2243702844] up:boot 2024-05-03T14:43:58.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:58 smithi149 ceph-mon[33332]: fsmap 8 up:standby 2024-05-03T14:43:58.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:58 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-03T14:43:58.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:58 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-03T14:43:58.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:58 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/1038437254' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T14:43:59.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:43:59 smithi149 ceph-mon[33332]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:59.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:43:59 smithi060 ceph-mon[28873]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:43:59.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:43:59 smithi090 ceph-mon[33568]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:00.173 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 03 14:43:59 smithi149 systemd[1]: Started Ceph mds.i for 285d64b2-095a-11ef-bc97-c7b262605968. 2024-05-03T14:44:00.324 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T14:44:00.787 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:44:01.243 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:01 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:01.244 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:01 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:01.244 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:01 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:01.244 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:01 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T14:44:01.244 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:01 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T14:44:01.244 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:01 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:44:01.244 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:01 smithi060 ceph-mon[28873]: Deploying daemon mds.g on smithi060 2024-05-03T14:44:01.244 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:01 smithi060 ceph-mon[28873]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:01.244 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:01 smithi060 ceph-mon[28873]: mds.? [v2:172.21.15.149:6838/2527622212,v1:172.21.15.149:6839/2527622212] up:boot 2024-05-03T14:44:01.244 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:01 smithi060 ceph-mon[28873]: fsmap 9 up:standby 2024-05-03T14:44:01.244 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:01 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-03T14:44:01.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:01 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:01.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:01 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:01.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:01 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:01.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:01 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T14:44:01.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:01 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T14:44:01.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:01 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:44:01.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:01 smithi090 ceph-mon[33568]: Deploying daemon mds.g on smithi060 2024-05-03T14:44:01.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:01 smithi090 ceph-mon[33568]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:01.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:01 smithi090 ceph-mon[33568]: mds.? [v2:172.21.15.149:6838/2527622212,v1:172.21.15.149:6839/2527622212] up:boot 2024-05-03T14:44:01.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:01 smithi090 ceph-mon[33568]: fsmap 9 up:standby 2024-05-03T14:44:01.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:01 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-03T14:44:01.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:01 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:01.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:01 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:01.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:01 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:01.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:01 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T14:44:01.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:01 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T14:44:01.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:01 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:44:01.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:01 smithi149 ceph-mon[33332]: Deploying daemon mds.g on smithi060 2024-05-03T14:44:01.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:01 smithi149 ceph-mon[33332]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:01.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:01 smithi149 ceph-mon[33332]: mds.? [v2:172.21.15.149:6838/2527622212,v1:172.21.15.149:6839/2527622212] up:boot 2024-05-03T14:44:01.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:01 smithi149 ceph-mon[33332]: fsmap 9 up:standby 2024-05-03T14:44:01.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:01 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-03T14:44:03.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:03.128+0000 7fea92c00700 1 -- 172.21.15.60:0/2244222304 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea8c10a490 msgr2=0x7fea8c1169a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:03.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:03.128+0000 7fea92c00700 1 --2- 172.21.15.60:0/2244222304 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea8c10a490 0x7fea8c1169a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fea8400b0a0 tx=0x7fea8400b3b0 comp rx=0 tx=0).stop 2024-05-03T14:44:03.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:03.128+0000 7fea92c00700 1 -- 172.21.15.60:0/2244222304 shutdown_connections 2024-05-03T14:44:03.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:03.128+0000 7fea92c00700 1 --2- 172.21.15.60:0/2244222304 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea8c10a490 0x7fea8c1169a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:03.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:03.128+0000 7fea92c00700 1 --2- 172.21.15.60:0/2244222304 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea8c10fb60 0x7fea8c10ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:03.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:03.128+0000 7fea92c00700 1 --2- 172.21.15.60:0/2244222304 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea8c109ad0 0x7fea8c109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:03.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:03.128+0000 7fea92c00700 1 -- 172.21.15.60:0/2244222304 >> 172.21.15.60:0/2244222304 conn(0x7fea8c076c70 msgr2=0x7fea8c1082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:44:03.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:03.128+0000 7fea92c00700 1 -- 172.21.15.60:0/2244222304 shutdown_connections 2024-05-03T14:44:03.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:03.128+0000 7fea92c00700 1 -- 172.21.15.60:0/2244222304 wait complete. 2024-05-03T14:44:03.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:03.129+0000 7fea92c00700 1 Processor -- start 2024-05-03T14:44:03.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:03.130+0000 7fea92c00700 1 -- start start 2024-05-03T14:44:03.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:03.130+0000 7fea92c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea8c109ad0 0x7fea8c1a02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:44:03.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:03.130+0000 7fea92c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea8c10a490 0x7fea8c1a0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:44:03.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:03.130+0000 7fea92c00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea8c10fb60 0x7fea8c1a4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:44:03.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:03.130+0000 7fea92c00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fea8c119f30 con 0x7fea8c10fb60 2024-05-03T14:44:03.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:03.130+0000 7fea92c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fea8c119db0 con 0x7fea8c10a490 2024-05-03T14:44:03.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:03.130+0000 7fea92c00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fea8c11a0b0 con 0x7fea8c109ad0 2024-05-03T14:44:03.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:03.130+0000 7fea8be00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea8c109ad0 0x7fea8c1a02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:44:03.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:03.130+0000 7fea8be00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea8c109ad0 0x7fea8c1a02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.60:55496/0 (socket says 172.21.15.60:55496) 2024-05-03T14:44:03.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:03.130+0000 7fea8be00700 1 -- 172.21.15.60:0/3821535763 learned_addr learned my addr 172.21.15.60:0/3821535763 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:44:03.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:03.130+0000 7fea83400700 1 --2- 172.21.15.60:0/3821535763 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea8c10a490 0x7fea8c1a0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:44:03.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:03.131+0000 7fea90c00700 1 --2- 172.21.15.60:0/3821535763 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea8c10fb60 0x7fea8c1a4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:44:03.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:03.131+0000 7fea8be00700 1 -- 172.21.15.60:0/3821535763 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea8c10a490 msgr2=0x7fea8c1a0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:03.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:03.131+0000 7fea8be00700 1 --2- 172.21.15.60:0/3821535763 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea8c10a490 0x7fea8c1a0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:03.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:03.131+0000 7fea8be00700 1 -- 172.21.15.60:0/3821535763 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea8c10fb60 msgr2=0x7fea8c1a4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:03.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:03.131+0000 7fea8be00700 1 --2- 172.21.15.60:0/3821535763 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea8c10fb60 0x7fea8c1a4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:03.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:03.131+0000 7fea8be00700 1 -- 172.21.15.60:0/3821535763 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea8c1a52d0 con 0x7fea8c109ad0 2024-05-03T14:44:03.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:03.131+0000 7fea8be00700 1 --2- 172.21.15.60:0/3821535763 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea8c109ad0 0x7fea8c1a02c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fea7c00d980 tx=0x7fea7c00dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:44:03.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:03.131+0000 7fea89600700 1 -- 172.21.15.60:0/3821535763 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7c016070 con 0x7fea8c109ad0 2024-05-03T14:44:03.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:03.131+0000 7fea89600700 1 -- 172.21.15.60:0/3821535763 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7fea7c004d30 con 0x7fea8c109ad0 2024-05-03T14:44:03.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:03.131+0000 7fea92c00700 1 -- 172.21.15.60:0/3821535763 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea8c1a55c0 con 0x7fea8c109ad0 2024-05-03T14:44:03.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:03.132+0000 7fea89600700 1 -- 172.21.15.60:0/3821535763 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7c005680 con 0x7fea8c109ad0 2024-05-03T14:44:03.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:03.132+0000 7fea92c00700 1 -- 172.21.15.60:0/3821535763 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea8c1ace40 con 0x7fea8c109ad0 2024-05-03T14:44:03.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:03.133+0000 7fea92c00700 1 -- 172.21.15.60:0/3821535763 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea8c06a190 con 0x7fea8c109ad0 2024-05-03T14:44:03.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:03.135+0000 7fea89600700 1 -- 172.21.15.60:0/3821535763 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fea7c004750 con 0x7fea8c109ad0 2024-05-03T14:44:03.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:03.136+0000 7fea89600700 1 --2- 172.21.15.60:0/3821535763 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fea700b2880 0x7fea700b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:44:03.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:03.137+0000 7fea89600700 1 -- 172.21.15.60:0/3821535763 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fea7c0e1b50 con 0x7fea8c109ad0 2024-05-03T14:44:03.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:03.137+0000 7fea83400700 1 --2- 172.21.15.60:0/3821535763 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fea700b2880 0x7fea700b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:44:03.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:03.137+0000 7fea83400700 1 --2- 172.21.15.60:0/3821535763 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fea700b2880 0x7fea700b4d40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fea8c0789f0 tx=0x7fea7400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:44:03.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:03.141+0000 7fea89600700 1 -- 172.21.15.60:0/3821535763 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fea7c01f020 con 0x7fea8c109ad0 2024-05-03T14:44:03.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:03.442+0000 7fea92c00700 1 -- 172.21.15.60:0/3821535763 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea8c1ad010 con 0x7fea8c109ad0 2024-05-03T14:44:03.445 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:03 smithi060 ceph-mon[28873]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:03.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:03.443+0000 7fea89600700 1 -- 172.21.15.60:0/3821535763 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8001 (secure 0 0 0) 0x7fea7c0a9c70 con 0x7fea8c109ad0 2024-05-03T14:44:03.446 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:44:03.446 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1849151522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1849151522},{"type":"v1","addr":"172.21.15.60:6835","nonce":1849151522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/523710346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":523710346},{"type":"v1","addr":"172.21.15.60:6837","nonce":523710346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/3033424631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":3033424631},{"type":"v1","addr":"172.21.15.90:6833","nonce":3033424631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/3454021291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3454021291},{"type":"v1","addr":"172.21.15.149:6833","nonce":3454021291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1057049585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1057049585},{"type":"v1","addr":"172.21.15.90:6835","nonce":1057049585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/882239924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":882239924},{"type":"v1","addr":"172.21.15.149:6835","nonce":882239924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/2243702844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":2243702844},{"type":"v1","addr":"172.21.15.90:6837","nonce":2243702844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/2611278690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2611278690},{"type":"v1","addr":"172.21.15.149:6837","nonce":2611278690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6839/2527622212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2527622212},{"type":"v1","addr":"172.21.15.149:6839","nonce":2527622212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-03T14:44:03.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:03.447+0000 7fea92c00700 1 -- 172.21.15.60:0/3821535763 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fea700b2880 msgr2=0x7fea700b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:03.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:03.447+0000 7fea92c00700 1 --2- 172.21.15.60:0/3821535763 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fea700b2880 0x7fea700b4d40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fea8c0789f0 tx=0x7fea7400b410 comp rx=0 tx=0).stop 2024-05-03T14:44:03.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:03.447+0000 7fea92c00700 1 -- 172.21.15.60:0/3821535763 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea8c109ad0 msgr2=0x7fea8c1a02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:03.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:03.447+0000 7fea92c00700 1 --2- 172.21.15.60:0/3821535763 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea8c109ad0 0x7fea8c1a02c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fea7c00d980 tx=0x7fea7c00dc90 comp rx=0 tx=0).stop 2024-05-03T14:44:03.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:03.447+0000 7fea92c00700 1 -- 172.21.15.60:0/3821535763 shutdown_connections 2024-05-03T14:44:03.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:03.447+0000 7fea92c00700 1 --2- 172.21.15.60:0/3821535763 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fea700b2880 0x7fea700b4d40 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:03.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:03.447+0000 7fea92c00700 1 --2- 172.21.15.60:0/3821535763 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea8c10fb60 0x7fea8c1a4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:03.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:03.447+0000 7fea92c00700 1 --2- 172.21.15.60:0/3821535763 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea8c109ad0 0x7fea8c1a02c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:03.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:03.447+0000 7fea92c00700 1 --2- 172.21.15.60:0/3821535763 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea8c10a490 0x7fea8c1a0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:03.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:03.447+0000 7fea92c00700 1 -- 172.21.15.60:0/3821535763 >> 172.21.15.60:0/3821535763 conn(0x7fea8c076c70 msgr2=0x7fea8c077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:44:03.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:03.447+0000 7fea92c00700 1 -- 172.21.15.60:0/3821535763 shutdown_connections 2024-05-03T14:44:03.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:03.447+0000 7fea92c00700 1 -- 172.21.15.60:0/3821535763 wait complete. 2024-05-03T14:44:03.450 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 10 2024-05-03T14:44:03.624 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:03 smithi090 ceph-mon[33568]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:03.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:03 smithi149 ceph-mon[33332]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:04.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:04 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/3821535763' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T14:44:04.624 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:04 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/3821535763' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T14:44:04.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:04 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/3821535763' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T14:44:05.624 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:05 smithi090 ceph-mon[33568]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:05.636 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:05 smithi060 ceph-mon[28873]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:05.637 INFO:journalctl@ceph.mds.g.smithi060.stdout:May 03 14:44:05 smithi060 podman[67837]: 2024-05-03 14:44:05.469736297 +0000 UTC m=+0.187041714 container create 596ec49ce14cdc814b880abb845f35f6de9bceb8d2fd194180ab565d2ad65279 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-g, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-05-03T14:44:05.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:05 smithi149 ceph-mon[33332]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:06.088 INFO:journalctl@ceph.mds.g.smithi060.stdout:May 03 14:44:05 smithi060 podman[67837]: 2024-05-03 14:44:05.737083671 +0000 UTC m=+0.454389089 container init 596ec49ce14cdc814b880abb845f35f6de9bceb8d2fd194180ab565d2ad65279 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-g, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-05-03T14:44:06.088 INFO:journalctl@ceph.mds.g.smithi060.stdout:May 03 14:44:05 smithi060 podman[67837]: 2024-05-03 14:44:05.747806529 +0000 UTC m=+0.465111906 container start 596ec49ce14cdc814b880abb845f35f6de9bceb8d2fd194180ab565d2ad65279 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-g, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2) 2024-05-03T14:44:06.088 INFO:journalctl@ceph.mds.g.smithi060.stdout:May 03 14:44:05 smithi060 ceph-mds[67856]: -- 172.21.15.60:0/479875063 <== mon.0 v2:172.21.15.60:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55601f5e9a40 con 0x55601f5bc800 2024-05-03T14:44:06.088 INFO:journalctl@ceph.mds.g.smithi060.stdout:May 03 14:44:05 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-g[67852]: starting mds.g at 2024-05-03T14:44:06.088 INFO:journalctl@ceph.mds.g.smithi060.stdout:May 03 14:44:05 smithi060 bash[67837]: 596ec49ce14cdc814b880abb845f35f6de9bceb8d2fd194180ab565d2ad65279 2024-05-03T14:44:06.088 INFO:journalctl@ceph.mds.g.smithi060.stdout:May 03 14:44:05 smithi060 systemd[1]: Started Ceph mds.g for 285d64b2-095a-11ef-bc97-c7b262605968. 2024-05-03T14:44:06.616 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T14:44:07.102 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:44:07.327 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:07 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:07.327 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:07 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:07.328 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:07 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:07.328 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:07 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T14:44:07.328 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:07 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T14:44:07.328 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:07 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:44:07.328 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:07 smithi060 ceph-mon[28873]: Deploying daemon mds.a on smithi060 2024-05-03T14:44:07.328 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:07 smithi060 ceph-mon[28873]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:07.328 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:07 smithi060 ceph-mon[28873]: mds.? [v2:172.21.15.60:6838/3846205424,v1:172.21.15.60:6839/3846205424] up:boot 2024-05-03T14:44:07.328 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:07 smithi060 ceph-mon[28873]: fsmap 10 up:standby 2024-05-03T14:44:07.328 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:07 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-03T14:44:07.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:07 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:07.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:07 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:07.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:07 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:07.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:07 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T14:44:07.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:07 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T14:44:07.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:07 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:44:07.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:07 smithi090 ceph-mon[33568]: Deploying daemon mds.a on smithi060 2024-05-03T14:44:07.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:07 smithi090 ceph-mon[33568]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:07.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:07 smithi090 ceph-mon[33568]: mds.? [v2:172.21.15.60:6838/3846205424,v1:172.21.15.60:6839/3846205424] up:boot 2024-05-03T14:44:07.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:07 smithi090 ceph-mon[33568]: fsmap 10 up:standby 2024-05-03T14:44:07.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:07 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-03T14:44:07.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:07 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:07.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:07 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:07.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:07 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:07.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:07 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T14:44:07.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:07 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T14:44:07.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:07 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:44:07.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:07 smithi149 ceph-mon[33332]: Deploying daemon mds.a on smithi060 2024-05-03T14:44:07.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:07 smithi149 ceph-mon[33332]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:07.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:07 smithi149 ceph-mon[33332]: mds.? [v2:172.21.15.60:6838/3846205424,v1:172.21.15.60:6839/3846205424] up:boot 2024-05-03T14:44:07.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:07 smithi149 ceph-mon[33332]: fsmap 10 up:standby 2024-05-03T14:44:07.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:07 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-03T14:44:09.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:09 smithi060 ceph-mon[28873]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:09.624 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:09 smithi090 ceph-mon[33568]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:09.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:09 smithi149 ceph-mon[33332]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:10.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:10.454+0000 7f05af200700 1 -- 172.21.15.60:0/3560271703 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05a8104630 msgr2=0x7f05a8104a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:10.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:10.454+0000 7f05af200700 1 --2- 172.21.15.60:0/3560271703 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05a8104630 0x7f05a8104a10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f059c009990 tx=0x7f059c009ca0 comp rx=0 tx=0).stop 2024-05-03T14:44:10.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:10.454+0000 7f05af200700 1 -- 172.21.15.60:0/3560271703 shutdown_connections 2024-05-03T14:44:10.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:10.454+0000 7f05af200700 1 --2- 172.21.15.60:0/3560271703 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f05a810da10 0x7f05a810fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:10.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:10.454+0000 7f05af200700 1 --2- 172.21.15.60:0/3560271703 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05a8104630 0x7f05a8104a10 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:10.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:10.454+0000 7f05af200700 1 --2- 172.21.15.60:0/3560271703 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f05a8104f50 0x7f05a810d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:10.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:10.454+0000 7f05af200700 1 -- 172.21.15.60:0/3560271703 >> 172.21.15.60:0/3560271703 conn(0x7f05a80fddc0 msgr2=0x7f05a81001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:44:10.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:10.455+0000 7f05af200700 1 -- 172.21.15.60:0/3560271703 shutdown_connections 2024-05-03T14:44:10.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:10.455+0000 7f05af200700 1 -- 172.21.15.60:0/3560271703 wait complete. 2024-05-03T14:44:10.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:10.456+0000 7f05af200700 1 Processor -- start 2024-05-03T14:44:10.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:10.456+0000 7f05af200700 1 -- start start 2024-05-03T14:44:10.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:10.456+0000 7f05af200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f05a8104630 0x7f05a8102290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:44:10.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:10.456+0000 7f05af200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f05a8104f50 0x7f05a81027d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:44:10.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:10.456+0000 7f05af200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05a810da10 0x7f05a8102d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:44:10.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:10.456+0000 7f05af200700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f05a8113870 con 0x7f05a810da10 2024-05-03T14:44:10.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:10.456+0000 7f05af200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f05a81136f0 con 0x7f05a8104630 2024-05-03T14:44:10.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:10.456+0000 7f05af200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f05a81139f0 con 0x7f05a8104f50 2024-05-03T14:44:10.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:10.457+0000 7f05ade00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f05a8104630 0x7f05a8102290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:44:10.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:10.457+0000 7f05ade00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f05a8104630 0x7f05a8102290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.60:36942/0 (socket says 172.21.15.60:36942) 2024-05-03T14:44:10.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:10.457+0000 7f05ade00700 1 -- 172.21.15.60:0/2009005533 learned_addr learned my addr 172.21.15.60:0/2009005533 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:44:10.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:10.457+0000 7f05ae800700 1 --2- 172.21.15.60:0/2009005533 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05a810da10 0x7f05a8102d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:44:10.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:10.457+0000 7f05ad400700 1 --2- 172.21.15.60:0/2009005533 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f05a8104f50 0x7f05a81027d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:44:10.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:10.457+0000 7f05ade00700 1 -- 172.21.15.60:0/2009005533 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f05a8104f50 msgr2=0x7f05a81027d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:10.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:10.457+0000 7f05ade00700 1 --2- 172.21.15.60:0/2009005533 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f05a8104f50 0x7f05a81027d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:10.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:10.457+0000 7f05ade00700 1 -- 172.21.15.60:0/2009005533 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05a810da10 msgr2=0x7f05a8102d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:10.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:10.457+0000 7f05ade00700 1 --2- 172.21.15.60:0/2009005533 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05a810da10 0x7f05a8102d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:10.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:10.457+0000 7f05ade00700 1 -- 172.21.15.60:0/2009005533 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05a81a81e0 con 0x7f05a8104630 2024-05-03T14:44:10.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:10.458+0000 7f05ade00700 1 --2- 172.21.15.60:0/2009005533 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f05a8104630 0x7f05a8102290 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f059c0095c0 tx=0x7f059c005590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:44:10.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:10.458+0000 7f05a2a00700 1 -- 172.21.15.60:0/2009005533 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059c026070 con 0x7f05a8104630 2024-05-03T14:44:10.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:10.458+0000 7f05af200700 1 -- 172.21.15.60:0/2009005533 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05a81a8470 con 0x7f05a8104630 2024-05-03T14:44:10.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:10.458+0000 7f05a2a00700 1 -- 172.21.15.60:0/2009005533 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f059c0056b0 con 0x7f05a8104630 2024-05-03T14:44:10.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:10.458+0000 7f05a2a00700 1 -- 172.21.15.60:0/2009005533 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059c0173f0 con 0x7f05a8104630 2024-05-03T14:44:10.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:10.458+0000 7f05af200700 1 -- 172.21.15.60:0/2009005533 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05a81a89a0 con 0x7f05a8104630 2024-05-03T14:44:10.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:10.459+0000 7f05af200700 1 -- 172.21.15.60:0/2009005533 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05a806a190 con 0x7f05a8104630 2024-05-03T14:44:10.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:10.462+0000 7f05a2a00700 1 -- 172.21.15.60:0/2009005533 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f059c029070 con 0x7f05a8104630 2024-05-03T14:44:10.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:10.463+0000 7f05a2a00700 1 --2- 172.21.15.60:0/2009005533 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f05900b2880 0x7f05900b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:44:10.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:10.463+0000 7f05a2a00700 1 -- 172.21.15.60:0/2009005533 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f059c0e2a40 con 0x7f05a8104630 2024-05-03T14:44:10.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:10.464+0000 7f05ad400700 1 --2- 172.21.15.60:0/2009005533 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f05900b2880 0x7f05900b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:44:10.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:10.464+0000 7f05ad400700 1 --2- 172.21.15.60:0/2009005533 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f05900b2880 0x7f05900b4d40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f05a8101990 tx=0x7f0598009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:44:10.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:10.468+0000 7f05a2a00700 1 -- 172.21.15.60:0/2009005533 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f059c013070 con 0x7f05a8104630 2024-05-03T14:44:10.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:10.780+0000 7f05af200700 1 -- 172.21.15.60:0/2009005533 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f05a8103770 con 0x7f05a8104630 2024-05-03T14:44:10.783 INFO:journalctl@ceph.mds.a.smithi060.stdout:May 03 14:44:10 smithi060 systemd[1]: Starting Ceph mds.a for 285d64b2-095a-11ef-bc97-c7b262605968... 2024-05-03T14:44:10.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:10.781+0000 7f05a2a00700 1 -- 172.21.15.60:0/2009005533 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8827 (secure 0 0 0) 0x7f059c0aabf0 con 0x7f05a8104630 2024-05-03T14:44:10.784 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:44:10.784 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1849151522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1849151522},{"type":"v1","addr":"172.21.15.60:6835","nonce":1849151522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/523710346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":523710346},{"type":"v1","addr":"172.21.15.60:6837","nonce":523710346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3846205424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3846205424},{"type":"v1","addr":"172.21.15.60:6839","nonce":3846205424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/3033424631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":3033424631},{"type":"v1","addr":"172.21.15.90:6833","nonce":3033424631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/3454021291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3454021291},{"type":"v1","addr":"172.21.15.149:6833","nonce":3454021291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1057049585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1057049585},{"type":"v1","addr":"172.21.15.90:6835","nonce":1057049585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/882239924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":882239924},{"type":"v1","addr":"172.21.15.149:6835","nonce":882239924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/2243702844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":2243702844},{"type":"v1","addr":"172.21.15.90:6837","nonce":2243702844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/2611278690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2611278690},{"type":"v1","addr":"172.21.15.149:6837","nonce":2611278690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6839/2527622212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2527622212},{"type":"v1","addr":"172.21.15.149:6839","nonce":2527622212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-03T14:44:10.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:10.786+0000 7f05af200700 1 -- 172.21.15.60:0/2009005533 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f05900b2880 msgr2=0x7f05900b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:10.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:10.786+0000 7f05af200700 1 --2- 172.21.15.60:0/2009005533 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f05900b2880 0x7f05900b4d40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f05a8101990 tx=0x7f0598009260 comp rx=0 tx=0).stop 2024-05-03T14:44:10.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:10.786+0000 7f05af200700 1 -- 172.21.15.60:0/2009005533 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f05a8104630 msgr2=0x7f05a8102290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:10.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:10.786+0000 7f05af200700 1 --2- 172.21.15.60:0/2009005533 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f05a8104630 0x7f05a8102290 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f059c0095c0 tx=0x7f059c005590 comp rx=0 tx=0).stop 2024-05-03T14:44:10.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:10.786+0000 7f05af200700 1 -- 172.21.15.60:0/2009005533 shutdown_connections 2024-05-03T14:44:10.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:10.786+0000 7f05af200700 1 --2- 172.21.15.60:0/2009005533 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f05900b2880 0x7f05900b4d40 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:10.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:10.786+0000 7f05af200700 1 --2- 172.21.15.60:0/2009005533 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05a810da10 0x7f05a8102d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:10.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:10.786+0000 7f05af200700 1 --2- 172.21.15.60:0/2009005533 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f05a8104630 0x7f05a8102290 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:10.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:10.786+0000 7f05af200700 1 --2- 172.21.15.60:0/2009005533 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f05a8104f50 0x7f05a81027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:10.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:10.786+0000 7f05af200700 1 -- 172.21.15.60:0/2009005533 >> 172.21.15.60:0/2009005533 conn(0x7f05a80fddc0 msgr2=0x7f05a80fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:44:10.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:10.786+0000 7f05af200700 1 -- 172.21.15.60:0/2009005533 shutdown_connections 2024-05-03T14:44:10.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:10.786+0000 7f05af200700 1 -- 172.21.15.60:0/2009005533 wait complete. 2024-05-03T14:44:10.790 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 11 2024-05-03T14:44:11.088 INFO:journalctl@ceph.mds.a.smithi060.stdout:May 03 14:44:10 smithi060 podman[68284]: 2024-05-03 14:44:10.787934727 +0000 UTC m=+0.035727198 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T14:44:11.088 INFO:journalctl@ceph.mds.a.smithi060.stdout:May 03 14:44:11 smithi060 podman[68284]: 2024-05-03 14:44:11.003003136 +0000 UTC m=+0.250795629 container create 4750ab8adb520a22152554b4c05a81b2e9c0ad385e86a86df3acd42deda7fcc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-a, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-05-03T14:44:11.452 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:11 smithi060 ceph-mon[28873]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:11.452 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:11 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/2009005533' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T14:44:11.624 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:11 smithi090 ceph-mon[33568]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:11.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:11 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/2009005533' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T14:44:11.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:11 smithi149 ceph-mon[33332]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:11.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:11 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/2009005533' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T14:44:11.820 INFO:journalctl@ceph.mds.a.smithi060.stdout:May 03 14:44:11 smithi060 podman[68284]: 2024-05-03 14:44:11.450342209 +0000 UTC m=+0.698134734 container init 4750ab8adb520a22152554b4c05a81b2e9c0ad385e86a86df3acd42deda7fcc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-a, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-03T14:44:11.820 INFO:journalctl@ceph.mds.a.smithi060.stdout:May 03 14:44:11 smithi060 podman[68284]: 2024-05-03 14:44:11.460920684 +0000 UTC m=+0.708713179 container start 4750ab8adb520a22152554b4c05a81b2e9c0ad385e86a86df3acd42deda7fcc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-a, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-03T14:44:11.820 INFO:journalctl@ceph.mds.a.smithi060.stdout:May 03 14:44:11 smithi060 ceph-mds[68315]: -- 172.21.15.60:0/3872172422 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56006e689a40 con 0x56006e65c800 2024-05-03T14:44:11.820 INFO:journalctl@ceph.mds.a.smithi060.stdout:May 03 14:44:11 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-a[68311]: starting mds.a at 2024-05-03T14:44:11.820 INFO:journalctl@ceph.mds.a.smithi060.stdout:May 03 14:44:11 smithi060 bash[68284]: 4750ab8adb520a22152554b4c05a81b2e9c0ad385e86a86df3acd42deda7fcc0 2024-05-03T14:44:11.820 INFO:journalctl@ceph.mds.a.smithi060.stdout:May 03 14:44:11 smithi060 systemd[1]: Started Ceph mds.a for 285d64b2-095a-11ef-bc97-c7b262605968. 2024-05-03T14:44:13.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:12 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:13.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:12 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:13.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:12 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:13.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:12 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T14:44:13.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:12 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T14:44:13.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:12 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:44:13.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:12 smithi060 ceph-mon[28873]: mds.? [v2:172.21.15.60:6840/924163182,v1:172.21.15.60:6841/924163182] up:boot 2024-05-03T14:44:13.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:12 smithi060 ceph-mon[28873]: fsmap 11 up:standby 2024-05-03T14:44:13.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:12 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-03T14:44:13.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:12 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:13.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:12 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:13.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:12 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:13.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:12 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T14:44:13.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:12 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T14:44:13.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:12 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:44:13.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:12 smithi090 ceph-mon[33568]: mds.? [v2:172.21.15.60:6840/924163182,v1:172.21.15.60:6841/924163182] up:boot 2024-05-03T14:44:13.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:12 smithi090 ceph-mon[33568]: fsmap 11 up:standby 2024-05-03T14:44:13.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:12 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-03T14:44:13.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:12 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:13.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:12 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:13.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:12 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:13.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:12 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T14:44:13.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:12 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T14:44:13.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:12 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:44:13.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:12 smithi149 ceph-mon[33332]: mds.? [v2:172.21.15.60:6840/924163182,v1:172.21.15.60:6841/924163182] up:boot 2024-05-03T14:44:13.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:12 smithi149 ceph-mon[33332]: fsmap 11 up:standby 2024-05-03T14:44:13.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:12 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-03T14:44:14.066 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:13 smithi090 ceph-mon[33568]: Deploying daemon mds.b on smithi090 2024-05-03T14:44:14.066 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:13 smithi090 ceph-mon[33568]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:14.067 INFO:journalctl@ceph.mds.b.smithi090.stdout:May 03 14:44:13 smithi090 systemd[1]: Starting Ceph mds.b for 285d64b2-095a-11ef-bc97-c7b262605968... 2024-05-03T14:44:14.067 INFO:journalctl@ceph.mds.b.smithi090.stdout:May 03 14:44:14 smithi090 podman[63716]: 2024-05-03 14:44:14.064765475 +0000 UTC m=+0.098245902 container create 0f91cd7392434763593d410402dcdff4871dc7077384f9ecb84a811defba4fec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-b, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T14:44:14.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:13 smithi060 ceph-mon[28873]: Deploying daemon mds.b on smithi090 2024-05-03T14:44:14.090 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:13 smithi060 ceph-mon[28873]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:14.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:13 smithi149 ceph-mon[33332]: Deploying daemon mds.b on smithi090 2024-05-03T14:44:14.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:13 smithi149 ceph-mon[33332]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:14.337 INFO:journalctl@ceph.mds.b.smithi090.stdout:May 03 14:44:14 smithi090 podman[63716]: 2024-05-03 14:44:13.984556254 +0000 UTC m=+0.018036694 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T14:44:14.337 INFO:journalctl@ceph.mds.b.smithi090.stdout:May 03 14:44:14 smithi090 podman[63716]: 2024-05-03 14:44:14.310717115 +0000 UTC m=+0.344197542 container init 0f91cd7392434763593d410402dcdff4871dc7077384f9ecb84a811defba4fec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True) 2024-05-03T14:44:14.337 INFO:journalctl@ceph.mds.b.smithi090.stdout:May 03 14:44:14 smithi090 podman[63716]: 2024-05-03 14:44:14.313192162 +0000 UTC m=+0.346672589 container start 0f91cd7392434763593d410402dcdff4871dc7077384f9ecb84a811defba4fec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-b, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-05-03T14:44:14.345 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T14:44:14.625 INFO:journalctl@ceph.mds.b.smithi090.stdout:May 03 14:44:14 smithi090 ceph-mds[63735]: -- 172.21.15.90:0/3592200939 <== mon.2 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56525fdafa40 con 0x56525fd82c00 2024-05-03T14:44:14.625 INFO:journalctl@ceph.mds.b.smithi090.stdout:May 03 14:44:14 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-b[63731]: starting mds.b at 2024-05-03T14:44:14.625 INFO:journalctl@ceph.mds.b.smithi090.stdout:May 03 14:44:14 smithi090 bash[63716]: 0f91cd7392434763593d410402dcdff4871dc7077384f9ecb84a811defba4fec 2024-05-03T14:44:14.625 INFO:journalctl@ceph.mds.b.smithi090.stdout:May 03 14:44:14 smithi090 systemd[1]: Started Ceph mds.b for 285d64b2-095a-11ef-bc97-c7b262605968. 2024-05-03T14:44:14.833 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:44:15.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:15 smithi060 ceph-mon[28873]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:15.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:15 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:15.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:15 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:15.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:15 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:15.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:15 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:15.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:15 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:15.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:15 smithi060 ceph-mon[28873]: mds.? [v2:172.21.15.90:6838/3512381172,v1:172.21.15.90:6839/3512381172] up:boot 2024-05-03T14:44:15.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:15 smithi060 ceph-mon[28873]: fsmap 12 up:standby 2024-05-03T14:44:15.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:15 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-03T14:44:15.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:15 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:44:15.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:15 smithi090 ceph-mon[33568]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:15.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:15 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:15.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:15 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:15.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:15 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:15.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:15 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:15.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:15 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:15.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:15 smithi090 ceph-mon[33568]: mds.? [v2:172.21.15.90:6838/3512381172,v1:172.21.15.90:6839/3512381172] up:boot 2024-05-03T14:44:15.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:15 smithi090 ceph-mon[33568]: fsmap 12 up:standby 2024-05-03T14:44:15.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:15 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-03T14:44:15.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:15 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:44:15.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:15 smithi149 ceph-mon[33332]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:15.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:15 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:15.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:15 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:15.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:15 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:15.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:15 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:15.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:15 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:15.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:15 smithi149 ceph-mon[33332]: mds.? [v2:172.21.15.90:6838/3512381172,v1:172.21.15.90:6839/3512381172] up:boot 2024-05-03T14:44:15.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:15 smithi149 ceph-mon[33332]: fsmap 12 up:standby 2024-05-03T14:44:15.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:15 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-03T14:44:15.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:15 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:44:17.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:17.142+0000 7f7418400700 1 -- 172.21.15.60:0/3515972307 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7410103ee0 msgr2=0x7f7410110370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:17.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:17.142+0000 7f7418400700 1 --2- 172.21.15.60:0/3515972307 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7410103ee0 0x7f7410110370 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7408009f90 tx=0x7f740800b330 comp rx=0 tx=0).stop 2024-05-03T14:44:17.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:17.142+0000 7f7418400700 1 -- 172.21.15.60:0/3515972307 shutdown_connections 2024-05-03T14:44:17.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:17.142+0000 7f7418400700 1 --2- 172.21.15.60:0/3515972307 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7410103ee0 0x7f7410110370 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:17.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:17.142+0000 7f7418400700 1 --2- 172.21.15.60:0/3515972307 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7410109530 0x7f7410109910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:17.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:17.142+0000 7f7418400700 1 --2- 172.21.15.60:0/3515972307 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7410103520 0x7f74101039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:17.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:17.142+0000 7f7418400700 1 -- 172.21.15.60:0/3515972307 >> 172.21.15.60:0/3515972307 conn(0x7f741007a570 msgr2=0x7f741007a980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:44:17.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:17.142+0000 7f7418400700 1 -- 172.21.15.60:0/3515972307 shutdown_connections 2024-05-03T14:44:17.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:17.142+0000 7f7418400700 1 -- 172.21.15.60:0/3515972307 wait complete. 2024-05-03T14:44:17.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:17.143+0000 7f7418400700 1 Processor -- start 2024-05-03T14:44:17.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:17.143+0000 7f7418400700 1 -- start start 2024-05-03T14:44:17.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:17.144+0000 7f7418400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7410103520 0x7f741019bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:44:17.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:17.144+0000 7f7418400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7410103ee0 0x7f741019c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:44:17.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:17.144+0000 7f7418400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7410109530 0x7f74101a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:44:17.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:17.144+0000 7f7418400700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f7410113940 con 0x7f7410103ee0 2024-05-03T14:44:17.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:17.144+0000 7f7418400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f74101137c0 con 0x7f7410109530 2024-05-03T14:44:17.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:17.144+0000 7f7418400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7410113ac0 con 0x7f7410103520 2024-05-03T14:44:17.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:17.144+0000 7f7415a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7410103520 0x7f741019bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:44:17.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:17.144+0000 7f7415000700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7410103ee0 0x7f741019c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:44:17.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:17.144+0000 7f7416400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7410109530 0x7f74101a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:44:17.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:17.144+0000 7f7415a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7410103520 0x7f741019bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.60:41972/0 (socket says 172.21.15.60:41972) 2024-05-03T14:44:17.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:17.144+0000 7f7415a00700 1 -- 172.21.15.60:0/471973137 learned_addr learned my addr 172.21.15.60:0/471973137 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:44:17.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:17.145+0000 7f7415a00700 1 -- 172.21.15.60:0/471973137 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7410109530 msgr2=0x7f74101a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:17.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:17.145+0000 7f7415a00700 1 --2- 172.21.15.60:0/471973137 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7410109530 0x7f74101a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:17.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:17.145+0000 7f7415a00700 1 -- 172.21.15.60:0/471973137 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7410103ee0 msgr2=0x7f741019c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:17.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:17.145+0000 7f7415a00700 1 --2- 172.21.15.60:0/471973137 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7410103ee0 0x7f741019c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:17.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:17.145+0000 7f7415a00700 1 -- 172.21.15.60:0/471973137 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74101a0e00 con 0x7f7410103520 2024-05-03T14:44:17.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:17.145+0000 7f7416400700 1 --2- 172.21.15.60:0/471973137 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7410109530 0x7f74101a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:44:17.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:17.145+0000 7f7415a00700 1 --2- 172.21.15.60:0/471973137 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7410103520 0x7f741019bdf0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f740400ec10 tx=0x7f740400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:44:17.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:17.145+0000 7f740e000700 1 -- 172.21.15.60:0/471973137 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740400cc90 con 0x7f7410103520 2024-05-03T14:44:17.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:17.145+0000 7f740e000700 1 -- 172.21.15.60:0/471973137 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f7404004520 con 0x7f7410103520 2024-05-03T14:44:17.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:17.145+0000 7f7418400700 1 -- 172.21.15.60:0/471973137 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74101a10f0 con 0x7f7410103520 2024-05-03T14:44:17.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:17.145+0000 7f740e000700 1 -- 172.21.15.60:0/471973137 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7404010690 con 0x7f7410103520 2024-05-03T14:44:17.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:17.145+0000 7f7418400700 1 -- 172.21.15.60:0/471973137 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74101a8970 con 0x7f7410103520 2024-05-03T14:44:17.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:17.146+0000 7f7418400700 1 -- 172.21.15.60:0/471973137 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f741006a190 con 0x7f7410103520 2024-05-03T14:44:17.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:17.149+0000 7f740e000700 1 -- 172.21.15.60:0/471973137 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f7404003f30 con 0x7f7410103520 2024-05-03T14:44:17.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:17.150+0000 7f740e000700 1 --2- 172.21.15.60:0/471973137 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f73f80b2880 0x7f73f80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:44:17.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:17.151+0000 7f740e000700 1 -- 172.21.15.60:0/471973137 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f74040e2100 con 0x7f7410103520 2024-05-03T14:44:17.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:17.151+0000 7f7415000700 1 --2- 172.21.15.60:0/471973137 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f73f80b2880 0x7f73f80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:44:17.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:17.152+0000 7f7415000700 1 --2- 172.21.15.60:0/471973137 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f73f80b2880 0x7f73f80b4d40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7410110730 tx=0x7f73fc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:44:17.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:17.152+0000 7f740e000700 1 -- 172.21.15.60:0/471973137 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f74040aa220 con 0x7f7410103520 2024-05-03T14:44:17.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:17.462+0000 7f7418400700 1 -- 172.21.15.60:0/471973137 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f741019d150 con 0x7f7410103520 2024-05-03T14:44:17.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:17.463+0000 7f740e000700 1 -- 172.21.15.60:0/471973137 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10476 (secure 0 0 0) 0x7f74040a9b20 con 0x7f7410103520 2024-05-03T14:44:17.466 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:44:17.466 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1849151522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1849151522},{"type":"v1","addr":"172.21.15.60:6835","nonce":1849151522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/523710346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":523710346},{"type":"v1","addr":"172.21.15.60:6837","nonce":523710346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3846205424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3846205424},{"type":"v1","addr":"172.21.15.60:6839","nonce":3846205424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/3033424631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":3033424631},{"type":"v1","addr":"172.21.15.90:6833","nonce":3033424631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/3454021291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3454021291},{"type":"v1","addr":"172.21.15.149:6833","nonce":3454021291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1057049585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1057049585},{"type":"v1","addr":"172.21.15.90:6835","nonce":1057049585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/882239924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":882239924},{"type":"v1","addr":"172.21.15.149:6835","nonce":882239924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/2243702844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":2243702844},{"type":"v1","addr":"172.21.15.90:6837","nonce":2243702844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/2611278690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2611278690},{"type":"v1","addr":"172.21.15.149:6837","nonce":2611278690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6839/2527622212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2527622212},{"type":"v1","addr":"172.21.15.149:6839","nonce":2527622212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24509,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6839/3512381172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3512381172},{"type":"v1","addr":"172.21.15.90:6839","nonce":3512381172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24511,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/924163182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":924163182},{"type":"v1","addr":"172.21.15.60:6841","nonce":924163182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[]} 2024-05-03T14:44:17.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:17.466+0000 7f7418400700 1 -- 172.21.15.60:0/471973137 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f73f80b2880 msgr2=0x7f73f80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:17.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:17.466+0000 7f7418400700 1 --2- 172.21.15.60:0/471973137 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f73f80b2880 0x7f73f80b4d40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7410110730 tx=0x7f73fc009260 comp rx=0 tx=0).stop 2024-05-03T14:44:17.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:17.466+0000 7f7418400700 1 -- 172.21.15.60:0/471973137 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7410103520 msgr2=0x7f741019bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:17.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:17.466+0000 7f7418400700 1 --2- 172.21.15.60:0/471973137 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7410103520 0x7f741019bdf0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f740400ec10 tx=0x7f740400ef20 comp rx=0 tx=0).stop 2024-05-03T14:44:17.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:17.466+0000 7f7418400700 1 -- 172.21.15.60:0/471973137 shutdown_connections 2024-05-03T14:44:17.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:17.466+0000 7f7418400700 1 --2- 172.21.15.60:0/471973137 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f73f80b2880 0x7f73f80b4d40 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:17.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:17.466+0000 7f7418400700 1 --2- 172.21.15.60:0/471973137 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7410109530 0x7f74101a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:17.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:17.466+0000 7f7418400700 1 --2- 172.21.15.60:0/471973137 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7410103520 0x7f741019bdf0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:17.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:17.466+0000 7f7418400700 1 --2- 172.21.15.60:0/471973137 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7410103ee0 0x7f741019c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:17.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:17.466+0000 7f7418400700 1 -- 172.21.15.60:0/471973137 >> 172.21.15.60:0/471973137 conn(0x7f741007a570 msgr2=0x7f741010e640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:44:17.470 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:17 smithi060 ceph-mon[28873]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:17.471 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:17 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:17.471 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:17 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:17.471 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:17 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:17.471 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:17 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:17.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:17.467+0000 7f7418400700 1 -- 172.21.15.60:0/471973137 shutdown_connections 2024-05-03T14:44:17.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:17.467+0000 7f7418400700 1 -- 172.21.15.60:0/471973137 wait complete. 2024-05-03T14:44:17.472 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 13 2024-05-03T14:44:17.726 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:17 smithi149 ceph-mon[33332]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:17.726 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:17 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:17.726 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:17 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:17.727 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:17 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:17.727 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:17 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:17.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:17 smithi090 ceph-mon[33568]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:17.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:17 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:17.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:17 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:17.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:17 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:17.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:17 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:18.672 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-05-03T14:44:18.739 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:18 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/471973137' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T14:44:18.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:18 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/471973137' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T14:44:18.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:18 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/471973137' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T14:44:19.118 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:44:19.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:19 smithi060 ceph-mon[28873]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:19.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:19 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:19.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:19 smithi090 ceph-mon[33568]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:19.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:19 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:19.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:19 smithi149 ceph-mon[33332]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:19.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:19 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:21.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:21.195+0000 7ff1ada00700 1 -- 172.21.15.60:0/3290955387 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1a8075ff0 msgr2=0x7ff1a810f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:21.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:21.195+0000 7ff1ada00700 1 --2- 172.21.15.60:0/3290955387 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1a8075ff0 0x7ff1a810f960 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff1940098f0 tx=0x7ff194009c00 comp rx=0 tx=0).stop 2024-05-03T14:44:21.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:21.195+0000 7ff1ada00700 1 -- 172.21.15.60:0/3290955387 shutdown_connections 2024-05-03T14:44:21.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:21.195+0000 7ff1ada00700 1 --2- 172.21.15.60:0/3290955387 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff1a810fea0 0x7ff1a8112290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:21.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:21.195+0000 7ff1ada00700 1 --2- 172.21.15.60:0/3290955387 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1a80756d0 0x7ff1a8075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:21.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:21.195+0000 7ff1ada00700 1 --2- 172.21.15.60:0/3290955387 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1a8075ff0 0x7ff1a810f960 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:21.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:21.195+0000 7ff1ada00700 1 -- 172.21.15.60:0/3290955387 >> 172.21.15.60:0/3290955387 conn(0x7ff1a807a600 msgr2=0x7ff1a807aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:44:21.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:21.195+0000 7ff1ada00700 1 -- 172.21.15.60:0/3290955387 shutdown_connections 2024-05-03T14:44:21.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:21.195+0000 7ff1ada00700 1 -- 172.21.15.60:0/3290955387 wait complete. 2024-05-03T14:44:21.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:21.197+0000 7ff1ada00700 1 Processor -- start 2024-05-03T14:44:21.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:21.197+0000 7ff1ada00700 1 -- start start 2024-05-03T14:44:21.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:21.197+0000 7ff1ada00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1a80756d0 0x7ff1a819c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:44:21.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:21.198+0000 7ff1ada00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff1a8075ff0 0x7ff1a819c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:44:21.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:21.198+0000 7ff1ada00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1a810fea0 0x7ff1a81a0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:44:21.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:21.198+0000 7ff1ada00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a8115b70 con 0x7ff1a80756d0 2024-05-03T14:44:21.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:21.198+0000 7ff1ada00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a81159f0 con 0x7ff1a8075ff0 2024-05-03T14:44:21.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:21.198+0000 7ff1ada00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a8115cf0 con 0x7ff1a810fea0 2024-05-03T14:44:21.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:21.198+0000 7ff1a6a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1a80756d0 0x7ff1a819c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:44:21.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:21.198+0000 7ff1a6000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff1a8075ff0 0x7ff1a819c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:44:21.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:21.198+0000 7ff1a6000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff1a8075ff0 0x7ff1a819c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.60:39692/0 (socket says 172.21.15.60:39692) 2024-05-03T14:44:21.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:21.198+0000 7ff1a6000700 1 -- 172.21.15.60:0/931857822 learned_addr learned my addr 172.21.15.60:0/931857822 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:44:21.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:21.198+0000 7ff1a6a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1a80756d0 0x7ff1a819c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58964/0 (socket says 172.21.15.60:58964) 2024-05-03T14:44:21.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:21.198+0000 7ff1a7400700 1 --2- 172.21.15.60:0/931857822 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1a810fea0 0x7ff1a81a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:44:21.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:21.198+0000 7ff1a7400700 1 -- 172.21.15.60:0/931857822 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff1a8075ff0 msgr2=0x7ff1a819c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:21.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:21.198+0000 7ff1a7400700 1 --2- 172.21.15.60:0/931857822 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff1a8075ff0 0x7ff1a819c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:21.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:21.198+0000 7ff1a7400700 1 -- 172.21.15.60:0/931857822 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1a80756d0 msgr2=0x7ff1a819c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:21.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:21.198+0000 7ff1a7400700 1 --2- 172.21.15.60:0/931857822 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1a80756d0 0x7ff1a819c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:21.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:21.198+0000 7ff1a7400700 1 -- 172.21.15.60:0/931857822 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1a81a1050 con 0x7ff1a810fea0 2024-05-03T14:44:21.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:21.199+0000 7ff1a6a00700 1 --2- 172.21.15.60:0/931857822 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1a80756d0 0x7ff1a819c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:44:21.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:21.199+0000 7ff1a6000700 1 --2- 172.21.15.60:0/931857822 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff1a8075ff0 0x7ff1a819c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:44:21.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:21.199+0000 7ff1a7400700 1 --2- 172.21.15.60:0/931857822 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1a810fea0 0x7ff1a81a0910 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff1a000ef90 tx=0x7ff1a000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:44:21.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:21.199+0000 7ff19f400700 1 -- 172.21.15.60:0/931857822 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a00092e0 con 0x7ff1a810fea0 2024-05-03T14:44:21.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:21.199+0000 7ff19f400700 1 -- 172.21.15.60:0/931857822 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7ff1a0004bb0 con 0x7ff1a810fea0 2024-05-03T14:44:21.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:21.199+0000 7ff1ada00700 1 -- 172.21.15.60:0/931857822 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1a81a12e0 con 0x7ff1a810fea0 2024-05-03T14:44:21.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:21.199+0000 7ff19f400700 1 -- 172.21.15.60:0/931857822 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a0004280 con 0x7ff1a810fea0 2024-05-03T14:44:21.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:21.199+0000 7ff1ada00700 1 -- 172.21.15.60:0/931857822 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1a81a8bc0 con 0x7ff1a810fea0 2024-05-03T14:44:21.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:21.200+0000 7ff1ada00700 1 -- 172.21.15.60:0/931857822 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1a806a190 con 0x7ff1a810fea0 2024-05-03T14:44:21.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:21.203+0000 7ff19f400700 1 -- 172.21.15.60:0/931857822 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7ff1a001b030 con 0x7ff1a810fea0 2024-05-03T14:44:21.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:21.205+0000 7ff19f400700 1 --2- 172.21.15.60:0/931857822 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7ff18c0b2900 0x7ff18c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:44:21.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:21.205+0000 7ff19f400700 1 -- 172.21.15.60:0/931857822 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7ff1a00e1810 con 0x7ff1a810fea0 2024-05-03T14:44:21.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:21.205+0000 7ff1a6a00700 1 --2- 172.21.15.60:0/931857822 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7ff18c0b2900 0x7ff18c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:44:21.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:21.206+0000 7ff1a6a00700 1 --2- 172.21.15.60:0/931857822 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7ff18c0b2900 0x7ff18c0b4dc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff198009750 tx=0x7ff198009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:44:21.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:21.208+0000 7ff19f400700 1 -- 172.21.15.60:0/931857822 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7ff1a0022020 con 0x7ff1a810fea0 2024-05-03T14:44:21.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:21 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:21.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:21 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:21.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:21 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:44:21.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:21 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:44:21.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:21 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:21.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:21 smithi060 ceph-mon[28873]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:21.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:21 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:44:21.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:21 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:21.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:21 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:21.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:21 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:44:21.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:21 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:44:21.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:21 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:21.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:21 smithi090 ceph-mon[33568]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:21.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:21 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:44:21.391 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:21 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:21.391 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:21 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:21.391 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:21 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:44:21.392 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:21 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:44:21.392 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:21 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:21.392 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:21 smithi149 ceph-mon[33332]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:21.392 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:21 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:44:21.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:21.457+0000 7ff1ada00700 1 -- 172.21.15.60:0/931857822 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7ff1a802e510 con 0x7ff1a810fea0 2024-05-03T14:44:21.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:21.457+0000 7ff19f400700 1 -- 172.21.15.60:0/931857822 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v71) ==== 84+0+65 (secure 0 0 0) 0x7ff1a0010040 con 0x7ff1a810fea0 2024-05-03T14:44:21.460 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:44:21.461 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2024-05-03T14:44:21.461 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-05-03T14:44:21.461 INFO:teuthology.orchestra.run.smithi060.stdout: "poolnum": 1, 2024-05-03T14:44:21.461 INFO:teuthology.orchestra.run.smithi060.stdout: "poolname": ".mgr" 2024-05-03T14:44:21.461 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-05-03T14:44:21.461 INFO:teuthology.orchestra.run.smithi060.stdout:] 2024-05-03T14:44:21.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:21.460+0000 7ff1ada00700 1 -- 172.21.15.60:0/931857822 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7ff18c0b2900 msgr2=0x7ff18c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:21.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:21.460+0000 7ff1ada00700 1 --2- 172.21.15.60:0/931857822 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7ff18c0b2900 0x7ff18c0b4dc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff198009750 tx=0x7ff198009310 comp rx=0 tx=0).stop 2024-05-03T14:44:21.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:21.460+0000 7ff1ada00700 1 -- 172.21.15.60:0/931857822 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1a810fea0 msgr2=0x7ff1a81a0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:21.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:21.460+0000 7ff1ada00700 1 --2- 172.21.15.60:0/931857822 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1a810fea0 0x7ff1a81a0910 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff1a000ef90 tx=0x7ff1a000c3a0 comp rx=0 tx=0).stop 2024-05-03T14:44:21.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:21.460+0000 7ff1ada00700 1 -- 172.21.15.60:0/931857822 shutdown_connections 2024-05-03T14:44:21.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:21.460+0000 7ff1ada00700 1 --2- 172.21.15.60:0/931857822 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7ff18c0b2900 0x7ff18c0b4dc0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:21.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:21.460+0000 7ff1ada00700 1 --2- 172.21.15.60:0/931857822 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1a810fea0 0x7ff1a81a0910 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:21.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:21.460+0000 7ff1ada00700 1 --2- 172.21.15.60:0/931857822 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1a80756d0 0x7ff1a819c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:21.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:21.460+0000 7ff1ada00700 1 --2- 172.21.15.60:0/931857822 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff1a8075ff0 0x7ff1a819c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:21.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:21.460+0000 7ff1ada00700 1 -- 172.21.15.60:0/931857822 >> 172.21.15.60:0/931857822 conn(0x7ff1a807a600 msgr2=0x7ff1a81006d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:44:21.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:21.460+0000 7ff1ada00700 1 -- 172.21.15.60:0/931857822 shutdown_connections 2024-05-03T14:44:21.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:21.461+0000 7ff1ada00700 1 -- 172.21.15.60:0/931857822 wait complete. 2024-05-03T14:44:22.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:22 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/931857822' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-03T14:44:22.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:22 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/931857822' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-03T14:44:22.600 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-03T14:44:22.600 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-03T14:44:22.624 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:22 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/931857822' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-03T14:44:23.097 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:44:23.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:23 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:23.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:23 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:23.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:23 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:23.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:23 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:23.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:23 smithi149 ceph-mon[33332]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:23.436 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:23 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:23.436 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:23 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:23.436 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:23 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:23.436 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:23 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:23.436 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:23 smithi060 ceph-mon[28873]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:23.624 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:23 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:23.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:23 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:23.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:23 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:23.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:23 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:23.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:23 smithi090 ceph-mon[33568]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:25.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:25.337+0000 7f788ac00700 1 -- 172.21.15.60:0/2553475247 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7884104160 msgr2=0x7f7884108150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:25.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:25.337+0000 7f788ac00700 1 --2- 172.21.15.60:0/2553475247 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7884104160 0x7f7884108150 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f78700098f0 tx=0x7f7870009c00 comp rx=0 tx=0).stop 2024-05-03T14:44:25.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:25.338+0000 7f788ac00700 1 -- 172.21.15.60:0/2553475247 shutdown_connections 2024-05-03T14:44:25.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:25.338+0000 7f788ac00700 1 --2- 172.21.15.60:0/2553475247 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7884108880 0x7f7884110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:25.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:25.338+0000 7f788ac00700 1 --2- 172.21.15.60:0/2553475247 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78841037b0 0x7f7884103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:25.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:25.338+0000 7f788ac00700 1 --2- 172.21.15.60:0/2553475247 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7884104160 0x7f7884108150 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:25.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:25.338+0000 7f788ac00700 1 -- 172.21.15.60:0/2553475247 >> 172.21.15.60:0/2553475247 conn(0x7f788407a580 msgr2=0x7f788407a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:44:25.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:25.338+0000 7f788ac00700 1 -- 172.21.15.60:0/2553475247 shutdown_connections 2024-05-03T14:44:25.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:25.338+0000 7f788ac00700 1 -- 172.21.15.60:0/2553475247 wait complete. 2024-05-03T14:44:25.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:25.339+0000 7f788ac00700 1 Processor -- start 2024-05-03T14:44:25.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:25.339+0000 7f788ac00700 1 -- start start 2024-05-03T14:44:25.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:25.340+0000 7f788ac00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78841037b0 0x7f788419bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:44:25.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:25.340+0000 7f788ac00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7884104160 0x7f788419c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:44:25.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:25.340+0000 7f788ac00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7884108880 0x7f78841a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:44:25.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:25.340+0000 7f788ac00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f7884113940 con 0x7f78841037b0 2024-05-03T14:44:25.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:25.340+0000 7f788ac00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f78841137c0 con 0x7f7884108880 2024-05-03T14:44:25.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:25.340+0000 7f788ac00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7884113ac0 con 0x7f7884104160 2024-05-03T14:44:25.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:25.340+0000 7f7883e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78841037b0 0x7f788419bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:44:25.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:25.340+0000 7f7883e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78841037b0 0x7f788419bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40766/0 (socket says 172.21.15.60:40766) 2024-05-03T14:44:25.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:25.340+0000 7f7883e00700 1 -- 172.21.15.60:0/462281092 learned_addr learned my addr 172.21.15.60:0/462281092 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:44:25.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:25.341+0000 7f7888c00700 1 --2- 172.21.15.60:0/462281092 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7884108880 0x7f78841a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:44:25.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:25.341+0000 7f7883400700 1 --2- 172.21.15.60:0/462281092 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7884104160 0x7f788419c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:44:25.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:25.341+0000 7f7883e00700 1 -- 172.21.15.60:0/462281092 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7884104160 msgr2=0x7f788419c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:25.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:25.341+0000 7f7883e00700 1 --2- 172.21.15.60:0/462281092 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7884104160 0x7f788419c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:25.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:25.341+0000 7f7883e00700 1 -- 172.21.15.60:0/462281092 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7884108880 msgr2=0x7f78841a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:25.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:25.341+0000 7f7883e00700 1 --2- 172.21.15.60:0/462281092 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7884108880 0x7f78841a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:25.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:25.341+0000 7f7883e00700 1 -- 172.21.15.60:0/462281092 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78841a0f60 con 0x7f78841037b0 2024-05-03T14:44:25.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:25.341+0000 7f7883e00700 1 --2- 172.21.15.60:0/462281092 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78841037b0 0x7f788419bfa0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f787400e9c0 tx=0x7f787400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:44:25.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:25.342+0000 7f7880c00700 1 -- 172.21.15.60:0/462281092 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f787400cbd0 con 0x7f78841037b0 2024-05-03T14:44:25.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:25.342+0000 7f788ac00700 1 -- 172.21.15.60:0/462281092 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78841a1250 con 0x7f78841037b0 2024-05-03T14:44:25.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:25.342+0000 7f7880c00700 1 -- 172.21.15.60:0/462281092 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f787400ee60 con 0x7f78841037b0 2024-05-03T14:44:25.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:25.342+0000 7f7880c00700 1 -- 172.21.15.60:0/462281092 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7874010430 con 0x7f78841037b0 2024-05-03T14:44:25.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:25.342+0000 7f788ac00700 1 -- 172.21.15.60:0/462281092 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78841a8ad0 con 0x7f78841037b0 2024-05-03T14:44:25.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:25.343+0000 7f788ac00700 1 -- 172.21.15.60:0/462281092 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f788406a190 con 0x7f78841037b0 2024-05-03T14:44:25.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:25.345+0000 7f7880c00700 1 -- 172.21.15.60:0/462281092 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f7874004750 con 0x7f78841037b0 2024-05-03T14:44:25.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:25.348+0000 7f7880c00700 1 --2- 172.21.15.60:0/462281092 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f78680b2880 0x7f78680b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:44:25.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:25.348+0000 7f7880c00700 1 -- 172.21.15.60:0/462281092 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f78740e22b0 con 0x7f78841037b0 2024-05-03T14:44:25.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:25.348+0000 7f7883400700 1 --2- 172.21.15.60:0/462281092 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f78680b2880 0x7f78680b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:44:25.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:25.349+0000 7f7883400700 1 --2- 172.21.15.60:0/462281092 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f78680b2880 0x7f78680b4d40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7870009810 tx=0x7f7870019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:44:25.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:25.351+0000 7f7880c00700 1 -- 172.21.15.60:0/462281092 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f7874054d70 con 0x7f78841037b0 2024-05-03T14:44:25.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:25 smithi060 ceph-mon[28873]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:25.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:25 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:25.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:25 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:25.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:25 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:44:25.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:25 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:44:25.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:25 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:25.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:25.591+0000 7f788ac00700 1 -- 172.21.15.60:0/462281092 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f78841a8d00 con 0x7f78841037b0 2024-05-03T14:44:25.624 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:25 smithi090 ceph-mon[33568]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:25.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:25 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:25.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:25 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:25.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:25 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:44:25.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:25 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:44:25.625 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:25 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:25.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:25 smithi149 ceph-mon[33332]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:25.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:25 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:25.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:25 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:25.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:25 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:44:25.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:25 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:44:25.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:25 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:44:26.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:26.373+0000 7f7880c00700 1 -- 172.21.15.60:0/462281092 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v72) ==== 138+0+0 (secure 0 0 0) 0x7f78740aa3d0 con 0x7f78841037b0 2024-05-03T14:44:26.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:26.381+0000 7f788ac00700 1 -- 172.21.15.60:0/462281092 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f78680b2880 msgr2=0x7f78680b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:26.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:26.381+0000 7f788ac00700 1 --2- 172.21.15.60:0/462281092 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f78680b2880 0x7f78680b4d40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7870009810 tx=0x7f7870019040 comp rx=0 tx=0).stop 2024-05-03T14:44:26.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:26.381+0000 7f788ac00700 1 -- 172.21.15.60:0/462281092 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78841037b0 msgr2=0x7f788419bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:26.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:26.381+0000 7f788ac00700 1 --2- 172.21.15.60:0/462281092 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78841037b0 0x7f788419bfa0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f787400e9c0 tx=0x7f787400ecd0 comp rx=0 tx=0).stop 2024-05-03T14:44:26.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:26.384+0000 7f788ac00700 1 -- 172.21.15.60:0/462281092 shutdown_connections 2024-05-03T14:44:26.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:26.384+0000 7f788ac00700 1 --2- 172.21.15.60:0/462281092 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f78680b2880 0x7f78680b4d40 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:26.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:26.384+0000 7f788ac00700 1 --2- 172.21.15.60:0/462281092 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7884108880 0x7f78841a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:26.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:26.384+0000 7f788ac00700 1 --2- 172.21.15.60:0/462281092 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78841037b0 0x7f788419bfa0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:26.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:26.384+0000 7f788ac00700 1 --2- 172.21.15.60:0/462281092 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7884104160 0x7f788419c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:26.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:26.384+0000 7f788ac00700 1 -- 172.21.15.60:0/462281092 >> 172.21.15.60:0/462281092 conn(0x7f788407a580 msgr2=0x7f7884101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:44:26.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:26.385+0000 7f788ac00700 1 -- 172.21.15.60:0/462281092 shutdown_connections 2024-05-03T14:44:26.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:26.385+0000 7f788ac00700 1 -- 172.21.15.60:0/462281092 wait complete. 2024-05-03T14:44:26.391 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' created 2024-05-03T14:44:26.624 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:26 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/462281092' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-03T14:44:26.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:26 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/462281092' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-03T14:44:26.760 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:26 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/462281092' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-03T14:44:27.265 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-05-03T14:44:27.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:27 smithi060 ceph-mon[28873]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:27.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:27 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/462281092' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-03T14:44:27.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:27 smithi060 ceph-mon[28873]: osdmap e72: 12 total, 12 up, 12 in 2024-05-03T14:44:27.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:27 smithi149 ceph-mon[33332]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:27.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:27 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/462281092' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-03T14:44:27.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:27 smithi149 ceph-mon[33332]: osdmap e72: 12 total, 12 up, 12 in 2024-05-03T14:44:27.742 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:44:27.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:27 smithi090 ceph-mon[33568]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:27.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:27 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/462281092' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-03T14:44:27.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:27 smithi090 ceph-mon[33568]: osdmap e72: 12 total, 12 up, 12 in 2024-05-03T14:44:28.660 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:28 smithi060 ceph-mon[28873]: osdmap e73: 12 total, 12 up, 12 in 2024-05-03T14:44:28.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:28 smithi149 ceph-mon[33332]: osdmap e73: 12 total, 12 up, 12 in 2024-05-03T14:44:28.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:28 smithi090 ceph-mon[33568]: osdmap e73: 12 total, 12 up, 12 in 2024-05-03T14:44:29.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:29 smithi149 ceph-mon[33332]: pgmap v315: 33 pgs: 2 creating+peering, 4 active+clean, 27 unknown; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:29.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:29 smithi149 ceph-mon[33332]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-03T14:44:29.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:29 smithi149 ceph-mon[33332]: osdmap e74: 12 total, 12 up, 12 in 2024-05-03T14:44:29.700 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:29 smithi060 ceph-mon[28873]: pgmap v315: 33 pgs: 2 creating+peering, 4 active+clean, 27 unknown; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:29.700 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:29 smithi060 ceph-mon[28873]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-03T14:44:29.700 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:29 smithi060 ceph-mon[28873]: osdmap e74: 12 total, 12 up, 12 in 2024-05-03T14:44:29.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:29 smithi090 ceph-mon[33568]: pgmap v315: 33 pgs: 2 creating+peering, 4 active+clean, 27 unknown; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:29.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:29 smithi090 ceph-mon[33568]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-03T14:44:29.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:29 smithi090 ceph-mon[33568]: osdmap e74: 12 total, 12 up, 12 in 2024-05-03T14:44:29.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:29.883+0000 7f5aeea00700 1 -- 172.21.15.60:0/3474086768 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ae8103520 msgr2=0x7f5ae81039a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:29.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:29.883+0000 7f5aeea00700 1 --2- 172.21.15.60:0/3474086768 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ae8103520 0x7f5ae81039a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5ad0009990 tx=0x7f5ad0009ca0 comp rx=0 tx=0).stop 2024-05-03T14:44:29.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:29.883+0000 7f5aeea00700 1 -- 172.21.15.60:0/3474086768 shutdown_connections 2024-05-03T14:44:29.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:29.883+0000 7f5aeea00700 1 --2- 172.21.15.60:0/3474086768 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ae8103ee0 0x7f5ae8110370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:29.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:29.883+0000 7f5aeea00700 1 --2- 172.21.15.60:0/3474086768 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ae8109530 0x7f5ae8109910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:29.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:29.883+0000 7f5aeea00700 1 --2- 172.21.15.60:0/3474086768 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ae8103520 0x7f5ae81039a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:29.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:29.883+0000 7f5aeea00700 1 -- 172.21.15.60:0/3474086768 >> 172.21.15.60:0/3474086768 conn(0x7f5ae807a570 msgr2=0x7f5ae807a980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:44:29.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:29.883+0000 7f5aeea00700 1 -- 172.21.15.60:0/3474086768 shutdown_connections 2024-05-03T14:44:29.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:29.883+0000 7f5aeea00700 1 -- 172.21.15.60:0/3474086768 wait complete. 2024-05-03T14:44:29.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:29.884+0000 7f5aeea00700 1 Processor -- start 2024-05-03T14:44:29.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:29.885+0000 7f5aeea00700 1 -- start start 2024-05-03T14:44:29.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:29.885+0000 7f5aeea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ae8103520 0x7f5ae819bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:44:29.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:29.885+0000 7f5aeea00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ae8103ee0 0x7f5ae819c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:44:29.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:29.885+0000 7f5aeea00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ae8109530 0x7f5ae81a07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:44:29.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:29.885+0000 7f5aeea00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae81138f0 con 0x7f5ae8109530 2024-05-03T14:44:29.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:29.885+0000 7f5aeea00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae8113770 con 0x7f5ae8103520 2024-05-03T14:44:29.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:29.885+0000 7f5aeea00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae8113a70 con 0x7f5ae8103ee0 2024-05-03T14:44:29.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:29.885+0000 7f5ae7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ae8103520 0x7f5ae819bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:44:29.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:29.885+0000 7f5ae7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ae8103520 0x7f5ae819bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.60:37986/0 (socket says 172.21.15.60:37986) 2024-05-03T14:44:29.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:29.885+0000 7f5ae7e00700 1 -- 172.21.15.60:0/3437858863 learned_addr learned my addr 172.21.15.60:0/3437858863 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:44:29.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:29.885+0000 7f5aeca00700 1 --2- 172.21.15.60:0/3437858863 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ae8109530 0x7f5ae81a07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:44:29.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:29.886+0000 7f5adf400700 1 --2- 172.21.15.60:0/3437858863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ae8103ee0 0x7f5ae819c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:44:29.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:29.886+0000 7f5ae7e00700 1 -- 172.21.15.60:0/3437858863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ae8103ee0 msgr2=0x7f5ae819c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:29.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:29.886+0000 7f5ae7e00700 1 --2- 172.21.15.60:0/3437858863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ae8103ee0 0x7f5ae819c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:29.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:29.886+0000 7f5ae7e00700 1 -- 172.21.15.60:0/3437858863 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ae8109530 msgr2=0x7f5ae81a07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:29.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:29.886+0000 7f5ae7e00700 1 --2- 172.21.15.60:0/3437858863 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ae8109530 0x7f5ae81a07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:29.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:29.886+0000 7f5ae7e00700 1 -- 172.21.15.60:0/3437858863 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ae81a0f10 con 0x7f5ae8103520 2024-05-03T14:44:29.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:29.886+0000 7f5adf400700 1 --2- 172.21.15.60:0/3437858863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ae8103ee0 0x7f5ae819c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T14:44:29.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:29.886+0000 7f5ae7e00700 1 --2- 172.21.15.60:0/3437858863 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ae8103520 0x7f5ae819bf00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5ad800e9a0 tx=0x7f5ad800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:44:29.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:29.887+0000 7f5ae5600700 1 -- 172.21.15.60:0/3437858863 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad800cbd0 con 0x7f5ae8103520 2024-05-03T14:44:29.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:29.887+0000 7f5ae5600700 1 -- 172.21.15.60:0/3437858863 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f5ad800cd70 con 0x7f5ae8103520 2024-05-03T14:44:29.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:29.887+0000 7f5aeea00700 1 -- 172.21.15.60:0/3437858863 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ae81a1200 con 0x7f5ae8103520 2024-05-03T14:44:29.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:29.887+0000 7f5ae5600700 1 -- 172.21.15.60:0/3437858863 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad8018820 con 0x7f5ae8103520 2024-05-03T14:44:29.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:29.887+0000 7f5aeea00700 1 -- 172.21.15.60:0/3437858863 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ae81a8a80 con 0x7f5ae8103520 2024-05-03T14:44:29.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:29.888+0000 7f5aeea00700 1 -- 172.21.15.60:0/3437858863 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ae806a190 con 0x7f5ae8103520 2024-05-03T14:44:29.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:29.890+0000 7f5ae5600700 1 -- 172.21.15.60:0/3437858863 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f5ad8010430 con 0x7f5ae8103520 2024-05-03T14:44:29.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:29.892+0000 7f5ae5600700 1 --2- 172.21.15.60:0/3437858863 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f5acc0b2880 0x7f5acc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:44:29.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:29.892+0000 7f5ae5600700 1 -- 172.21.15.60:0/3437858863 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7497+0+0 (secure 0 0 0) 0x7f5ad80e2150 con 0x7f5ae8103520 2024-05-03T14:44:29.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:29.892+0000 7f5adf400700 1 --2- 172.21.15.60:0/3437858863 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f5acc0b2880 0x7f5acc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:44:29.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:29.893+0000 7f5adf400700 1 --2- 172.21.15.60:0/3437858863 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f5acc0b2880 0x7f5acc0b4d40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5ad0006010 tx=0x7f5ad001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:44:29.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:29.896+0000 7f5ae5600700 1 -- 172.21.15.60:0/3437858863 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f5ad80aa100 con 0x7f5ae8103520 2024-05-03T14:44:30.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:30.134+0000 7f5aeea00700 1 -- 172.21.15.60:0/3437858863 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f5ae81a8c50 con 0x7f5ae8103520 2024-05-03T14:44:30.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:30.426+0000 7f5ae5600700 1 -- 172.21.15.60:0/3437858863 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v75) ==== 171+0+0 (secure 0 0 0) 0x7f5ad80b2180 con 0x7f5ae8103520 2024-05-03T14:44:30.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:30.436+0000 7f5aeea00700 1 -- 172.21.15.60:0/3437858863 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f5acc0b2880 msgr2=0x7f5acc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:30.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:30.436+0000 7f5aeea00700 1 --2- 172.21.15.60:0/3437858863 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f5acc0b2880 0x7f5acc0b4d40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5ad0006010 tx=0x7f5ad001a040 comp rx=0 tx=0).stop 2024-05-03T14:44:30.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:30.436+0000 7f5aeea00700 1 -- 172.21.15.60:0/3437858863 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ae8103520 msgr2=0x7f5ae819bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:30.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:30.436+0000 7f5aeea00700 1 --2- 172.21.15.60:0/3437858863 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ae8103520 0x7f5ae819bf00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5ad800e9a0 tx=0x7f5ad800ecb0 comp rx=0 tx=0).stop 2024-05-03T14:44:30.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:30.437+0000 7f5aeea00700 1 -- 172.21.15.60:0/3437858863 shutdown_connections 2024-05-03T14:44:30.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:30.437+0000 7f5aeea00700 1 --2- 172.21.15.60:0/3437858863 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f5acc0b2880 0x7f5acc0b4d40 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:30.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:30.437+0000 7f5aeea00700 1 --2- 172.21.15.60:0/3437858863 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ae8109530 0x7f5ae81a07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:30.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:30.437+0000 7f5aeea00700 1 --2- 172.21.15.60:0/3437858863 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ae8103520 0x7f5ae819bf00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:30.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:30.437+0000 7f5aeea00700 1 --2- 172.21.15.60:0/3437858863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ae8103ee0 0x7f5ae819c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:30.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:30.437+0000 7f5aeea00700 1 -- 172.21.15.60:0/3437858863 >> 172.21.15.60:0/3437858863 conn(0x7f5ae807a570 msgr2=0x7f5ae8105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:44:30.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:30.440+0000 7f5aeea00700 1 -- 172.21.15.60:0/3437858863 shutdown_connections 2024-05-03T14:44:30.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:30.440+0000 7f5aeea00700 1 -- 172.21.15.60:0/3437858863 wait complete. 2024-05-03T14:44:30.447 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' created 2024-05-03T14:44:30.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:30 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/3437858863' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-03T14:44:30.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:30 smithi149 ceph-mon[33332]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-03T14:44:30.817 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:30 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/3437858863' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-03T14:44:30.818 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:30 smithi060 ceph-mon[28873]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-03T14:44:30.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:30 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/3437858863' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-03T14:44:30.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:30 smithi090 ceph-mon[33568]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-03T14:44:31.310 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-03T14:44:31.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:31 smithi060 ceph-mon[28873]: pgmap v317: 33 pgs: 2 creating+peering, 19 active+clean, 12 unknown; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:31.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:31 smithi060 ceph-mon[28873]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-03T14:44:31.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:31 smithi060 ceph-mon[28873]: osdmap e75: 12 total, 12 up, 12 in 2024-05-03T14:44:31.801 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:44:31.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:31 smithi090 ceph-mon[33568]: pgmap v317: 33 pgs: 2 creating+peering, 19 active+clean, 12 unknown; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:31.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:31 smithi090 ceph-mon[33568]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-03T14:44:31.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:31 smithi090 ceph-mon[33568]: osdmap e75: 12 total, 12 up, 12 in 2024-05-03T14:44:31.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:31 smithi149 ceph-mon[33332]: pgmap v317: 33 pgs: 2 creating+peering, 19 active+clean, 12 unknown; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:31.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:31 smithi149 ceph-mon[33332]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-03T14:44:31.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:31 smithi149 ceph-mon[33332]: osdmap e75: 12 total, 12 up, 12 in 2024-05-03T14:44:32.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:32 smithi060 ceph-mon[28873]: osdmap e76: 12 total, 12 up, 12 in 2024-05-03T14:44:32.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:32 smithi090 ceph-mon[33568]: osdmap e76: 12 total, 12 up, 12 in 2024-05-03T14:44:32.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:32 smithi149 ceph-mon[33332]: osdmap e76: 12 total, 12 up, 12 in 2024-05-03T14:44:33.629 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:33 smithi060 ceph-mon[28873]: pgmap v320: 97 pgs: 13 creating+peering, 31 active+clean, 53 unknown; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:33.629 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:33 smithi060 ceph-mon[28873]: osdmap e77: 12 total, 12 up, 12 in 2024-05-03T14:44:33.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:33 smithi090 ceph-mon[33568]: pgmap v320: 97 pgs: 13 creating+peering, 31 active+clean, 53 unknown; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:33.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:33 smithi090 ceph-mon[33568]: osdmap e77: 12 total, 12 up, 12 in 2024-05-03T14:44:33.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:33.917+0000 7fb804e00700 1 -- 172.21.15.60:0/3862862341 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb80006c0b0 msgr2=0x7fb80006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:33.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:33.917+0000 7fb804e00700 1 --2- 172.21.15.60:0/3862862341 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb80006c0b0 0x7fb80006c510 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8009990 tx=0x7fb7e8009ca0 comp rx=0 tx=0).stop 2024-05-03T14:44:33.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:33.917+0000 7fb804e00700 1 -- 172.21.15.60:0/3862862341 shutdown_connections 2024-05-03T14:44:33.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:33.917+0000 7fb804e00700 1 --2- 172.21.15.60:0/3862862341 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb80010fdf0 0x7fb8001121e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:33.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:33.917+0000 7fb804e00700 1 --2- 172.21.15.60:0/3862862341 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8001043b0 0x7fb80006bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:33.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:33.917+0000 7fb804e00700 1 --2- 172.21.15.60:0/3862862341 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb80006c0b0 0x7fb80006c510 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:33.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:33.917+0000 7fb804e00700 1 -- 172.21.15.60:0/3862862341 >> 172.21.15.60:0/3862862341 conn(0x7fb800079350 msgr2=0x7fb800079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:44:33.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:33.917+0000 7fb804e00700 1 -- 172.21.15.60:0/3862862341 shutdown_connections 2024-05-03T14:44:33.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:33.917+0000 7fb804e00700 1 -- 172.21.15.60:0/3862862341 wait complete. 2024-05-03T14:44:33.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:33.919+0000 7fb804e00700 1 Processor -- start 2024-05-03T14:44:33.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:33.920+0000 7fb804e00700 1 -- start start 2024-05-03T14:44:33.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:33.920+0000 7fb804e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb80006c0b0 0x7fb80019bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:44:33.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:33.920+0000 7fb804e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8001043b0 0x7fb80019c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:44:33.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:33.920+0000 7fb804e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb80010fdf0 0x7fb8001a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:44:33.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:33.920+0000 7fb804e00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fb800115b00 con 0x7fb80006c0b0 2024-05-03T14:44:33.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:33.920+0000 7fb804e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb800115980 con 0x7fb8001043b0 2024-05-03T14:44:33.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:33.920+0000 7fb804e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb800115c80 con 0x7fb80010fdf0 2024-05-03T14:44:33.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:33 smithi149 ceph-mon[33332]: pgmap v320: 97 pgs: 13 creating+peering, 31 active+clean, 53 unknown; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:33.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:33 smithi149 ceph-mon[33332]: osdmap e77: 12 total, 12 up, 12 in 2024-05-03T14:44:33.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:33.921+0000 7fb7f7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8001043b0 0x7fb80019c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:44:33.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:33.921+0000 7fb7fde00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb80006c0b0 0x7fb80019bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:44:33.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:33.921+0000 7fb7f7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8001043b0 0x7fb80019c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.60:45024/0 (socket says 172.21.15.60:45024) 2024-05-03T14:44:33.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:33.921+0000 7fb7f7e00700 1 -- 172.21.15.60:0/1453159578 learned_addr learned my addr 172.21.15.60:0/1453159578 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:44:33.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:33.921+0000 7fb7fe800700 1 --2- 172.21.15.60:0/1453159578 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb80010fdf0 0x7fb8001a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:44:33.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:33.921+0000 7fb7fde00700 1 -- 172.21.15.60:0/1453159578 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb80010fdf0 msgr2=0x7fb8001a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:33.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:33.921+0000 7fb7fde00700 1 --2- 172.21.15.60:0/1453159578 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb80010fdf0 0x7fb8001a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:33.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:33.921+0000 7fb7fde00700 1 -- 172.21.15.60:0/1453159578 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8001043b0 msgr2=0x7fb80019c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:33.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:33.921+0000 7fb7fde00700 1 --2- 172.21.15.60:0/1453159578 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8001043b0 0x7fb80019c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:33.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:33.921+0000 7fb7fde00700 1 -- 172.21.15.60:0/1453159578 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8001a0f60 con 0x7fb80006c0b0 2024-05-03T14:44:33.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:33.922+0000 7fb7fde00700 1 --2- 172.21.15.60:0/1453159578 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb80006c0b0 0x7fb80019bf50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb7f000e9f0 tx=0x7fb7f000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:44:33.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:33.922+0000 7fb7f6a00700 1 -- 172.21.15.60:0/1453159578 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f000c9c0 con 0x7fb80006c0b0 2024-05-03T14:44:33.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:33.922+0000 7fb804e00700 1 -- 172.21.15.60:0/1453159578 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8001a1250 con 0x7fb80006c0b0 2024-05-03T14:44:33.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:33.922+0000 7fb7f6a00700 1 -- 172.21.15.60:0/1453159578 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7fb7f000cb60 con 0x7fb80006c0b0 2024-05-03T14:44:33.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:33.922+0000 7fb7f6a00700 1 -- 172.21.15.60:0/1453159578 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f0018660 con 0x7fb80006c0b0 2024-05-03T14:44:33.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:33.922+0000 7fb804e00700 1 -- 172.21.15.60:0/1453159578 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8001a8ad0 con 0x7fb80006c0b0 2024-05-03T14:44:33.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:33.923+0000 7fb804e00700 1 -- 172.21.15.60:0/1453159578 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8001029e0 con 0x7fb80006c0b0 2024-05-03T14:44:33.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:33.927+0000 7fb7f6a00700 1 -- 172.21.15.60:0/1453159578 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fb7f000cd10 con 0x7fb80006c0b0 2024-05-03T14:44:33.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:33.929+0000 7fb7f6a00700 1 --2- 172.21.15.60:0/1453159578 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fb7e40b2880 0x7fb7e40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:44:33.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:33.929+0000 7fb7f6a00700 1 -- 172.21.15.60:0/1453159578 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7879+0+0 (secure 0 0 0) 0x7fb7f00e26f0 con 0x7fb80006c0b0 2024-05-03T14:44:33.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:33.929+0000 7fb7f7e00700 1 --2- 172.21.15.60:0/1453159578 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fb7e40b2880 0x7fb7e40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:44:33.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:33.930+0000 7fb7f7e00700 1 --2- 172.21.15.60:0/1453159578 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fb7e40b2880 0x7fb7e40b4d40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb800079ae0 tx=0x7fb7e8009590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:44:33.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:33.932+0000 7fb7f6a00700 1 -- 172.21.15.60:0/1453159578 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fb7f00aa520 con 0x7fb80006c0b0 2024-05-03T14:44:34.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:34.223+0000 7fb804e00700 1 -- 172.21.15.60:0/1453159578 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fb8001a8d70 con 0x7fb80006c0b0 2024-05-03T14:44:34.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:34.458+0000 7fb7f6a00700 1 -- 172.21.15.60:0/1453159578 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fb7f00a9e20 con 0x7fb80006c0b0 2024-05-03T14:44:34.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:34.464+0000 7fb804e00700 1 -- 172.21.15.60:0/1453159578 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fb7e40b2880 msgr2=0x7fb7e40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:34.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:34.464+0000 7fb804e00700 1 --2- 172.21.15.60:0/1453159578 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fb7e40b2880 0x7fb7e40b4d40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb800079ae0 tx=0x7fb7e8009590 comp rx=0 tx=0).stop 2024-05-03T14:44:34.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:34.464+0000 7fb804e00700 1 -- 172.21.15.60:0/1453159578 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb80006c0b0 msgr2=0x7fb80019bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:34.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:34.464+0000 7fb804e00700 1 --2- 172.21.15.60:0/1453159578 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb80006c0b0 0x7fb80019bf50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb7f000e9f0 tx=0x7fb7f000ed00 comp rx=0 tx=0).stop 2024-05-03T14:44:34.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:34.464+0000 7fb804e00700 1 -- 172.21.15.60:0/1453159578 shutdown_connections 2024-05-03T14:44:34.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:34.464+0000 7fb804e00700 1 --2- 172.21.15.60:0/1453159578 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fb7e40b2880 0x7fb7e40b4d40 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:34.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:34.464+0000 7fb804e00700 1 --2- 172.21.15.60:0/1453159578 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb80010fdf0 0x7fb8001a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:34.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:34.464+0000 7fb804e00700 1 --2- 172.21.15.60:0/1453159578 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb80006c0b0 0x7fb80019bf50 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:34.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:34.464+0000 7fb804e00700 1 --2- 172.21.15.60:0/1453159578 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8001043b0 0x7fb80019c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:34.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:34.464+0000 7fb804e00700 1 -- 172.21.15.60:0/1453159578 >> 172.21.15.60:0/1453159578 conn(0x7fb800079350 msgr2=0x7fb800101770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:44:34.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:34.465+0000 7fb804e00700 1 -- 172.21.15.60:0/1453159578 shutdown_connections 2024-05-03T14:44:34.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:34.465+0000 7fb804e00700 1 -- 172.21.15.60:0/1453159578 wait complete. 2024-05-03T14:44:34.472 INFO:teuthology.orchestra.run.smithi060.stderr:new fs with metadata pool 2 and data pool 3 2024-05-03T14:44:34.498 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:34 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/1453159578' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-03T14:44:34.498 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:34 smithi060 ceph-mon[28873]: daemon mds.a assigned to filesystem cephfs as rank 0 2024-05-03T14:44:34.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:34 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/1453159578' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-03T14:44:34.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:34 smithi090 ceph-mon[33568]: daemon mds.a assigned to filesystem cephfs as rank 0 2024-05-03T14:44:34.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:34 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/1453159578' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-03T14:44:34.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:34 smithi149 ceph-mon[33332]: daemon mds.a assigned to filesystem cephfs as rank 0 2024-05-03T14:44:35.329 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-05-03T14:44:35.329 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-05-03T14:44:35.590 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:35 smithi060 ceph-mon[28873]: pgmap v322: 97 pgs: 11 creating+peering, 49 active+clean, 37 unknown; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:35.590 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:35 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/1453159578' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-03T14:44:35.590 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:35 smithi060 ceph-mon[28873]: osdmap e78: 12 total, 12 up, 12 in 2024-05-03T14:44:35.590 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:35 smithi060 ceph-mon[28873]: fsmap cephfs:1 {0=a=up:creating} 11 up:standby 2024-05-03T14:44:35.590 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:35 smithi060 ceph-mon[28873]: daemon mds.a is now active in filesystem cephfs as rank 0 2024-05-03T14:44:35.800 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:44:35.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:35 smithi090 ceph-mon[33568]: pgmap v322: 97 pgs: 11 creating+peering, 49 active+clean, 37 unknown; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:35.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:35 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/1453159578' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-03T14:44:35.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:35 smithi090 ceph-mon[33568]: osdmap e78: 12 total, 12 up, 12 in 2024-05-03T14:44:35.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:35 smithi090 ceph-mon[33568]: fsmap cephfs:1 {0=a=up:creating} 11 up:standby 2024-05-03T14:44:35.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:35 smithi090 ceph-mon[33568]: daemon mds.a is now active in filesystem cephfs as rank 0 2024-05-03T14:44:35.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:35 smithi149 ceph-mon[33332]: pgmap v322: 97 pgs: 11 creating+peering, 49 active+clean, 37 unknown; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:35.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:35 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/1453159578' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-03T14:44:35.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:35 smithi149 ceph-mon[33332]: osdmap e78: 12 total, 12 up, 12 in 2024-05-03T14:44:35.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:35 smithi149 ceph-mon[33332]: fsmap cephfs:1 {0=a=up:creating} 11 up:standby 2024-05-03T14:44:35.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:35 smithi149 ceph-mon[33332]: daemon mds.a is now active in filesystem cephfs as rank 0 2024-05-03T14:44:36.585 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:36 smithi060 ceph-mon[28873]: mds.? [v2:172.21.15.60:6840/924163182,v1:172.21.15.60:6841/924163182] up:active 2024-05-03T14:44:36.585 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:36 smithi060 ceph-mon[28873]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2024-05-03T14:44:36.585 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:36 smithi060 ceph-mon[28873]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2024-05-03T14:44:36.585 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:36 smithi060 ceph-mon[28873]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-03T14:44:36.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:36 smithi090 ceph-mon[33568]: mds.? [v2:172.21.15.60:6840/924163182,v1:172.21.15.60:6841/924163182] up:active 2024-05-03T14:44:36.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:36 smithi090 ceph-mon[33568]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2024-05-03T14:44:36.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:36 smithi090 ceph-mon[33568]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2024-05-03T14:44:36.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:36 smithi090 ceph-mon[33568]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-03T14:44:36.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:36 smithi149 ceph-mon[33332]: mds.? [v2:172.21.15.60:6840/924163182,v1:172.21.15.60:6841/924163182] up:active 2024-05-03T14:44:36.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:36 smithi149 ceph-mon[33332]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2024-05-03T14:44:36.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:36 smithi149 ceph-mon[33332]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2024-05-03T14:44:36.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:36 smithi149 ceph-mon[33332]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-03T14:44:37.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:37 smithi060 ceph-mon[28873]: pgmap v324: 97 pgs: 4 creating+peering, 93 active+clean; 578 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 522 B/s wr, 1 op/s 2024-05-03T14:44:37.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:37 smithi060 ceph-mon[28873]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-03T14:44:37.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:37 smithi060 ceph-mon[28873]: Cluster is now healthy 2024-05-03T14:44:37.571 INFO:journalctl@ceph.mds.a.smithi060.stdout:May 03 14:44:37 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-a[68311]: 2024-05-03T14:44:37.379+0000 7ff15dc00700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-03T14:44:37.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:37 smithi090 ceph-mon[33568]: pgmap v324: 97 pgs: 4 creating+peering, 93 active+clean; 578 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 522 B/s wr, 1 op/s 2024-05-03T14:44:37.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:37 smithi090 ceph-mon[33568]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-03T14:44:37.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:37 smithi090 ceph-mon[33568]: Cluster is now healthy 2024-05-03T14:44:37.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:37 smithi149 ceph-mon[33332]: pgmap v324: 97 pgs: 4 creating+peering, 93 active+clean; 578 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 522 B/s wr, 1 op/s 2024-05-03T14:44:37.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:37 smithi149 ceph-mon[33332]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-03T14:44:37.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:37 smithi149 ceph-mon[33332]: Cluster is now healthy 2024-05-03T14:44:38.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:38.067+0000 7f8407c00700 1 -- 172.21.15.60:0/456974134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f84001056c0 msgr2=0x7f8400105b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:38.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:38.067+0000 7f8407c00700 1 --2- 172.21.15.60:0/456974134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f84001056c0 0x7f8400105b40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f83ec0098f0 tx=0x7f83ec009c00 comp rx=0 tx=0).stop 2024-05-03T14:44:38.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:38.067+0000 7f8407c00700 1 -- 172.21.15.60:0/456974134 shutdown_connections 2024-05-03T14:44:38.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:38.067+0000 7f8407c00700 1 --2- 172.21.15.60:0/456974134 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8400106080 0x7f8400112590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:38.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:38.067+0000 7f8407c00700 1 --2- 172.21.15.60:0/456974134 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f840010b6c0 0x7f840010baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:38.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:38.067+0000 7f8407c00700 1 --2- 172.21.15.60:0/456974134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f84001056c0 0x7f8400105b40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:38.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:38.067+0000 7f8407c00700 1 -- 172.21.15.60:0/456974134 >> 172.21.15.60:0/456974134 conn(0x7f84001011b0 msgr2=0x7f84001035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:44:38.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:38.067+0000 7f8407c00700 1 -- 172.21.15.60:0/456974134 shutdown_connections 2024-05-03T14:44:38.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:38.067+0000 7f8407c00700 1 -- 172.21.15.60:0/456974134 wait complete. 2024-05-03T14:44:38.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:38.068+0000 7f8407c00700 1 Processor -- start 2024-05-03T14:44:38.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:38.069+0000 7f8407c00700 1 -- start start 2024-05-03T14:44:38.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:38.069+0000 7f8407c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84001056c0 0x7f840019bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:44:38.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:38.069+0000 7f8407c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8400106080 0x7f840019c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:44:38.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:38.069+0000 7f8407c00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f840010b6c0 0x7f84001a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:44:38.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:38.069+0000 7f8407c00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f8400115b80 con 0x7f840010b6c0 2024-05-03T14:44:38.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:38.069+0000 7f8407c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8400115a00 con 0x7f8400106080 2024-05-03T14:44:38.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:38.069+0000 7f8407c00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8400115d00 con 0x7f84001056c0 2024-05-03T14:44:38.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:38.069+0000 7f83ffe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8400106080 0x7f840019c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:44:38.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:38.069+0000 7f83ffe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8400106080 0x7f840019c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.60:45046/0 (socket says 172.21.15.60:45046) 2024-05-03T14:44:38.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:38.069+0000 7f83ffe00700 1 -- 172.21.15.60:0/1518126083 learned_addr learned my addr 172.21.15.60:0/1518126083 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:44:38.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:38.069+0000 7f8405c00700 1 --2- 172.21.15.60:0/1518126083 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f840010b6c0 0x7f84001a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:44:38.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:38.069+0000 7f8405200700 1 --2- 172.21.15.60:0/1518126083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84001056c0 0x7f840019bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:44:38.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:38.070+0000 7f83ffe00700 1 -- 172.21.15.60:0/1518126083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84001056c0 msgr2=0x7f840019bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:38.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:38.070+0000 7f83ffe00700 1 --2- 172.21.15.60:0/1518126083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84001056c0 0x7f840019bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:38.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:38.070+0000 7f83ffe00700 1 -- 172.21.15.60:0/1518126083 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f840010b6c0 msgr2=0x7f84001a08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:38.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:38.070+0000 7f83ffe00700 1 --2- 172.21.15.60:0/1518126083 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f840010b6c0 0x7f84001a08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:38.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:38.070+0000 7f83ffe00700 1 -- 172.21.15.60:0/1518126083 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84001a0fe0 con 0x7f8400106080 2024-05-03T14:44:38.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:38.070+0000 7f83ffe00700 1 --2- 172.21.15.60:0/1518126083 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8400106080 0x7f840019c510 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f83ec010f80 tx=0x7f83ec010fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:44:38.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:38.070+0000 7f83fd600700 1 -- 172.21.15.60:0/1518126083 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ec027070 con 0x7f8400106080 2024-05-03T14:44:38.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:38.070+0000 7f8407c00700 1 -- 172.21.15.60:0/1518126083 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84001a1270 con 0x7f8400106080 2024-05-03T14:44:38.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:38.071+0000 7f83fd600700 1 -- 172.21.15.60:0/1518126083 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f83ec00f460 con 0x7f8400106080 2024-05-03T14:44:38.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:38.071+0000 7f83fd600700 1 -- 172.21.15.60:0/1518126083 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ec004a00 con 0x7f8400106080 2024-05-03T14:44:38.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:38.071+0000 7f8407c00700 1 -- 172.21.15.60:0/1518126083 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84001a8b50 con 0x7f8400106080 2024-05-03T14:44:38.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:38.072+0000 7f8407c00700 1 -- 172.21.15.60:0/1518126083 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f840006a190 con 0x7f8400106080 2024-05-03T14:44:38.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:38.074+0000 7f83fd600700 1 -- 172.21.15.60:0/1518126083 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f83ec02a070 con 0x7f8400106080 2024-05-03T14:44:38.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:38.076+0000 7f83fd600700 1 --2- 172.21.15.60:0/1518126083 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f83e80b2900 0x7f83e80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:44:38.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:38.076+0000 7f83fd600700 1 -- 172.21.15.60:0/1518126083 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f83ec0e2ef0 con 0x7f8400106080 2024-05-03T14:44:38.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:38.076+0000 7f8405200700 1 --2- 172.21.15.60:0/1518126083 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f83e80b2900 0x7f83e80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:44:38.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:38.076+0000 7f8405200700 1 --2- 172.21.15.60:0/1518126083 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f83e80b2900 0x7f83e80b4dc0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f83f4005fd0 tx=0x7f83f4009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:44:38.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:38.077+0000 7f83fd600700 1 -- 172.21.15.60:0/1518126083 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f83ec031080 con 0x7f8400106080 2024-05-03T14:44:38.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:38.325+0000 7f8407c00700 1 -- 172.21.15.60:0/1518126083 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f840019d330 con 0x7f8400106080 2024-05-03T14:44:38.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:38.506+0000 7f83fd600700 1 -- 172.21.15.60:0/1518126083 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v79) ==== 157+0+0 (secure 0 0 0) 0x7f83ec013030 con 0x7f8400106080 2024-05-03T14:44:38.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:38.510+0000 7f8407c00700 1 -- 172.21.15.60:0/1518126083 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f83e80b2900 msgr2=0x7f83e80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:38.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:38.510+0000 7f8407c00700 1 --2- 172.21.15.60:0/1518126083 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f83e80b2900 0x7f83e80b4dc0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f83f4005fd0 tx=0x7f83f4009310 comp rx=0 tx=0).stop 2024-05-03T14:44:38.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:38.511+0000 7f8407c00700 1 -- 172.21.15.60:0/1518126083 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8400106080 msgr2=0x7f840019c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:38.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:38.511+0000 7f8407c00700 1 --2- 172.21.15.60:0/1518126083 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8400106080 0x7f840019c510 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f83ec010f80 tx=0x7f83ec010fb0 comp rx=0 tx=0).stop 2024-05-03T14:44:38.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:38.511+0000 7f8407c00700 1 -- 172.21.15.60:0/1518126083 shutdown_connections 2024-05-03T14:44:38.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:38.511+0000 7f8407c00700 1 --2- 172.21.15.60:0/1518126083 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f83e80b2900 0x7f83e80b4dc0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:38.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:38.511+0000 7f8407c00700 1 --2- 172.21.15.60:0/1518126083 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f840010b6c0 0x7f84001a08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:38.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:38.511+0000 7f8407c00700 1 --2- 172.21.15.60:0/1518126083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84001056c0 0x7f840019bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:38.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:38.511+0000 7f8407c00700 1 --2- 172.21.15.60:0/1518126083 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8400106080 0x7f840019c510 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:38.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:38.511+0000 7f8407c00700 1 -- 172.21.15.60:0/1518126083 >> 172.21.15.60:0/1518126083 conn(0x7f84001011b0 msgr2=0x7f84001017f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:44:38.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:38.511+0000 7f8407c00700 1 -- 172.21.15.60:0/1518126083 shutdown_connections 2024-05-03T14:44:38.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:38.511+0000 7f8407c00700 1 -- 172.21.15.60:0/1518126083 wait complete. 2024-05-03T14:44:38.586 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:38 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/1518126083' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-03T14:44:38.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:38 smithi060 ceph-mon[28873]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-03T14:44:38.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:38 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/1518126083' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-03T14:44:38.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:38 smithi090 ceph-mon[33568]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-03T14:44:38.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:38 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/1518126083' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-03T14:44:38.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:38 smithi149 ceph-mon[33332]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-03T14:44:39.383 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-05-03T14:44:39.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:39 smithi060 ceph-mon[28873]: pgmap v325: 97 pgs: 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 744 B/s wr, 4 op/s 2024-05-03T14:44:39.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:39 smithi060 ceph-mon[28873]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-03T14:44:39.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:39 smithi060 ceph-mon[28873]: osdmap e79: 12 total, 12 up, 12 in 2024-05-03T14:44:39.864 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:44:39.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:39 smithi090 ceph-mon[33568]: pgmap v325: 97 pgs: 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 744 B/s wr, 4 op/s 2024-05-03T14:44:39.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:39 smithi090 ceph-mon[33568]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-03T14:44:39.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:39 smithi090 ceph-mon[33568]: osdmap e79: 12 total, 12 up, 12 in 2024-05-03T14:44:39.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:39 smithi149 ceph-mon[33332]: pgmap v325: 97 pgs: 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 744 B/s wr, 4 op/s 2024-05-03T14:44:39.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:39 smithi149 ceph-mon[33332]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-03T14:44:39.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:39 smithi149 ceph-mon[33332]: osdmap e79: 12 total, 12 up, 12 in 2024-05-03T14:44:41.801 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:41 smithi060 ceph-mon[28873]: pgmap v327: 97 pgs: 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-05-03T14:44:41.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:41 smithi090 ceph-mon[33568]: pgmap v327: 97 pgs: 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-05-03T14:44:41.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:41 smithi149 ceph-mon[33332]: pgmap v327: 97 pgs: 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-05-03T14:44:42.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:42.102+0000 7f93b5200700 1 -- 172.21.15.60:0/252331608 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93b00756d0 msgr2=0x7f93b0075ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:42.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:42.102+0000 7f93b5200700 1 --2- 172.21.15.60:0/252331608 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93b00756d0 0x7f93b0075ab0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f93a00098f0 tx=0x7f93a0009c00 comp rx=0 tx=0).stop 2024-05-03T14:44:42.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:42.102+0000 7f93b5200700 1 -- 172.21.15.60:0/252331608 shutdown_connections 2024-05-03T14:44:42.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:42.102+0000 7f93b5200700 1 --2- 172.21.15.60:0/252331608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93b010ff00 0x7f93b01122f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:42.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:42.102+0000 7f93b5200700 1 --2- 172.21.15.60:0/252331608 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93b00756d0 0x7f93b0075ab0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:42.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:42.102+0000 7f93b5200700 1 --2- 172.21.15.60:0/252331608 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f93b0075ff0 0x7f93b010f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:42.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:42.102+0000 7f93b5200700 1 -- 172.21.15.60:0/252331608 >> 172.21.15.60:0/252331608 conn(0x7f93b007a570 msgr2=0x7f93b007a980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:44:42.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:42.102+0000 7f93b5200700 1 -- 172.21.15.60:0/252331608 shutdown_connections 2024-05-03T14:44:42.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:42.102+0000 7f93b5200700 1 -- 172.21.15.60:0/252331608 wait complete. 2024-05-03T14:44:42.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:42.103+0000 7f93b5200700 1 Processor -- start 2024-05-03T14:44:42.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:42.104+0000 7f93b5200700 1 -- start start 2024-05-03T14:44:42.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:42.104+0000 7f93b5200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f93b0075ff0 0x7f93b019c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:44:42.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:42.104+0000 7f93b5200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93b010ff00 0x7f93b019c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:44:42.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:42.104+0000 7f93b5200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93b01a09e0 0x7f93b01a0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:44:42.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:42.104+0000 7f93b5200700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f93b0115b70 con 0x7f93b01a09e0 2024-05-03T14:44:42.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:42.104+0000 7f93b5200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f93b01159f0 con 0x7f93b0075ff0 2024-05-03T14:44:42.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:42.104+0000 7f93b5200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f93b0115cf0 con 0x7f93b010ff00 2024-05-03T14:44:42.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:42.104+0000 7f93ae800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93b01a09e0 0x7f93b01a0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:44:42.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:42.104+0000 7f93ae800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93b01a09e0 0x7f93b01a0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57886/0 (socket says 172.21.15.60:57886) 2024-05-03T14:44:42.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:42.104+0000 7f93ae800700 1 -- 172.21.15.60:0/396864031 learned_addr learned my addr 172.21.15.60:0/396864031 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:44:42.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:42.104+0000 7f93ade00700 1 --2- 172.21.15.60:0/396864031 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f93b0075ff0 0x7f93b019c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:44:42.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:42.104+0000 7f93ad400700 1 --2- 172.21.15.60:0/396864031 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93b010ff00 0x7f93b019c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:44:42.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:42.105+0000 7f93ade00700 1 -- 172.21.15.60:0/396864031 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93b010ff00 msgr2=0x7f93b019c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:42.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:42.105+0000 7f93ade00700 1 --2- 172.21.15.60:0/396864031 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93b010ff00 0x7f93b019c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:42.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:42.105+0000 7f93ade00700 1 -- 172.21.15.60:0/396864031 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93b01a09e0 msgr2=0x7f93b01a0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:42.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:42.105+0000 7f93ade00700 1 --2- 172.21.15.60:0/396864031 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93b01a09e0 0x7f93b01a0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:42.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:42.105+0000 7f93ade00700 1 -- 172.21.15.60:0/396864031 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93b01a1500 con 0x7f93b0075ff0 2024-05-03T14:44:42.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:42.105+0000 7f93ae800700 1 --2- 172.21.15.60:0/396864031 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93b01a09e0 0x7f93b01a0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:44:42.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:42.105+0000 7f93ade00700 1 --2- 172.21.15.60:0/396864031 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f93b0075ff0 0x7f93b019c160 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f93a0008000 tx=0x7f93a00094b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:44:42.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:42.105+0000 7f93a6a00700 1 -- 172.21.15.60:0/396864031 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a000b6e0 con 0x7f93b0075ff0 2024-05-03T14:44:42.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:42.105+0000 7f93a6a00700 1 -- 172.21.15.60:0/396864031 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f93a000f460 con 0x7f93b0075ff0 2024-05-03T14:44:42.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:42.106+0000 7f93b5200700 1 -- 172.21.15.60:0/396864031 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93b01a1790 con 0x7f93b0075ff0 2024-05-03T14:44:42.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:42.106+0000 7f93a6a00700 1 -- 172.21.15.60:0/396864031 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a0003a10 con 0x7f93b0075ff0 2024-05-03T14:44:42.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:42.106+0000 7f93b5200700 1 -- 172.21.15.60:0/396864031 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93b01a8ec0 con 0x7f93b0075ff0 2024-05-03T14:44:42.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:42.107+0000 7f93b5200700 1 -- 172.21.15.60:0/396864031 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93b00519e0 con 0x7f93b0075ff0 2024-05-03T14:44:42.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:42.109+0000 7f93a6a00700 1 -- 172.21.15.60:0/396864031 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f93a000f9e0 con 0x7f93b0075ff0 2024-05-03T14:44:42.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:42.110+0000 7f93a6a00700 1 --2- 172.21.15.60:0/396864031 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f93940b2930 0x7f93940b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:44:42.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:42.111+0000 7f93a6a00700 1 -- 172.21.15.60:0/396864031 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8277+0+0 (secure 0 0 0) 0x7f93a00e31c0 con 0x7f93b0075ff0 2024-05-03T14:44:42.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:42.111+0000 7f93ad400700 1 --2- 172.21.15.60:0/396864031 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f93940b2930 0x7f93940b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:44:42.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:42.112+0000 7f93ad400700 1 --2- 172.21.15.60:0/396864031 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f93940b2930 0x7f93940b4df0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f93b0111c20 tx=0x7f939c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:44:42.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:42.115+0000 7f93a6a00700 1 -- 172.21.15.60:0/396864031 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f93a0013030 con 0x7f93b0075ff0 2024-05-03T14:44:42.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:42.355+0000 7f93b5200700 1 -- 172.21.15.60:0/396864031 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f93b002e6d0 con 0x7f93b0075ff0 2024-05-03T14:44:42.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:42 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/396864031' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-03T14:44:42.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:42 smithi060 ceph-mon[28873]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-03T14:44:42.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:42 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/396864031' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-03T14:44:42.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:42 smithi090 ceph-mon[33568]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-03T14:44:42.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:42 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/396864031' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-03T14:44:42.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:42 smithi149 ceph-mon[33332]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-03T14:44:43.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:43.548+0000 7f93a6a00700 1 -- 172.21.15.60:0/396864031 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v81) ==== 228+0+0 (secure 0 0 0) 0x7f93a00aaee0 con 0x7f93b0075ff0 2024-05-03T14:44:43.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:43.558+0000 7f93b5200700 1 -- 172.21.15.60:0/396864031 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f93940b2930 msgr2=0x7f93940b4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:43.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:43.558+0000 7f93b5200700 1 --2- 172.21.15.60:0/396864031 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f93940b2930 0x7f93940b4df0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f93b0111c20 tx=0x7f939c009310 comp rx=0 tx=0).stop 2024-05-03T14:44:43.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:43.558+0000 7f93b5200700 1 -- 172.21.15.60:0/396864031 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f93b0075ff0 msgr2=0x7f93b019c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:43.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:43.558+0000 7f93b5200700 1 --2- 172.21.15.60:0/396864031 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f93b0075ff0 0x7f93b019c160 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f93a0008000 tx=0x7f93a00094b0 comp rx=0 tx=0).stop 2024-05-03T14:44:43.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:43.558+0000 7f93b5200700 1 -- 172.21.15.60:0/396864031 shutdown_connections 2024-05-03T14:44:43.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:43.558+0000 7f93b5200700 1 --2- 172.21.15.60:0/396864031 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f93940b2930 0x7f93940b4df0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:43.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:43.558+0000 7f93b5200700 1 --2- 172.21.15.60:0/396864031 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93b01a09e0 0x7f93b01a0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:43.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:43.558+0000 7f93b5200700 1 --2- 172.21.15.60:0/396864031 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f93b0075ff0 0x7f93b019c160 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:43.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:43.558+0000 7f93b5200700 1 --2- 172.21.15.60:0/396864031 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93b010ff00 0x7f93b019c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:43.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:43.558+0000 7f93b5200700 1 -- 172.21.15.60:0/396864031 >> 172.21.15.60:0/396864031 conn(0x7f93b007a570 msgr2=0x7f93b0100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:44:43.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:43.559+0000 7f93b5200700 1 -- 172.21.15.60:0/396864031 shutdown_connections 2024-05-03T14:44:43.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:43.559+0000 7f93b5200700 1 -- 172.21.15.60:0/396864031 wait complete. 2024-05-03T14:44:43.567 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data_ec' created 2024-05-03T14:44:43.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:43 smithi060 ceph-mon[28873]: pgmap v328: 97 pgs: 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-03T14:44:43.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:43 smithi060 ceph-mon[28873]: osdmap e80: 12 total, 12 up, 12 in 2024-05-03T14:44:43.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:43 smithi090 ceph-mon[33568]: pgmap v328: 97 pgs: 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-03T14:44:43.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:43 smithi090 ceph-mon[33568]: osdmap e80: 12 total, 12 up, 12 in 2024-05-03T14:44:43.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:43 smithi149 ceph-mon[33332]: pgmap v328: 97 pgs: 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-03T14:44:43.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:43 smithi149 ceph-mon[33332]: osdmap e80: 12 total, 12 up, 12 in 2024-05-03T14:44:44.472 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-05-03T14:44:44.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:44 smithi060 ceph-mon[28873]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-03T14:44:44.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:44 smithi060 ceph-mon[28873]: osdmap e81: 12 total, 12 up, 12 in 2024-05-03T14:44:44.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:44 smithi090 ceph-mon[33568]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-03T14:44:44.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:44 smithi090 ceph-mon[33568]: osdmap e81: 12 total, 12 up, 12 in 2024-05-03T14:44:44.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:44 smithi149 ceph-mon[33332]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-03T14:44:44.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:44 smithi149 ceph-mon[33332]: osdmap e81: 12 total, 12 up, 12 in 2024-05-03T14:44:44.962 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:44:45.739 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:45 smithi060 ceph-mon[28873]: pgmap v331: 129 pgs: 11 creating+peering, 21 unknown, 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 1 op/s 2024-05-03T14:44:45.740 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:45 smithi060 ceph-mon[28873]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-03T14:44:45.740 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:45 smithi060 ceph-mon[28873]: osdmap e82: 12 total, 12 up, 12 in 2024-05-03T14:44:45.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:45 smithi090 ceph-mon[33568]: pgmap v331: 129 pgs: 11 creating+peering, 21 unknown, 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 1 op/s 2024-05-03T14:44:45.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:45 smithi090 ceph-mon[33568]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-03T14:44:45.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:45 smithi090 ceph-mon[33568]: osdmap e82: 12 total, 12 up, 12 in 2024-05-03T14:44:45.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:45 smithi149 ceph-mon[33332]: pgmap v331: 129 pgs: 11 creating+peering, 21 unknown, 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 1 op/s 2024-05-03T14:44:45.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:45 smithi149 ceph-mon[33332]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-03T14:44:45.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:45 smithi149 ceph-mon[33332]: osdmap e82: 12 total, 12 up, 12 in 2024-05-03T14:44:46.829 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:46 smithi060 ceph-mon[28873]: osdmap e83: 12 total, 12 up, 12 in 2024-05-03T14:44:46.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:46 smithi090 ceph-mon[33568]: osdmap e83: 12 total, 12 up, 12 in 2024-05-03T14:44:46.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:46 smithi149 ceph-mon[33332]: osdmap e83: 12 total, 12 up, 12 in 2024-05-03T14:44:47.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:47.353+0000 7f3f3b000700 1 -- 172.21.15.60:0/3041916760 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3f3410b680 msgr2=0x7f3f3410ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:47.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:47.353+0000 7f3f3b000700 1 --2- 172.21.15.60:0/3041916760 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3f3410b680 0x7f3f3410ba60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3f28009f90 tx=0x7f3f2800b330 comp rx=0 tx=0).stop 2024-05-03T14:44:47.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:47.353+0000 7f3f3b000700 1 -- 172.21.15.60:0/3041916760 shutdown_connections 2024-05-03T14:44:47.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:47.353+0000 7f3f3b000700 1 --2- 172.21.15.60:0/3041916760 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f34105f90 0x7f3f34112450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:47.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:47.353+0000 7f3f3b000700 1 --2- 172.21.15.60:0/3041916760 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3f3410b680 0x7f3f3410ba60 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:47.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:47.353+0000 7f3f3b000700 1 --2- 172.21.15.60:0/3041916760 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3f341055d0 0x7f3f34105a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:47.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:47.353+0000 7f3f3b000700 1 -- 172.21.15.60:0/3041916760 >> 172.21.15.60:0/3041916760 conn(0x7f3f34101080 msgr2=0x7f3f341034a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:44:47.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:47.353+0000 7f3f3b000700 1 -- 172.21.15.60:0/3041916760 shutdown_connections 2024-05-03T14:44:47.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:47.353+0000 7f3f3b000700 1 -- 172.21.15.60:0/3041916760 wait complete. 2024-05-03T14:44:47.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:47.354+0000 7f3f3b000700 1 Processor -- start 2024-05-03T14:44:47.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:47.354+0000 7f3f3b000700 1 -- start start 2024-05-03T14:44:47.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:47.355+0000 7f3f3b000700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f341055d0 0x7f3f3419bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:44:47.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:47.355+0000 7f3f3b000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3f34105f90 0x7f3f3419c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:44:47.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:47.355+0000 7f3f3b000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3f3410b680 0x7f3f341a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:44:47.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:47.355+0000 7f3f3b000700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f3f34115a00 con 0x7f3f341055d0 2024-05-03T14:44:47.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:47.355+0000 7f3f3b000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3f34115880 con 0x7f3f34105f90 2024-05-03T14:44:47.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:47.355+0000 7f3f3b000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3f34115b80 con 0x7f3f3410b680 2024-05-03T14:44:47.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:47.355+0000 7f3f39c00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f341055d0 0x7f3f3419bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:44:47.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:47.355+0000 7f3f39c00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f341055d0 0x7f3f3419bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57896/0 (socket says 172.21.15.60:57896) 2024-05-03T14:44:47.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:47.355+0000 7f3f39c00700 1 -- 172.21.15.60:0/2092117965 learned_addr learned my addr 172.21.15.60:0/2092117965 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:44:47.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:47.355+0000 7f3f39200700 1 --2- 172.21.15.60:0/2092117965 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3f34105f90 0x7f3f3419c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:44:47.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:47.355+0000 7f3f3a600700 1 --2- 172.21.15.60:0/2092117965 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3f3410b680 0x7f3f341a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:44:47.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:47.355+0000 7f3f39c00700 1 -- 172.21.15.60:0/2092117965 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3f3410b680 msgr2=0x7f3f341a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:47.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:47.355+0000 7f3f39c00700 1 --2- 172.21.15.60:0/2092117965 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3f3410b680 0x7f3f341a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:47.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:47.355+0000 7f3f39c00700 1 -- 172.21.15.60:0/2092117965 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3f34105f90 msgr2=0x7f3f3419c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:47.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:47.355+0000 7f3f39c00700 1 --2- 172.21.15.60:0/2092117965 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3f34105f90 0x7f3f3419c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:47.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:47.355+0000 7f3f39c00700 1 -- 172.21.15.60:0/2092117965 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f341a0d70 con 0x7f3f341055d0 2024-05-03T14:44:47.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:47.356+0000 7f3f39c00700 1 --2- 172.21.15.60:0/2092117965 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f341055d0 0x7f3f3419bdb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3f2800bf30 tx=0x7f3f28007960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:44:47.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:47.356+0000 7f3f32000700 1 -- 172.21.15.60:0/2092117965 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f28021020 con 0x7f3f341055d0 2024-05-03T14:44:47.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:47.356+0000 7f3f3b000700 1 -- 172.21.15.60:0/2092117965 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f341a1000 con 0x7f3f341055d0 2024-05-03T14:44:47.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:47.356+0000 7f3f32000700 1 -- 172.21.15.60:0/2092117965 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f3f2801d470 con 0x7f3f341055d0 2024-05-03T14:44:47.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:47.356+0000 7f3f32000700 1 -- 172.21.15.60:0/2092117965 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f28005290 con 0x7f3f341055d0 2024-05-03T14:44:47.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:47.356+0000 7f3f3b000700 1 -- 172.21.15.60:0/2092117965 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f341a88e0 con 0x7f3f341055d0 2024-05-03T14:44:47.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:47.357+0000 7f3f3b000700 1 -- 172.21.15.60:0/2092117965 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f3410db10 con 0x7f3f341055d0 2024-05-03T14:44:47.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:47.362+0000 7f3f32000700 1 -- 172.21.15.60:0/2092117965 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f3f28024020 con 0x7f3f341055d0 2024-05-03T14:44:47.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:47.363+0000 7f3f32000700 1 --2- 172.21.15.60:0/2092117965 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f3f1c0b2880 0x7f3f1c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:44:47.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:47.363+0000 7f3f32000700 1 -- 172.21.15.60:0/2092117965 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8770+0+0 (secure 0 0 0) 0x7f3f2800f020 con 0x7f3f341055d0 2024-05-03T14:44:47.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:47.363+0000 7f3f39200700 1 --2- 172.21.15.60:0/2092117965 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f3f1c0b2880 0x7f3f1c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:44:47.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:47.364+0000 7f3f39200700 1 --2- 172.21.15.60:0/2092117965 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f3f1c0b2880 0x7f3f1c0b4d40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3f20009750 tx=0x7f3f20009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:44:47.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:47.367+0000 7f3f32000700 1 -- 172.21.15.60:0/2092117965 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f3f280ab3e0 con 0x7f3f341055d0 2024-05-03T14:44:47.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:47.613+0000 7f3f3b000700 1 -- 172.21.15.60:0/2092117965 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f3f3406a190 con 0x7f3f341055d0 2024-05-03T14:44:47.617 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:47 smithi060 ceph-mon[28873]: pgmap v334: 129 pgs: 1 creating+activating, 15 creating+peering, 7 unknown, 106 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:47.847 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:47 smithi149 ceph-mon[33332]: pgmap v334: 129 pgs: 1 creating+activating, 15 creating+peering, 7 unknown, 106 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:47.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:47 smithi090 ceph-mon[33568]: pgmap v334: 129 pgs: 1 creating+activating, 15 creating+peering, 7 unknown, 106 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:48.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:48.587+0000 7f3f32000700 1 -- 172.21.15.60:0/2092117965 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v84) ==== 169+0+0 (secure 0 0 0) 0x7f3f280aace0 con 0x7f3f341055d0 2024-05-03T14:44:48.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:48.593+0000 7f3f3b000700 1 -- 172.21.15.60:0/2092117965 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f3f1c0b2880 msgr2=0x7f3f1c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:48.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:48.593+0000 7f3f3b000700 1 --2- 172.21.15.60:0/2092117965 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f3f1c0b2880 0x7f3f1c0b4d40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3f20009750 tx=0x7f3f20009310 comp rx=0 tx=0).stop 2024-05-03T14:44:48.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:48.593+0000 7f3f3b000700 1 -- 172.21.15.60:0/2092117965 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f341055d0 msgr2=0x7f3f3419bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:48.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:48.593+0000 7f3f3b000700 1 --2- 172.21.15.60:0/2092117965 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f341055d0 0x7f3f3419bdb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3f2800bf30 tx=0x7f3f28007960 comp rx=0 tx=0).stop 2024-05-03T14:44:48.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:48.593+0000 7f3f3b000700 1 -- 172.21.15.60:0/2092117965 shutdown_connections 2024-05-03T14:44:48.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:48.593+0000 7f3f3b000700 1 --2- 172.21.15.60:0/2092117965 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f3f1c0b2880 0x7f3f1c0b4d40 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:48.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:48.593+0000 7f3f3b000700 1 --2- 172.21.15.60:0/2092117965 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3f3410b680 0x7f3f341a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:48.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:48.593+0000 7f3f3b000700 1 --2- 172.21.15.60:0/2092117965 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f341055d0 0x7f3f3419bdb0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:48.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:48.593+0000 7f3f3b000700 1 --2- 172.21.15.60:0/2092117965 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3f34105f90 0x7f3f3419c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:48.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:48.593+0000 7f3f3b000700 1 -- 172.21.15.60:0/2092117965 >> 172.21.15.60:0/2092117965 conn(0x7f3f34101080 msgr2=0x7f3f34102640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:44:48.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:48.593+0000 7f3f3b000700 1 -- 172.21.15.60:0/2092117965 shutdown_connections 2024-05-03T14:44:48.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:48.593+0000 7f3f3b000700 1 -- 172.21.15.60:0/2092117965 wait complete. 2024-05-03T14:44:48.598 INFO:teuthology.orchestra.run.smithi060.stderr:set pool 4 allow_ec_overwrites to true 2024-05-03T14:44:48.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:48 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/2092117965' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-03T14:44:48.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:48 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/2092117965' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-03T14:44:48.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:48 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/2092117965' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-03T14:44:49.546 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-05-03T14:44:49.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:49 smithi060 ceph-mon[28873]: pgmap v335: 129 pgs: 1 creating+activating, 15 creating+peering, 113 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:49.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:49 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/2092117965' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-03T14:44:49.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:49 smithi060 ceph-mon[28873]: osdmap e84: 12 total, 12 up, 12 in 2024-05-03T14:44:49.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:49 smithi090 ceph-mon[33568]: pgmap v335: 129 pgs: 1 creating+activating, 15 creating+peering, 113 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:49.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:49 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/2092117965' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-03T14:44:49.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:49 smithi090 ceph-mon[33568]: osdmap e84: 12 total, 12 up, 12 in 2024-05-03T14:44:49.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:49 smithi149 ceph-mon[33332]: pgmap v335: 129 pgs: 1 creating+activating, 15 creating+peering, 113 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:49.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:49 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/2092117965' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-03T14:44:49.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:49 smithi149 ceph-mon[33332]: osdmap e84: 12 total, 12 up, 12 in 2024-05-03T14:44:50.019 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:44:51.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:51 smithi090 ceph-mon[33568]: pgmap v337: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:51.879 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:51 smithi060 ceph-mon[28873]: pgmap v337: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:51.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:51 smithi149 ceph-mon[33332]: pgmap v337: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:52.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:52.168+0000 7f7141000700 1 -- 172.21.15.60:0/3591864708 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f713c10b6c0 msgr2=0x7f713c10baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:52.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:52.168+0000 7f7141000700 1 --2- 172.21.15.60:0/3591864708 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f713c10b6c0 0x7f713c10baa0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f712c0098f0 tx=0x7f712c009c00 comp rx=0 tx=0).stop 2024-05-03T14:44:52.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:52.168+0000 7f7141000700 1 -- 172.21.15.60:0/3591864708 shutdown_connections 2024-05-03T14:44:52.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:52.168+0000 7f7141000700 1 --2- 172.21.15.60:0/3591864708 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f713c106080 0x7f713c112590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:52.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:52.168+0000 7f7141000700 1 --2- 172.21.15.60:0/3591864708 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f713c10b6c0 0x7f713c10baa0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:52.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:52.168+0000 7f7141000700 1 --2- 172.21.15.60:0/3591864708 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f713c1056c0 0x7f713c105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:52.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:52.168+0000 7f7141000700 1 -- 172.21.15.60:0/3591864708 >> 172.21.15.60:0/3591864708 conn(0x7f713c1011b0 msgr2=0x7f713c1035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:44:52.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:52.168+0000 7f7141000700 1 -- 172.21.15.60:0/3591864708 shutdown_connections 2024-05-03T14:44:52.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:52.168+0000 7f7141000700 1 -- 172.21.15.60:0/3591864708 wait complete. 2024-05-03T14:44:52.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:52.169+0000 7f7141000700 1 Processor -- start 2024-05-03T14:44:52.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:52.169+0000 7f7141000700 1 -- start start 2024-05-03T14:44:52.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:52.170+0000 7f7141000700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f713c1056c0 0x7f713c19bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:44:52.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:52.170+0000 7f7141000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f713c106080 0x7f713c19c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:44:52.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:52.170+0000 7f7141000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f713c10b6c0 0x7f713c1a0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:44:52.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:52.170+0000 7f7141000700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f713c115b60 con 0x7f713c1056c0 2024-05-03T14:44:52.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:52.170+0000 7f7141000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f713c1159e0 con 0x7f713c106080 2024-05-03T14:44:52.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:52.170+0000 7f7141000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f713c115ce0 con 0x7f713c10b6c0 2024-05-03T14:44:52.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:52.170+0000 7f7139e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f713c1056c0 0x7f713c19bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:44:52.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:52.170+0000 7f7133e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f713c106080 0x7f713c19c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:44:52.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:52.170+0000 7f7139e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f713c1056c0 0x7f713c19bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55264/0 (socket says 172.21.15.60:55264) 2024-05-03T14:44:52.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:52.170+0000 7f7139e00700 1 -- 172.21.15.60:0/1412293476 learned_addr learned my addr 172.21.15.60:0/1412293476 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:44:52.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:52.171+0000 7f713a800700 1 --2- 172.21.15.60:0/1412293476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f713c10b6c0 0x7f713c1a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:44:52.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:52.171+0000 7f7133e00700 1 -- 172.21.15.60:0/1412293476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f713c10b6c0 msgr2=0x7f713c1a0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:52.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:52.171+0000 7f7133e00700 1 --2- 172.21.15.60:0/1412293476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f713c10b6c0 0x7f713c1a0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:52.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:52.171+0000 7f7133e00700 1 -- 172.21.15.60:0/1412293476 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f713c1056c0 msgr2=0x7f713c19bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:52.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:52.171+0000 7f7133e00700 1 --2- 172.21.15.60:0/1412293476 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f713c1056c0 0x7f713c19bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:52.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:52.171+0000 7f7133e00700 1 -- 172.21.15.60:0/1412293476 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f713c1a0f80 con 0x7f713c106080 2024-05-03T14:44:52.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:52.171+0000 7f713a800700 1 --2- 172.21.15.60:0/1412293476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f713c10b6c0 0x7f713c1a0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T14:44:52.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:52.171+0000 7f7133e00700 1 --2- 172.21.15.60:0/1412293476 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f713c106080 0x7f713c19c4b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f712400e9c0 tx=0x7f712400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:44:52.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:52.171+0000 7f7132a00700 1 -- 172.21.15.60:0/1412293476 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712400c9c0 con 0x7f713c106080 2024-05-03T14:44:52.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:52.172+0000 7f7132a00700 1 -- 172.21.15.60:0/1412293476 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f712400cb60 con 0x7f713c106080 2024-05-03T14:44:52.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:52.172+0000 7f7132a00700 1 -- 172.21.15.60:0/1412293476 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7124018680 con 0x7f713c106080 2024-05-03T14:44:52.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:52.172+0000 7f7141000700 1 -- 172.21.15.60:0/1412293476 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f713c1a1270 con 0x7f713c106080 2024-05-03T14:44:52.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:52.172+0000 7f7141000700 1 -- 172.21.15.60:0/1412293476 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f713c1a8af0 con 0x7f713c106080 2024-05-03T14:44:52.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:52.173+0000 7f7141000700 1 -- 172.21.15.60:0/1412293476 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f713c10dc50 con 0x7f713c106080 2024-05-03T14:44:52.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:52.176+0000 7f7132a00700 1 -- 172.21.15.60:0/1412293476 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f712400cd10 con 0x7f713c106080 2024-05-03T14:44:52.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:52.178+0000 7f7132a00700 1 --2- 172.21.15.60:0/1412293476 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f71200b2900 0x7f71200b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:44:52.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:52.178+0000 7f7132a00700 1 -- 172.21.15.60:0/1412293476 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8770+0+0 (secure 0 0 0) 0x7f71240e2500 con 0x7f713c106080 2024-05-03T14:44:52.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:52.178+0000 7f7139e00700 1 --2- 172.21.15.60:0/1412293476 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f71200b2900 0x7f71200b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:44:52.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:52.179+0000 7f7139e00700 1 --2- 172.21.15.60:0/1412293476 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f71200b2900 0x7f71200b4dc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f712c009810 tx=0x7f712c019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:44:52.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:52.179+0000 7f7132a00700 1 -- 172.21.15.60:0/1412293476 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f71240a9fa0 con 0x7f713c106080 2024-05-03T14:44:52.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:52.468+0000 7f7141000700 1 -- 172.21.15.60:0/1412293476 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f713c06a190 con 0x7f713c106080 2024-05-03T14:44:52.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:52.614+0000 7f7132a00700 1 -- 172.21.15.60:0/1412293476 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f71240a98a0 con 0x7f713c106080 2024-05-03T14:44:52.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:52.617+0000 7f7141000700 1 -- 172.21.15.60:0/1412293476 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f71200b2900 msgr2=0x7f71200b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:52.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:52.617+0000 7f7141000700 1 --2- 172.21.15.60:0/1412293476 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f71200b2900 0x7f71200b4dc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f712c009810 tx=0x7f712c019040 comp rx=0 tx=0).stop 2024-05-03T14:44:52.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:52.617+0000 7f7141000700 1 -- 172.21.15.60:0/1412293476 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f713c106080 msgr2=0x7f713c19c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:52.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:52.617+0000 7f7141000700 1 --2- 172.21.15.60:0/1412293476 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f713c106080 0x7f713c19c4b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f712400e9c0 tx=0x7f712400ecd0 comp rx=0 tx=0).stop 2024-05-03T14:44:52.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:52.618+0000 7f7141000700 1 -- 172.21.15.60:0/1412293476 shutdown_connections 2024-05-03T14:44:52.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:52.618+0000 7f7141000700 1 --2- 172.21.15.60:0/1412293476 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f71200b2900 0x7f71200b4dc0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:52.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:52.618+0000 7f7141000700 1 --2- 172.21.15.60:0/1412293476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f713c10b6c0 0x7f713c1a0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:52.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:52.618+0000 7f7141000700 1 --2- 172.21.15.60:0/1412293476 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f713c1056c0 0x7f713c19bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:52.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:52.618+0000 7f7141000700 1 --2- 172.21.15.60:0/1412293476 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f713c106080 0x7f713c19c4b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:52.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:52.618+0000 7f7141000700 1 -- 172.21.15.60:0/1412293476 >> 172.21.15.60:0/1412293476 conn(0x7f713c1011b0 msgr2=0x7f713c101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:44:52.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:52.618+0000 7f7141000700 1 -- 172.21.15.60:0/1412293476 shutdown_connections 2024-05-03T14:44:52.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:52.618+0000 7f7141000700 1 -- 172.21.15.60:0/1412293476 wait complete. 2024-05-03T14:44:52.622 INFO:teuthology.orchestra.run.smithi060.stderr:added data pool 4 to fsmap 2024-05-03T14:44:52.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:52 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/1412293476' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-03T14:44:52.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:52 smithi060 ceph-mon[28873]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-03T14:44:52.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:52 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/1412293476' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-03T14:44:52.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:52 smithi090 ceph-mon[33568]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-03T14:44:52.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:52 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/1412293476' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-03T14:44:52.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:52 smithi149 ceph-mon[33332]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-03T14:44:53.355 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T14:44:53.835 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:44:53.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:53 smithi090 ceph-mon[33568]: pgmap v338: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:53.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:53 smithi090 ceph-mon[33568]: osdmap e85: 12 total, 12 up, 12 in 2024-05-03T14:44:53.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:53 smithi090 ceph-mon[33568]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-03T14:44:53.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:53 smithi090 ceph-mon[33568]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2024-05-03T14:44:53.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:53 smithi149 ceph-mon[33332]: pgmap v338: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:53.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:53 smithi149 ceph-mon[33332]: osdmap e85: 12 total, 12 up, 12 in 2024-05-03T14:44:53.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:53 smithi149 ceph-mon[33332]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-03T14:44:53.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:53 smithi149 ceph-mon[33332]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2024-05-03T14:44:53.994 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:53 smithi060 ceph-mon[28873]: pgmap v338: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:53.994 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:53 smithi060 ceph-mon[28873]: osdmap e85: 12 total, 12 up, 12 in 2024-05-03T14:44:53.994 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:53 smithi060 ceph-mon[28873]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-03T14:44:53.995 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:53 smithi060 ceph-mon[28873]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2024-05-03T14:44:54.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:54 smithi149 ceph-mon[33332]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-03T14:44:54.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:54 smithi149 ceph-mon[33332]: Cluster is now healthy 2024-05-03T14:44:54.953 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:54 smithi060 ceph-mon[28873]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-03T14:44:54.953 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:54 smithi060 ceph-mon[28873]: Cluster is now healthy 2024-05-03T14:44:55.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:54 smithi090 ceph-mon[33568]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-03T14:44:55.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:54 smithi090 ceph-mon[33568]: Cluster is now healthy 2024-05-03T14:44:55.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:55 smithi060 ceph-mon[28873]: pgmap v340: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:55.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:55 smithi149 ceph-mon[33332]: pgmap v340: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:56.026 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:55 smithi090 ceph-mon[33568]: pgmap v340: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:56.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:56.150+0000 7f399c400700 1 -- 172.21.15.60:0/2240757704 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39941023d0 msgr2=0x7f3994102850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:56.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:56.150+0000 7f399c400700 1 --2- 172.21.15.60:0/2240757704 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39941023d0 0x7f3994102850 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3980009990 tx=0x7f3980009ca0 comp rx=0 tx=0).stop 2024-05-03T14:44:56.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:56.150+0000 7f399c400700 1 -- 172.21.15.60:0/2240757704 shutdown_connections 2024-05-03T14:44:56.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:56.150+0000 7f399c400700 1 --2- 172.21.15.60:0/2240757704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3994102d90 0x7f399410f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:56.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:56.150+0000 7f399c400700 1 --2- 172.21.15.60:0/2240757704 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3994108440 0x7f3994108820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:56.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:56.150+0000 7f399c400700 1 --2- 172.21.15.60:0/2240757704 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39941023d0 0x7f3994102850 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:56.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:56.150+0000 7f399c400700 1 -- 172.21.15.60:0/2240757704 >> 172.21.15.60:0/2240757704 conn(0x7f39940793e0 msgr2=0x7f39940797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:44:56.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:56.150+0000 7f399c400700 1 -- 172.21.15.60:0/2240757704 shutdown_connections 2024-05-03T14:44:56.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:56.152+0000 7f399c400700 1 -- 172.21.15.60:0/2240757704 wait complete. 2024-05-03T14:44:56.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:56.153+0000 7f399c400700 1 Processor -- start 2024-05-03T14:44:56.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:56.153+0000 7f399c400700 1 -- start start 2024-05-03T14:44:56.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:56.153+0000 7f399c400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f39941023d0 0x7f39941040b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:44:56.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:56.153+0000 7f399c400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3994102d90 0x7f39941045f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:44:56.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:56.153+0000 7f399c400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3994108440 0x7f3994104b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:44:56.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:56.153+0000 7f399c400700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f3994112840 con 0x7f3994102d90 2024-05-03T14:44:56.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:56.153+0000 7f399c400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f39941126c0 con 0x7f39941023d0 2024-05-03T14:44:56.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:56.153+0000 7f399c400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f39941129c0 con 0x7f3994108440 2024-05-03T14:44:56.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:56.153+0000 7f3999000700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3994102d90 0x7f39941045f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:44:56.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:56.153+0000 7f3999000700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3994102d90 0x7f39941045f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55288/0 (socket says 172.21.15.60:55288) 2024-05-03T14:44:56.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:56.154+0000 7f3999000700 1 -- 172.21.15.60:0/1246108213 learned_addr learned my addr 172.21.15.60:0/1246108213 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:44:56.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:56.154+0000 7f3999a00700 1 --2- 172.21.15.60:0/1246108213 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f39941023d0 0x7f39941040b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:44:56.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:56.154+0000 7f399a400700 1 --2- 172.21.15.60:0/1246108213 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3994108440 0x7f3994104b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:44:56.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:56.154+0000 7f3999a00700 1 -- 172.21.15.60:0/1246108213 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3994108440 msgr2=0x7f3994104b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:56.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:56.154+0000 7f3999a00700 1 --2- 172.21.15.60:0/1246108213 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3994108440 0x7f3994104b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:56.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:56.154+0000 7f3999a00700 1 -- 172.21.15.60:0/1246108213 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3994102d90 msgr2=0x7f39941045f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:56.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:56.154+0000 7f3999a00700 1 --2- 172.21.15.60:0/1246108213 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3994102d90 0x7f39941045f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:56.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:56.154+0000 7f3999a00700 1 -- 172.21.15.60:0/1246108213 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39941a8310 con 0x7f39941023d0 2024-05-03T14:44:56.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:56.154+0000 7f3999000700 1 --2- 172.21.15.60:0/1246108213 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3994102d90 0x7f39941045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:44:56.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:56.154+0000 7f3999a00700 1 --2- 172.21.15.60:0/1246108213 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f39941023d0 0x7f39941040b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f398800e9f0 tx=0x7f398800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:44:56.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:56.155+0000 7f3992000700 1 -- 172.21.15.60:0/1246108213 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f398800c9c0 con 0x7f39941023d0 2024-05-03T14:44:56.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:56.155+0000 7f399c400700 1 -- 172.21.15.60:0/1246108213 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39941a8600 con 0x7f39941023d0 2024-05-03T14:44:56.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:56.155+0000 7f3992000700 1 -- 172.21.15.60:0/1246108213 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f398800cb60 con 0x7f39941023d0 2024-05-03T14:44:56.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:56.155+0000 7f3992000700 1 -- 172.21.15.60:0/1246108213 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3988018660 con 0x7f39941023d0 2024-05-03T14:44:56.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:56.155+0000 7f399c400700 1 -- 172.21.15.60:0/1246108213 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39941a8b90 con 0x7f39941023d0 2024-05-03T14:44:56.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:56.156+0000 7f399c400700 1 -- 172.21.15.60:0/1246108213 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3994103af0 con 0x7f39941023d0 2024-05-03T14:44:56.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:56.158+0000 7f3992000700 1 -- 172.21.15.60:0/1246108213 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f398800cd10 con 0x7f39941023d0 2024-05-03T14:44:56.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:56.160+0000 7f3992000700 1 --2- 172.21.15.60:0/1246108213 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f397c0b2900 0x7f397c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:44:56.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:56.160+0000 7f3992000700 1 -- 172.21.15.60:0/1246108213 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f39880e2990 con 0x7f39941023d0 2024-05-03T14:44:56.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:56.160+0000 7f3999000700 1 --2- 172.21.15.60:0/1246108213 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f397c0b2900 0x7f397c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:44:56.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:56.161+0000 7f3999000700 1 --2- 172.21.15.60:0/1246108213 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f397c0b2900 0x7f397c0b4dc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3994079970 tx=0x7f398001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:44:56.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:56.162+0000 7f3992000700 1 -- 172.21.15.60:0/1246108213 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f39880aa410 con 0x7f39941023d0 2024-05-03T14:44:56.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:56.450+0000 7f399c400700 1 -- 172.21.15.60:0/1246108213 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f399406a190 con 0x7f39941023d0 2024-05-03T14:44:56.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:56.451+0000 7f3992000700 1 -- 172.21.15.60:0/1246108213 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11794 (secure 0 0 0) 0x7f39880a9d10 con 0x7f39941023d0 2024-05-03T14:44:56.453 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:44:56.454 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1849151522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1849151522},{"type":"v1","addr":"172.21.15.60:6835","nonce":1849151522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/523710346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":523710346},{"type":"v1","addr":"172.21.15.60:6837","nonce":523710346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3846205424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3846205424},{"type":"v1","addr":"172.21.15.60:6839","nonce":3846205424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/3033424631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":3033424631},{"type":"v1","addr":"172.21.15.90:6833","nonce":3033424631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/3454021291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3454021291},{"type":"v1","addr":"172.21.15.149:6833","nonce":3454021291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1057049585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1057049585},{"type":"v1","addr":"172.21.15.90:6835","nonce":1057049585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/882239924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":882239924},{"type":"v1","addr":"172.21.15.149:6835","nonce":882239924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/2243702844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":2243702844},{"type":"v1","addr":"172.21.15.90:6837","nonce":2243702844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/2611278690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2611278690},{"type":"v1","addr":"172.21.15.149:6837","nonce":2611278690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6839/2527622212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2527622212},{"type":"v1","addr":"172.21.15.149:6839","nonce":2527622212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24509,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6839/3512381172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3512381172},{"type":"v1","addr":"172.21.15.90:6839","nonce":3512381172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-03T14:44:34.226052+0000","modified":"2024-05-03T14:44:52.471988+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.60:6841/924163182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":924163182},{"type":"v1","addr":"172.21.15.60:6841","nonce":924163182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24511,"qdb_cluster":[24511]},"id":1}]} 2024-05-03T14:44:56.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:56.454+0000 7f399c400700 1 -- 172.21.15.60:0/1246108213 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f397c0b2900 msgr2=0x7f397c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:56.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:56.454+0000 7f399c400700 1 --2- 172.21.15.60:0/1246108213 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f397c0b2900 0x7f397c0b4dc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3994079970 tx=0x7f398001a040 comp rx=0 tx=0).stop 2024-05-03T14:44:56.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:56.454+0000 7f399c400700 1 -- 172.21.15.60:0/1246108213 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f39941023d0 msgr2=0x7f39941040b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:56.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:56.454+0000 7f399c400700 1 --2- 172.21.15.60:0/1246108213 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f39941023d0 0x7f39941040b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f398800e9f0 tx=0x7f398800ed00 comp rx=0 tx=0).stop 2024-05-03T14:44:56.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:56.455+0000 7f399c400700 1 -- 172.21.15.60:0/1246108213 shutdown_connections 2024-05-03T14:44:56.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:56.455+0000 7f399c400700 1 --2- 172.21.15.60:0/1246108213 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f397c0b2900 0x7f397c0b4dc0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:56.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:56.455+0000 7f399c400700 1 --2- 172.21.15.60:0/1246108213 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3994108440 0x7f3994104b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:56.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:56.455+0000 7f399c400700 1 --2- 172.21.15.60:0/1246108213 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f39941023d0 0x7f39941040b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:56.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:56.455+0000 7f399c400700 1 --2- 172.21.15.60:0/1246108213 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3994102d90 0x7f39941045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:56.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:56.455+0000 7f399c400700 1 -- 172.21.15.60:0/1246108213 >> 172.21.15.60:0/1246108213 conn(0x7f39940793e0 msgr2=0x7f3994101640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:44:56.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:56.455+0000 7f399c400700 1 -- 172.21.15.60:0/1246108213 shutdown_connections 2024-05-03T14:44:56.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:56.455+0000 7f399c400700 1 -- 172.21.15.60:0/1246108213 wait complete. 2024-05-03T14:44:56.458 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 17 2024-05-03T14:44:57.259 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-03T14:44:57.335 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:57 smithi060 ceph-mon[28873]: pgmap v341: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:57.335 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:57 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/1246108213' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T14:44:57.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:57 smithi090 ceph-mon[33568]: pgmap v341: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:57.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:57 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/1246108213' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T14:44:57.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:57 smithi149 ceph-mon[33332]: pgmap v341: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:57.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:57 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/1246108213' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T14:44:57.720 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:44:59.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:44:59 smithi060 ceph-mon[28873]: pgmap v342: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:59.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:44:59 smithi149 ceph-mon[33332]: pgmap v342: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:44:59.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:59.837+0000 7f1abb800700 1 -- 172.21.15.60:0/848902409 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ab410ff40 msgr2=0x7f1ab4112330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:59.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:59.837+0000 7f1abb800700 1 --2- 172.21.15.60:0/848902409 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ab410ff40 0x7f1ab4112330 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1aac009f90 tx=0x7f1aac00b330 comp rx=0 tx=0).stop 2024-05-03T14:44:59.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:59.837+0000 7f1abb800700 1 -- 172.21.15.60:0/848902409 shutdown_connections 2024-05-03T14:44:59.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:59.837+0000 7f1abb800700 1 --2- 172.21.15.60:0/848902409 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ab410ff40 0x7f1ab4112330 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:59.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:59.837+0000 7f1abb800700 1 --2- 172.21.15.60:0/848902409 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ab406ba40 0x7f1ab406be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:59.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:59.837+0000 7f1abb800700 1 --2- 172.21.15.60:0/848902409 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1ab406c360 0x7f1ab410fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:59.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:59.837+0000 7f1abb800700 1 -- 172.21.15.60:0/848902409 >> 172.21.15.60:0/848902409 conn(0x7f1ab4079310 msgr2=0x7f1ab4079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:44:59.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:59.837+0000 7f1abb800700 1 -- 172.21.15.60:0/848902409 shutdown_connections 2024-05-03T14:44:59.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:59.837+0000 7f1abb800700 1 -- 172.21.15.60:0/848902409 wait complete. 2024-05-03T14:44:59.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:59.838+0000 7f1abb800700 1 Processor -- start 2024-05-03T14:44:59.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:59.838+0000 7f1abb800700 1 -- start start 2024-05-03T14:44:59.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:59.839+0000 7f1abb800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ab406ba40 0x7f1ab419bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:44:59.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:59.839+0000 7f1abb800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ab406c360 0x7f1ab419c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:44:59.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:59.839+0000 7f1abb800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1ab410ff40 0x7f1ab41a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:44:59.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:59.839+0000 7f1abb800700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab4115b80 con 0x7f1ab406ba40 2024-05-03T14:44:59.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:59.839+0000 7f1abb800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab4115a00 con 0x7f1ab410ff40 2024-05-03T14:44:59.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:59.839+0000 7f1abb800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab4115d00 con 0x7f1ab406c360 2024-05-03T14:44:59.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:59.839+0000 7f1ab8e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ab406ba40 0x7f1ab419bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:44:59.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:59.839+0000 7f1ab8e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ab406ba40 0x7f1ab419bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55312/0 (socket says 172.21.15.60:55312) 2024-05-03T14:44:59.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:59.839+0000 7f1ab8e00700 1 -- 172.21.15.60:0/2316838411 learned_addr learned my addr 172.21.15.60:0/2316838411 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:44:59.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:59.839+0000 7f1aabe00700 1 --2- 172.21.15.60:0/2316838411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ab406c360 0x7f1ab419c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:44:59.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:59.839+0000 7f1ab9800700 1 --2- 172.21.15.60:0/2316838411 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1ab410ff40 0x7f1ab41a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:44:59.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:59.840+0000 7f1ab8e00700 1 -- 172.21.15.60:0/2316838411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ab406c360 msgr2=0x7f1ab419c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:59.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:59.840+0000 7f1ab8e00700 1 --2- 172.21.15.60:0/2316838411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ab406c360 0x7f1ab419c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:59.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:59.840+0000 7f1ab8e00700 1 -- 172.21.15.60:0/2316838411 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1ab410ff40 msgr2=0x7f1ab41a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:44:59.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:59.840+0000 7f1ab8e00700 1 --2- 172.21.15.60:0/2316838411 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1ab410ff40 0x7f1ab41a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:44:59.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:59.840+0000 7f1ab8e00700 1 -- 172.21.15.60:0/2316838411 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ab41a0f20 con 0x7f1ab406ba40 2024-05-03T14:44:59.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:59.840+0000 7f1ab8e00700 1 --2- 172.21.15.60:0/2316838411 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ab406ba40 0x7f1ab419bf10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1a9800e9c0 tx=0x7f1a9800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:44:59.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:59.840+0000 7f1ab2000700 1 -- 172.21.15.60:0/2316838411 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9800cbd0 con 0x7f1ab406ba40 2024-05-03T14:44:59.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:59.840+0000 7f1ab2000700 1 -- 172.21.15.60:0/2316838411 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f1a9800ee60 con 0x7f1ab406ba40 2024-05-03T14:44:59.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:59.840+0000 7f1abb800700 1 -- 172.21.15.60:0/2316838411 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ab41a1210 con 0x7f1ab406ba40 2024-05-03T14:44:59.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:59.840+0000 7f1ab2000700 1 -- 172.21.15.60:0/2316838411 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a98010430 con 0x7f1ab406ba40 2024-05-03T14:44:59.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:59.840+0000 7f1abb800700 1 -- 172.21.15.60:0/2316838411 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ab41a8a90 con 0x7f1ab406ba40 2024-05-03T14:44:59.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:59.842+0000 7f1abb800700 1 -- 172.21.15.60:0/2316838411 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ab406a190 con 0x7f1ab406ba40 2024-05-03T14:44:59.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:59.844+0000 7f1ab2000700 1 -- 172.21.15.60:0/2316838411 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f1a98004750 con 0x7f1ab406ba40 2024-05-03T14:44:59.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:59.846+0000 7f1ab2000700 1 --2- 172.21.15.60:0/2316838411 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f1a900b2880 0x7f1a900b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:44:59.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:59.846+0000 7f1ab2000700 1 -- 172.21.15.60:0/2316838411 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f1a980e2950 con 0x7f1ab406ba40 2024-05-03T14:44:59.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:59.846+0000 7f1aabe00700 1 --2- 172.21.15.60:0/2316838411 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f1a900b2880 0x7f1a900b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:44:59.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:59.847+0000 7f1aabe00700 1 --2- 172.21.15.60:0/2316838411 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f1a900b2880 0x7f1a900b4d40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0009750 tx=0x7f1aa0009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:44:59.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:44:59.848+0000 7f1ab2000700 1 -- 172.21.15.60:0/2316838411 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f1a98054d70 con 0x7f1ab406ba40 2024-05-03T14:44:59.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:44:59 smithi090 ceph-mon[33568]: pgmap v342: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:45:00.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:00.086+0000 7f1abb800700 1 -- 172.21.15.60:0/2316838411 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1ab419d270 con 0x7f1ab406ba40 2024-05-03T14:45:00.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:00.091+0000 7f1ab2000700 1 -- 172.21.15.60:0/2316838411 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+24100 (secure 0 0 0) 0x7f1a980aa3d0 con 0x7f1ab406ba40 2024-05-03T14:45:00.093 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:45:00.094 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":85,"fsid":"285d64b2-095a-11ef-bc97-c7b262605968","created":"2024-05-03T14:34:53.560460+0000","modified":"2024-05-03T14:44:52.471819+0000","last_up_change":"2024-05-03T14:43:14.877043+0000","last_in_change":"2024-05-03T14:42:55.085959+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-03T14:38:54.333714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-03T14:44:25.608599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-03T14:44:30.154009+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-03T14:44:42.549502+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f0797730-c4c1-47ea-9dce-9270d5ebb4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2952930162},{"type":"v1","addr":"172.21.15.60:6803","nonce":2952930162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2952930162},{"type":"v1","addr":"172.21.15.60:6805","nonce":2952930162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":2952930162},{"type":"v1","addr":"172.21.15.60:6809","nonce":2952930162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2952930162},{"type":"v1","addr":"172.21.15.60:6807","nonce":2952930162}]},"public_addr":"172.21.15.60:6803/2952930162","cluster_addr":"172.21.15.60:6805/2952930162","heartbeat_back_addr":"172.21.15.60:6809/2952930162","heartbeat_front_addr":"172.21.15.60:6807/2952930162","state":["exists","up"]},{"osd":1,"uuid":"96d0ad08-cda4-4824-bad4-e64973508c0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":720063242},{"type":"v1","addr":"172.21.15.90:6801","nonce":720063242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":720063242},{"type":"v1","addr":"172.21.15.90:6803","nonce":720063242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":720063242},{"type":"v1","addr":"172.21.15.90:6807","nonce":720063242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":720063242},{"type":"v1","addr":"172.21.15.90:6805","nonce":720063242}]},"public_addr":"172.21.15.90:6801/720063242","cluster_addr":"172.21.15.90:6803/720063242","heartbeat_back_addr":"172.21.15.90:6807/720063242","heartbeat_front_addr":"172.21.15.90:6805/720063242","state":["exists","up"]},{"osd":2,"uuid":"ce95fb7c-1611-4dd1-aee7-97c420d6a466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1359662077},{"type":"v1","addr":"172.21.15.149:6801","nonce":1359662077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1359662077},{"type":"v1","addr":"172.21.15.149:6803","nonce":1359662077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1359662077},{"type":"v1","addr":"172.21.15.149:6807","nonce":1359662077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1359662077},{"type":"v1","addr":"172.21.15.149:6805","nonce":1359662077}]},"public_addr":"172.21.15.149:6801/1359662077","cluster_addr":"172.21.15.149:6803/1359662077","heartbeat_back_addr":"172.21.15.149:6807/1359662077","heartbeat_front_addr":"172.21.15.149:6805/1359662077","state":["exists","up"]},{"osd":3,"uuid":"ef31136a-9b36-4b87-8389-f53d311bc5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3226133334},{"type":"v1","addr":"172.21.15.60:6811","nonce":3226133334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3226133334},{"type":"v1","addr":"172.21.15.60:6813","nonce":3226133334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3226133334},{"type":"v1","addr":"172.21.15.60:6817","nonce":3226133334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3226133334},{"type":"v1","addr":"172.21.15.60:6815","nonce":3226133334}]},"public_addr":"172.21.15.60:6811/3226133334","cluster_addr":"172.21.15.60:6813/3226133334","heartbeat_back_addr":"172.21.15.60:6817/3226133334","heartbeat_front_addr":"172.21.15.60:6815/3226133334","state":["exists","up"]},{"osd":4,"uuid":"73db7408-a2ff-47e9-b13d-d31688a93610","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2201391883},{"type":"v1","addr":"172.21.15.90:6809","nonce":2201391883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":2201391883},{"type":"v1","addr":"172.21.15.90:6811","nonce":2201391883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":2201391883},{"type":"v1","addr":"172.21.15.90:6815","nonce":2201391883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":2201391883},{"type":"v1","addr":"172.21.15.90:6813","nonce":2201391883}]},"public_addr":"172.21.15.90:6809/2201391883","cluster_addr":"172.21.15.90:6811/2201391883","heartbeat_back_addr":"172.21.15.90:6815/2201391883","heartbeat_front_addr":"172.21.15.90:6813/2201391883","state":["exists","up"]},{"osd":5,"uuid":"aefb78a2-522a-4b35-92b8-e5c04fa47b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2139732836},{"type":"v1","addr":"172.21.15.149:6809","nonce":2139732836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":2139732836},{"type":"v1","addr":"172.21.15.149:6811","nonce":2139732836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":2139732836},{"type":"v1","addr":"172.21.15.149:6815","nonce":2139732836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":2139732836},{"type":"v1","addr":"172.21.15.149:6813","nonce":2139732836}]},"public_addr":"172.21.15.149:6809/2139732836","cluster_addr":"172.21.15.149:6811/2139732836","heartbeat_back_addr":"172.21.15.149:6815/2139732836","heartbeat_front_addr":"172.21.15.149:6813/2139732836","state":["exists","up"]},{"osd":6,"uuid":"7c887819-6182-4636-8cfd-d845c30bd13c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3738521002},{"type":"v1","addr":"172.21.15.60:6819","nonce":3738521002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3738521002},{"type":"v1","addr":"172.21.15.60:6821","nonce":3738521002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3738521002},{"type":"v1","addr":"172.21.15.60:6825","nonce":3738521002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3738521002},{"type":"v1","addr":"172.21.15.60:6823","nonce":3738521002}]},"public_addr":"172.21.15.60:6819/3738521002","cluster_addr":"172.21.15.60:6821/3738521002","heartbeat_back_addr":"172.21.15.60:6825/3738521002","heartbeat_front_addr":"172.21.15.60:6823/3738521002","state":["exists","up"]},{"osd":7,"uuid":"ebce6acf-8873-4b35-bd5f-80de2163a307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":1686298661},{"type":"v1","addr":"172.21.15.90:6817","nonce":1686298661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":1686298661},{"type":"v1","addr":"172.21.15.90:6819","nonce":1686298661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":1686298661},{"type":"v1","addr":"172.21.15.90:6823","nonce":1686298661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1686298661},{"type":"v1","addr":"172.21.15.90:6821","nonce":1686298661}]},"public_addr":"172.21.15.90:6817/1686298661","cluster_addr":"172.21.15.90:6819/1686298661","heartbeat_back_addr":"172.21.15.90:6823/1686298661","heartbeat_front_addr":"172.21.15.90:6821/1686298661","state":["exists","up"]},{"osd":8,"uuid":"4ae04b98-e9b2-4956-83d9-17e5ac3c963f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":2608348248},{"type":"v1","addr":"172.21.15.149:6817","nonce":2608348248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2608348248},{"type":"v1","addr":"172.21.15.149:6819","nonce":2608348248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2608348248},{"type":"v1","addr":"172.21.15.149:6823","nonce":2608348248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2608348248},{"type":"v1","addr":"172.21.15.149:6821","nonce":2608348248}]},"public_addr":"172.21.15.149:6817/2608348248","cluster_addr":"172.21.15.149:6819/2608348248","heartbeat_back_addr":"172.21.15.149:6823/2608348248","heartbeat_front_addr":"172.21.15.149:6821/2608348248","state":["exists","up"]},{"osd":9,"uuid":"d0c0a70d-aa76-47e4-91e8-c343c8c41197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":2156130999},{"type":"v1","addr":"172.21.15.60:6827","nonce":2156130999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":2156130999},{"type":"v1","addr":"172.21.15.60:6829","nonce":2156130999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":2156130999},{"type":"v1","addr":"172.21.15.60:6833","nonce":2156130999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":2156130999},{"type":"v1","addr":"172.21.15.60:6831","nonce":2156130999}]},"public_addr":"172.21.15.60:6827/2156130999","cluster_addr":"172.21.15.60:6829/2156130999","heartbeat_back_addr":"172.21.15.60:6833/2156130999","heartbeat_front_addr":"172.21.15.60:6831/2156130999","state":["exists","up"]},{"osd":10,"uuid":"cdc3d7b8-33c7-4cc4-be4e-6887d15a3d8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":2492593590},{"type":"v1","addr":"172.21.15.90:6825","nonce":2492593590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":2492593590},{"type":"v1","addr":"172.21.15.90:6827","nonce":2492593590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":2492593590},{"type":"v1","addr":"172.21.15.90:6831","nonce":2492593590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":2492593590},{"type":"v1","addr":"172.21.15.90:6829","nonce":2492593590}]},"public_addr":"172.21.15.90:6825/2492593590","cluster_addr":"172.21.15.90:6827/2492593590","heartbeat_back_addr":"172.21.15.90:6831/2492593590","heartbeat_front_addr":"172.21.15.90:6829/2492593590","state":["exists","up"]},{"osd":11,"uuid":"8d17ec1a-bdd5-41f1-9b59-c2b6a527bfbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":1063245126},{"type":"v1","addr":"172.21.15.149:6825","nonce":1063245126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":1063245126},{"type":"v1","addr":"172.21.15.149:6827","nonce":1063245126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":1063245126},{"type":"v1","addr":"172.21.15.149:6831","nonce":1063245126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":1063245126},{"type":"v1","addr":"172.21.15.149:6829","nonce":1063245126}]},"public_addr":"172.21.15.149:6825/1063245126","cluster_addr":"172.21.15.149:6827/1063245126","heartbeat_back_addr":"172.21.15.149:6831/1063245126","heartbeat_front_addr":"172.21.15.149:6829/1063245126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:38:02.056563+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:38:27.140677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:38:51.287070+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:39:25.484410+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:39:51.483133+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:40:17.200423+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:40:51.979804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:41:18.742542+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:41:44.922541+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:42:19.778451+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:42:47.438440+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:43:13.429419+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6801/424391417":"2024-05-04T14:36:04.154333+0000","172.21.15.60:6800/424391417":"2024-05-04T14:36:04.154333+0000","172.21.15.60:0/1426532561":"2024-05-04T14:36:04.154333+0000","172.21.15.60:0/3230079926":"2024-05-04T14:35:28.066961+0000","172.21.15.60:0/2334895610":"2024-05-04T14:35:28.066961+0000","172.21.15.60:6800/888447185":"2024-05-04T14:35:28.066961+0000","172.21.15.60:0/3996899937":"2024-05-04T14:35:28.066961+0000","172.21.15.60:0/2779644018":"2024-05-04T14:36:04.154333+0000","172.21.15.60:0/71440838":"2024-05-04T14:36:04.154333+0000","172.21.15.60:6801/888447185":"2024-05-04T14:35:28.066961+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-03T14:45:00.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:00.094+0000 7f1abb800700 1 -- 172.21.15.60:0/2316838411 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f1a900b2880 msgr2=0x7f1a900b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:00.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:00.094+0000 7f1abb800700 1 --2- 172.21.15.60:0/2316838411 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f1a900b2880 0x7f1a900b4d40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0009750 tx=0x7f1aa0009310 comp rx=0 tx=0).stop 2024-05-03T14:45:00.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:00.095+0000 7f1abb800700 1 -- 172.21.15.60:0/2316838411 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ab406ba40 msgr2=0x7f1ab419bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:00.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:00.095+0000 7f1abb800700 1 --2- 172.21.15.60:0/2316838411 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ab406ba40 0x7f1ab419bf10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1a9800e9c0 tx=0x7f1a9800ecd0 comp rx=0 tx=0).stop 2024-05-03T14:45:00.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:00.095+0000 7f1abb800700 1 -- 172.21.15.60:0/2316838411 shutdown_connections 2024-05-03T14:45:00.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:00.095+0000 7f1abb800700 1 --2- 172.21.15.60:0/2316838411 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f1a900b2880 0x7f1a900b4d40 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:00.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:00.095+0000 7f1abb800700 1 --2- 172.21.15.60:0/2316838411 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1ab410ff40 0x7f1ab41a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:00.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:00.095+0000 7f1abb800700 1 --2- 172.21.15.60:0/2316838411 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ab406ba40 0x7f1ab419bf10 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:00.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:00.095+0000 7f1abb800700 1 --2- 172.21.15.60:0/2316838411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ab406c360 0x7f1ab419c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:00.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:00.095+0000 7f1abb800700 1 -- 172.21.15.60:0/2316838411 >> 172.21.15.60:0/2316838411 conn(0x7f1ab4079310 msgr2=0x7f1ab4111350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:45:00.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:00.095+0000 7f1abb800700 1 -- 172.21.15.60:0/2316838411 shutdown_connections 2024-05-03T14:45:00.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:00.095+0000 7f1abb800700 1 -- 172.21.15.60:0/2316838411 wait complete. 2024-05-03T14:45:00.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:00 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/2316838411' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T14:45:00.792 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:00 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/2316838411' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T14:45:00.849 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-03T14:45:00.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:00 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/2316838411' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T14:45:01.328 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:45:01.667 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:01 smithi060 ceph-mon[28873]: pgmap v343: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:45:01.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:01 smithi149 ceph-mon[33332]: pgmap v343: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:45:01.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:01 smithi090 ceph-mon[33568]: pgmap v343: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:45:03.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:03.372+0000 7fd41f200700 1 -- 172.21.15.60:0/1907509930 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd418104100 msgr2=0x7fd418108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:03.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:03.372+0000 7fd41f200700 1 --2- 172.21.15.60:0/1907509930 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd418104100 0x7fd418108120 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd400009920 tx=0x7fd400009c30 comp rx=0 tx=0).stop 2024-05-03T14:45:03.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:03.372+0000 7fd41f200700 1 -- 172.21.15.60:0/1907509930 shutdown_connections 2024-05-03T14:45:03.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:03.372+0000 7fd41f200700 1 --2- 172.21.15.60:0/1907509930 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd418108850 0x7fd4181103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:03.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:03.372+0000 7fd41f200700 1 --2- 172.21.15.60:0/1907509930 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd418103750 0x7fd418103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:03.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:03.372+0000 7fd41f200700 1 --2- 172.21.15.60:0/1907509930 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd418104100 0x7fd418108120 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:03.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:03.372+0000 7fd41f200700 1 -- 172.21.15.60:0/1907509930 >> 172.21.15.60:0/1907509930 conn(0x7fd41807a600 msgr2=0x7fd41807aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:45:03.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:03.372+0000 7fd41f200700 1 -- 172.21.15.60:0/1907509930 shutdown_connections 2024-05-03T14:45:03.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:03.372+0000 7fd41f200700 1 -- 172.21.15.60:0/1907509930 wait complete. 2024-05-03T14:45:03.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:03.374+0000 7fd41f200700 1 Processor -- start 2024-05-03T14:45:03.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:03.374+0000 7fd41f200700 1 -- start start 2024-05-03T14:45:03.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:03.375+0000 7fd41f200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd418103750 0x7fd4180757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:45:03.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:03.375+0000 7fd41f200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd418104100 0x7fd418075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:45:03.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:03.375+0000 7fd41f200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd418108850 0x7fd418076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:45:03.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:03.375+0000 7fd41f200700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd4181138a0 con 0x7fd418104100 2024-05-03T14:45:03.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:03.375+0000 7fd41f200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd418113720 con 0x7fd418103750 2024-05-03T14:45:03.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:03.375+0000 7fd41f200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd418113a20 con 0x7fd418108850 2024-05-03T14:45:03.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:03.375+0000 7fd40fe00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd418104100 0x7fd418075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:45:03.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:03.375+0000 7fd417e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd418103750 0x7fd4180757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:45:03.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:03.375+0000 7fd40fe00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd418104100 0x7fd418075ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43632/0 (socket says 172.21.15.60:43632) 2024-05-03T14:45:03.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:03.375+0000 7fd40fe00700 1 -- 172.21.15.60:0/4157993779 learned_addr learned my addr 172.21.15.60:0/4157993779 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:45:03.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:03.375+0000 7fd41d200700 1 --2- 172.21.15.60:0/4157993779 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd418108850 0x7fd418076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:45:03.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:03.376+0000 7fd417e00700 1 -- 172.21.15.60:0/4157993779 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd418108850 msgr2=0x7fd418076220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:03.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:03.376+0000 7fd417e00700 1 --2- 172.21.15.60:0/4157993779 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd418108850 0x7fd418076220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:03.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:03.376+0000 7fd417e00700 1 -- 172.21.15.60:0/4157993779 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd418104100 msgr2=0x7fd418075ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:03.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:03.376+0000 7fd417e00700 1 --2- 172.21.15.60:0/4157993779 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd418104100 0x7fd418075ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:03.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:03.376+0000 7fd417e00700 1 -- 172.21.15.60:0/4157993779 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4181a8280 con 0x7fd418103750 2024-05-03T14:45:03.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:03.376+0000 7fd40fe00700 1 --2- 172.21.15.60:0/4157993779 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd418104100 0x7fd418075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:45:03.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:03.376+0000 7fd41d200700 1 --2- 172.21.15.60:0/4157993779 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd418108850 0x7fd418076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:45:03.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:03.376+0000 7fd417e00700 1 --2- 172.21.15.60:0/4157993779 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd418103750 0x7fd4180757a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd40800eb90 tx=0x7fd40800eea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:45:03.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:03.376+0000 7fd415600700 1 -- 172.21.15.60:0/4157993779 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40800cc90 con 0x7fd418103750 2024-05-03T14:45:03.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:03.376+0000 7fd415600700 1 -- 172.21.15.60:0/4157993779 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7fd408004520 con 0x7fd418103750 2024-05-03T14:45:03.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:03.376+0000 7fd41f200700 1 -- 172.21.15.60:0/4157993779 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4181a8570 con 0x7fd418103750 2024-05-03T14:45:03.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:03.376+0000 7fd41f200700 1 -- 172.21.15.60:0/4157993779 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4181a8b00 con 0x7fd418103750 2024-05-03T14:45:03.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:03.376+0000 7fd415600700 1 -- 172.21.15.60:0/4157993779 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd408010690 con 0x7fd418103750 2024-05-03T14:45:03.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:03.378+0000 7fd41f200700 1 -- 172.21.15.60:0/4157993779 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd41806a190 con 0x7fd418103750 2024-05-03T14:45:03.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:03.380+0000 7fd415600700 1 -- 172.21.15.60:0/4157993779 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fd408003e50 con 0x7fd418103750 2024-05-03T14:45:03.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:03.381+0000 7fd415600700 1 --2- 172.21.15.60:0/4157993779 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fd3fc0b2900 0x7fd3fc0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:45:03.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:03.382+0000 7fd415600700 1 -- 172.21.15.60:0/4157993779 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7fd4080e2780 con 0x7fd418103750 2024-05-03T14:45:03.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:03.382+0000 7fd40fe00700 1 --2- 172.21.15.60:0/4157993779 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fd3fc0b2900 0x7fd3fc0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:45:03.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:03.382+0000 7fd40fe00700 1 --2- 172.21.15.60:0/4157993779 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fd3fc0b2900 0x7fd3fc0b4dc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd418103100 tx=0x7fd400019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:45:03.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:03.384+0000 7fd415600700 1 -- 172.21.15.60:0/4157993779 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fd4080aa200 con 0x7fd418103750 2024-05-03T14:45:03.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:03 smithi060 ceph-mon[28873]: pgmap v344: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:45:03.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:03.632+0000 7fd41f200700 1 -- 172.21.15.60:0/4157993779 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd41802e510 con 0x7fd418103750 2024-05-03T14:45:03.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:03.634+0000 7fd415600700 1 -- 172.21.15.60:0/4157993779 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+24100 (secure 0 0 0) 0x7fd4080a9b00 con 0x7fd418103750 2024-05-03T14:45:03.637 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:45:03.637 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":85,"fsid":"285d64b2-095a-11ef-bc97-c7b262605968","created":"2024-05-03T14:34:53.560460+0000","modified":"2024-05-03T14:44:52.471819+0000","last_up_change":"2024-05-03T14:43:14.877043+0000","last_in_change":"2024-05-03T14:42:55.085959+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-03T14:38:54.333714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-03T14:44:25.608599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-03T14:44:30.154009+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-03T14:44:42.549502+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f0797730-c4c1-47ea-9dce-9270d5ebb4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2952930162},{"type":"v1","addr":"172.21.15.60:6803","nonce":2952930162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2952930162},{"type":"v1","addr":"172.21.15.60:6805","nonce":2952930162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":2952930162},{"type":"v1","addr":"172.21.15.60:6809","nonce":2952930162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2952930162},{"type":"v1","addr":"172.21.15.60:6807","nonce":2952930162}]},"public_addr":"172.21.15.60:6803/2952930162","cluster_addr":"172.21.15.60:6805/2952930162","heartbeat_back_addr":"172.21.15.60:6809/2952930162","heartbeat_front_addr":"172.21.15.60:6807/2952930162","state":["exists","up"]},{"osd":1,"uuid":"96d0ad08-cda4-4824-bad4-e64973508c0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":720063242},{"type":"v1","addr":"172.21.15.90:6801","nonce":720063242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":720063242},{"type":"v1","addr":"172.21.15.90:6803","nonce":720063242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":720063242},{"type":"v1","addr":"172.21.15.90:6807","nonce":720063242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":720063242},{"type":"v1","addr":"172.21.15.90:6805","nonce":720063242}]},"public_addr":"172.21.15.90:6801/720063242","cluster_addr":"172.21.15.90:6803/720063242","heartbeat_back_addr":"172.21.15.90:6807/720063242","heartbeat_front_addr":"172.21.15.90:6805/720063242","state":["exists","up"]},{"osd":2,"uuid":"ce95fb7c-1611-4dd1-aee7-97c420d6a466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1359662077},{"type":"v1","addr":"172.21.15.149:6801","nonce":1359662077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1359662077},{"type":"v1","addr":"172.21.15.149:6803","nonce":1359662077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1359662077},{"type":"v1","addr":"172.21.15.149:6807","nonce":1359662077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1359662077},{"type":"v1","addr":"172.21.15.149:6805","nonce":1359662077}]},"public_addr":"172.21.15.149:6801/1359662077","cluster_addr":"172.21.15.149:6803/1359662077","heartbeat_back_addr":"172.21.15.149:6807/1359662077","heartbeat_front_addr":"172.21.15.149:6805/1359662077","state":["exists","up"]},{"osd":3,"uuid":"ef31136a-9b36-4b87-8389-f53d311bc5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3226133334},{"type":"v1","addr":"172.21.15.60:6811","nonce":3226133334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3226133334},{"type":"v1","addr":"172.21.15.60:6813","nonce":3226133334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3226133334},{"type":"v1","addr":"172.21.15.60:6817","nonce":3226133334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3226133334},{"type":"v1","addr":"172.21.15.60:6815","nonce":3226133334}]},"public_addr":"172.21.15.60:6811/3226133334","cluster_addr":"172.21.15.60:6813/3226133334","heartbeat_back_addr":"172.21.15.60:6817/3226133334","heartbeat_front_addr":"172.21.15.60:6815/3226133334","state":["exists","up"]},{"osd":4,"uuid":"73db7408-a2ff-47e9-b13d-d31688a93610","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2201391883},{"type":"v1","addr":"172.21.15.90:6809","nonce":2201391883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":2201391883},{"type":"v1","addr":"172.21.15.90:6811","nonce":2201391883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":2201391883},{"type":"v1","addr":"172.21.15.90:6815","nonce":2201391883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":2201391883},{"type":"v1","addr":"172.21.15.90:6813","nonce":2201391883}]},"public_addr":"172.21.15.90:6809/2201391883","cluster_addr":"172.21.15.90:6811/2201391883","heartbeat_back_addr":"172.21.15.90:6815/2201391883","heartbeat_front_addr":"172.21.15.90:6813/2201391883","state":["exists","up"]},{"osd":5,"uuid":"aefb78a2-522a-4b35-92b8-e5c04fa47b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2139732836},{"type":"v1","addr":"172.21.15.149:6809","nonce":2139732836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":2139732836},{"type":"v1","addr":"172.21.15.149:6811","nonce":2139732836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":2139732836},{"type":"v1","addr":"172.21.15.149:6815","nonce":2139732836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":2139732836},{"type":"v1","addr":"172.21.15.149:6813","nonce":2139732836}]},"public_addr":"172.21.15.149:6809/2139732836","cluster_addr":"172.21.15.149:6811/2139732836","heartbeat_back_addr":"172.21.15.149:6815/2139732836","heartbeat_front_addr":"172.21.15.149:6813/2139732836","state":["exists","up"]},{"osd":6,"uuid":"7c887819-6182-4636-8cfd-d845c30bd13c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3738521002},{"type":"v1","addr":"172.21.15.60:6819","nonce":3738521002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3738521002},{"type":"v1","addr":"172.21.15.60:6821","nonce":3738521002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3738521002},{"type":"v1","addr":"172.21.15.60:6825","nonce":3738521002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3738521002},{"type":"v1","addr":"172.21.15.60:6823","nonce":3738521002}]},"public_addr":"172.21.15.60:6819/3738521002","cluster_addr":"172.21.15.60:6821/3738521002","heartbeat_back_addr":"172.21.15.60:6825/3738521002","heartbeat_front_addr":"172.21.15.60:6823/3738521002","state":["exists","up"]},{"osd":7,"uuid":"ebce6acf-8873-4b35-bd5f-80de2163a307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":1686298661},{"type":"v1","addr":"172.21.15.90:6817","nonce":1686298661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":1686298661},{"type":"v1","addr":"172.21.15.90:6819","nonce":1686298661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":1686298661},{"type":"v1","addr":"172.21.15.90:6823","nonce":1686298661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1686298661},{"type":"v1","addr":"172.21.15.90:6821","nonce":1686298661}]},"public_addr":"172.21.15.90:6817/1686298661","cluster_addr":"172.21.15.90:6819/1686298661","heartbeat_back_addr":"172.21.15.90:6823/1686298661","heartbeat_front_addr":"172.21.15.90:6821/1686298661","state":["exists","up"]},{"osd":8,"uuid":"4ae04b98-e9b2-4956-83d9-17e5ac3c963f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":2608348248},{"type":"v1","addr":"172.21.15.149:6817","nonce":2608348248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2608348248},{"type":"v1","addr":"172.21.15.149:6819","nonce":2608348248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2608348248},{"type":"v1","addr":"172.21.15.149:6823","nonce":2608348248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2608348248},{"type":"v1","addr":"172.21.15.149:6821","nonce":2608348248}]},"public_addr":"172.21.15.149:6817/2608348248","cluster_addr":"172.21.15.149:6819/2608348248","heartbeat_back_addr":"172.21.15.149:6823/2608348248","heartbeat_front_addr":"172.21.15.149:6821/2608348248","state":["exists","up"]},{"osd":9,"uuid":"d0c0a70d-aa76-47e4-91e8-c343c8c41197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":2156130999},{"type":"v1","addr":"172.21.15.60:6827","nonce":2156130999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":2156130999},{"type":"v1","addr":"172.21.15.60:6829","nonce":2156130999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":2156130999},{"type":"v1","addr":"172.21.15.60:6833","nonce":2156130999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":2156130999},{"type":"v1","addr":"172.21.15.60:6831","nonce":2156130999}]},"public_addr":"172.21.15.60:6827/2156130999","cluster_addr":"172.21.15.60:6829/2156130999","heartbeat_back_addr":"172.21.15.60:6833/2156130999","heartbeat_front_addr":"172.21.15.60:6831/2156130999","state":["exists","up"]},{"osd":10,"uuid":"cdc3d7b8-33c7-4cc4-be4e-6887d15a3d8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":2492593590},{"type":"v1","addr":"172.21.15.90:6825","nonce":2492593590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":2492593590},{"type":"v1","addr":"172.21.15.90:6827","nonce":2492593590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":2492593590},{"type":"v1","addr":"172.21.15.90:6831","nonce":2492593590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":2492593590},{"type":"v1","addr":"172.21.15.90:6829","nonce":2492593590}]},"public_addr":"172.21.15.90:6825/2492593590","cluster_addr":"172.21.15.90:6827/2492593590","heartbeat_back_addr":"172.21.15.90:6831/2492593590","heartbeat_front_addr":"172.21.15.90:6829/2492593590","state":["exists","up"]},{"osd":11,"uuid":"8d17ec1a-bdd5-41f1-9b59-c2b6a527bfbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":1063245126},{"type":"v1","addr":"172.21.15.149:6825","nonce":1063245126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":1063245126},{"type":"v1","addr":"172.21.15.149:6827","nonce":1063245126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":1063245126},{"type":"v1","addr":"172.21.15.149:6831","nonce":1063245126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":1063245126},{"type":"v1","addr":"172.21.15.149:6829","nonce":1063245126}]},"public_addr":"172.21.15.149:6825/1063245126","cluster_addr":"172.21.15.149:6827/1063245126","heartbeat_back_addr":"172.21.15.149:6831/1063245126","heartbeat_front_addr":"172.21.15.149:6829/1063245126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:38:02.056563+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:38:27.140677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:38:51.287070+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:39:25.484410+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:39:51.483133+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:40:17.200423+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:40:51.979804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:41:18.742542+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:41:44.922541+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:42:19.778451+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:42:47.438440+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:43:13.429419+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6801/424391417":"2024-05-04T14:36:04.154333+0000","172.21.15.60:6800/424391417":"2024-05-04T14:36:04.154333+0000","172.21.15.60:0/1426532561":"2024-05-04T14:36:04.154333+0000","172.21.15.60:0/3230079926":"2024-05-04T14:35:28.066961+0000","172.21.15.60:0/2334895610":"2024-05-04T14:35:28.066961+0000","172.21.15.60:6800/888447185":"2024-05-04T14:35:28.066961+0000","172.21.15.60:0/3996899937":"2024-05-04T14:35:28.066961+0000","172.21.15.60:0/2779644018":"2024-05-04T14:36:04.154333+0000","172.21.15.60:0/71440838":"2024-05-04T14:36:04.154333+0000","172.21.15.60:6801/888447185":"2024-05-04T14:35:28.066961+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-03T14:45:03.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:03.637+0000 7fd41f200700 1 -- 172.21.15.60:0/4157993779 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fd3fc0b2900 msgr2=0x7fd3fc0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:03.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:03.637+0000 7fd41f200700 1 --2- 172.21.15.60:0/4157993779 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fd3fc0b2900 0x7fd3fc0b4dc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd418103100 tx=0x7fd400019040 comp rx=0 tx=0).stop 2024-05-03T14:45:03.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:03.638+0000 7fd41f200700 1 -- 172.21.15.60:0/4157993779 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd418103750 msgr2=0x7fd4180757a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:03.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:03.638+0000 7fd41f200700 1 --2- 172.21.15.60:0/4157993779 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd418103750 0x7fd4180757a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd40800eb90 tx=0x7fd40800eea0 comp rx=0 tx=0).stop 2024-05-03T14:45:03.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:03.638+0000 7fd41f200700 1 -- 172.21.15.60:0/4157993779 shutdown_connections 2024-05-03T14:45:03.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:03.638+0000 7fd41f200700 1 --2- 172.21.15.60:0/4157993779 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fd3fc0b2900 0x7fd3fc0b4dc0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:03.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:03.638+0000 7fd41f200700 1 --2- 172.21.15.60:0/4157993779 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd418108850 0x7fd418076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:03.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:03.638+0000 7fd41f200700 1 --2- 172.21.15.60:0/4157993779 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd418103750 0x7fd4180757a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:03.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:03.638+0000 7fd41f200700 1 --2- 172.21.15.60:0/4157993779 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd418104100 0x7fd418075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:03.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:03.638+0000 7fd41f200700 1 -- 172.21.15.60:0/4157993779 >> 172.21.15.60:0/4157993779 conn(0x7fd41807a600 msgr2=0x7fd418107350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:45:03.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:03.638+0000 7fd41f200700 1 -- 172.21.15.60:0/4157993779 shutdown_connections 2024-05-03T14:45:03.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:03.638+0000 7fd41f200700 1 -- 172.21.15.60:0/4157993779 wait complete. 2024-05-03T14:45:03.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:03 smithi149 ceph-mon[33332]: pgmap v344: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:45:03.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:03 smithi090 ceph-mon[33568]: pgmap v344: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:45:04.447 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T14:45:04.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:04 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/4157993779' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T14:45:04.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:04 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-05-03T14:45:04.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:04 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-05-03T14:45:04.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:04 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]: dispatch 2024-05-03T14:45:04.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:04 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2024-05-03T14:45:04.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:04 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-05-03T14:45:04.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:04 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-05-03T14:45:04.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:04 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2024-05-03T14:45:04.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:04 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2024-05-03T14:45:04.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:04 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2024-05-03T14:45:04.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:04 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]: dispatch 2024-05-03T14:45:04.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:04 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/4157993779' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T14:45:04.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:04 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-05-03T14:45:04.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:04 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-05-03T14:45:04.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:04 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]: dispatch 2024-05-03T14:45:04.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:04 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2024-05-03T14:45:04.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:04 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-05-03T14:45:04.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:04 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-05-03T14:45:04.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:04 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2024-05-03T14:45:04.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:04 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2024-05-03T14:45:04.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:04 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2024-05-03T14:45:04.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:04 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]: dispatch 2024-05-03T14:45:04.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:04 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/4157993779' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T14:45:04.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:04 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-05-03T14:45:04.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:04 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-05-03T14:45:04.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:04 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]: dispatch 2024-05-03T14:45:04.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:04 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2024-05-03T14:45:04.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:04 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-05-03T14:45:04.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:04 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-05-03T14:45:04.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:04 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2024-05-03T14:45:04.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:04 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2024-05-03T14:45:04.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:04 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2024-05-03T14:45:04.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:04 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]: dispatch 2024-05-03T14:45:04.887 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:45:05.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:05 smithi149 ceph-mon[33332]: pgmap v345: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:45:05.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:05 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-05-03T14:45:05.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:05 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-05-03T14:45:05.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:05 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]': finished 2024-05-03T14:45:05.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:05 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2024-05-03T14:45:05.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:05 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-05-03T14:45:05.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:05 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-05-03T14:45:05.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:05 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-05-03T14:45:05.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:05 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-05-03T14:45:05.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:05 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-05-03T14:45:05.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:05 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-05-03T14:45:05.674 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:05 smithi149 ceph-mon[33332]: osdmap e86: 12 total, 12 up, 12 in 2024-05-03T14:45:05.742 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:05 smithi060 ceph-mon[28873]: pgmap v345: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:45:05.743 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:05 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-05-03T14:45:05.743 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:05 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-05-03T14:45:05.743 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:05 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]': finished 2024-05-03T14:45:05.743 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:05 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2024-05-03T14:45:05.743 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:05 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-05-03T14:45:05.743 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:05 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-05-03T14:45:05.743 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:05 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-05-03T14:45:05.743 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:05 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-05-03T14:45:05.743 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:05 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-05-03T14:45:05.743 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:05 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-05-03T14:45:05.743 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:05 smithi060 ceph-mon[28873]: osdmap e86: 12 total, 12 up, 12 in 2024-05-03T14:45:05.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:05 smithi090 ceph-mon[33568]: pgmap v345: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:45:05.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:05 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-05-03T14:45:05.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:05 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-05-03T14:45:05.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:05 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]': finished 2024-05-03T14:45:05.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:05 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2024-05-03T14:45:05.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:05 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-05-03T14:45:05.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:05 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-05-03T14:45:05.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:05 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-05-03T14:45:05.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:05 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-05-03T14:45:05.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:05 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-05-03T14:45:05.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:05 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-05-03T14:45:05.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:05 smithi090 ceph-mon[33568]: osdmap e86: 12 total, 12 up, 12 in 2024-05-03T14:45:06.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:06 smithi149 ceph-mon[33332]: osdmap e87: 12 total, 12 up, 12 in 2024-05-03T14:45:06.731 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:06 smithi060 ceph-mon[28873]: osdmap e87: 12 total, 12 up, 12 in 2024-05-03T14:45:06.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:06 smithi090 ceph-mon[33568]: osdmap e87: 12 total, 12 up, 12 in 2024-05-03T14:45:06.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:06.893+0000 7f3565c00700 1 -- 172.21.15.60:0/4008077692 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3560104f10 msgr2=0x7f356010d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:06.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:06.893+0000 7f3565c00700 1 --2- 172.21.15.60:0/4008077692 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3560104f10 0x7f356010d400 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f354c0098f0 tx=0x7f354c009c00 comp rx=0 tx=0).stop 2024-05-03T14:45:06.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:06.894+0000 7f3565c00700 1 -- 172.21.15.60:0/4008077692 shutdown_connections 2024-05-03T14:45:06.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:06.894+0000 7f3565c00700 1 --2- 172.21.15.60:0/4008077692 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f356010d9d0 0x7f356010fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:06.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:06.894+0000 7f3565c00700 1 --2- 172.21.15.60:0/4008077692 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35601045f0 0x7f35601049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:06.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:06.894+0000 7f3565c00700 1 --2- 172.21.15.60:0/4008077692 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3560104f10 0x7f356010d400 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:06.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:06.894+0000 7f3565c00700 1 -- 172.21.15.60:0/4008077692 >> 172.21.15.60:0/4008077692 conn(0x7f35600fddc0 msgr2=0x7f35601001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:45:06.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:06.894+0000 7f3565c00700 1 -- 172.21.15.60:0/4008077692 shutdown_connections 2024-05-03T14:45:06.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:06.894+0000 7f3565c00700 1 -- 172.21.15.60:0/4008077692 wait complete. 2024-05-03T14:45:06.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:06.895+0000 7f3565c00700 1 Processor -- start 2024-05-03T14:45:06.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:06.895+0000 7f3565c00700 1 -- start start 2024-05-03T14:45:06.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:06.895+0000 7f3565c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35601045f0 0x7f3560102250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:45:06.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:06.895+0000 7f3565c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3560104f10 0x7f3560102790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:45:06.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:06.895+0000 7f3565c00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f356010d9d0 0x7f3560102cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:45:06.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:06.895+0000 7f3565c00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f3560113890 con 0x7f356010d9d0 2024-05-03T14:45:06.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:06.895+0000 7f3565c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3560113710 con 0x7f35601045f0 2024-05-03T14:45:06.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:06.895+0000 7f3565c00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3560113a10 con 0x7f3560104f10 2024-05-03T14:45:06.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:06.896+0000 7f355fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35601045f0 0x7f3560102250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:45:06.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:06.896+0000 7f3565200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f356010d9d0 0x7f3560102cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:45:06.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:06.896+0000 7f355fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35601045f0 0x7f3560102250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.60:50318/0 (socket says 172.21.15.60:50318) 2024-05-03T14:45:06.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:06.896+0000 7f355fe00700 1 -- 172.21.15.60:0/2770966727 learned_addr learned my addr 172.21.15.60:0/2770966727 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:45:06.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:06.896+0000 7f355f400700 1 --2- 172.21.15.60:0/2770966727 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3560104f10 0x7f3560102790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:45:06.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:06.896+0000 7f355fe00700 1 -- 172.21.15.60:0/2770966727 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3560104f10 msgr2=0x7f3560102790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:06.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:06.896+0000 7f355fe00700 1 --2- 172.21.15.60:0/2770966727 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3560104f10 0x7f3560102790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:06.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:06.896+0000 7f355fe00700 1 -- 172.21.15.60:0/2770966727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f356010d9d0 msgr2=0x7f3560102cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:06.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:06.896+0000 7f355fe00700 1 --2- 172.21.15.60:0/2770966727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f356010d9d0 0x7f3560102cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:06.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:06.896+0000 7f355fe00700 1 -- 172.21.15.60:0/2770966727 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35601a8230 con 0x7f35601045f0 2024-05-03T14:45:06.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:06.896+0000 7f3565200700 1 --2- 172.21.15.60:0/2770966727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f356010d9d0 0x7f3560102cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:45:06.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:06.896+0000 7f355fe00700 1 --2- 172.21.15.60:0/2770966727 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35601045f0 0x7f3560102250 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f355000eb30 tx=0x7f355000ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:45:06.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:06.897+0000 7f355cc00700 1 -- 172.21.15.60:0/2770966727 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f355000cbd0 con 0x7f35601045f0 2024-05-03T14:45:06.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:06.897+0000 7f355cc00700 1 -- 172.21.15.60:0/2770966727 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f3550004520 con 0x7f35601045f0 2024-05-03T14:45:06.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:06.897+0000 7f3565c00700 1 -- 172.21.15.60:0/2770966727 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35601a8520 con 0x7f35601045f0 2024-05-03T14:45:06.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:06.897+0000 7f355cc00700 1 -- 172.21.15.60:0/2770966727 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3550010430 con 0x7f35601045f0 2024-05-03T14:45:06.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:06.897+0000 7f3565c00700 1 -- 172.21.15.60:0/2770966727 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35601a8a30 con 0x7f35601045f0 2024-05-03T14:45:06.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:06.898+0000 7f3565c00700 1 -- 172.21.15.60:0/2770966727 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f356006a190 con 0x7f35601045f0 2024-05-03T14:45:06.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:06.900+0000 7f355cc00700 1 -- 172.21.15.60:0/2770966727 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f3550003f70 con 0x7f35601045f0 2024-05-03T14:45:06.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:06.902+0000 7f355cc00700 1 --2- 172.21.15.60:0/2770966727 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f35440b2880 0x7f35440b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:45:06.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:06.902+0000 7f355cc00700 1 -- 172.21.15.60:0/2770966727 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f35500e29d0 con 0x7f35601045f0 2024-05-03T14:45:06.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:06.902+0000 7f355f400700 1 --2- 172.21.15.60:0/2770966727 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f35440b2880 0x7f35440b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:45:06.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:06.903+0000 7f355f400700 1 --2- 172.21.15.60:0/2770966727 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f35440b2880 0x7f35440b4d40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f354c0098c0 tx=0x7f354c019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:45:06.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:06.904+0000 7f355cc00700 1 -- 172.21.15.60:0/2770966727 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f35500aa3c0 con 0x7f35601045f0 2024-05-03T14:45:07.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:07.186+0000 7f3565c00700 1 -- 172.21.15.60:0/2770966727 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3560103780 con 0x7f35601045f0 2024-05-03T14:45:07.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:07.187+0000 7f355cc00700 1 -- 172.21.15.60:0/2770966727 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11794 (secure 0 0 0) 0x7f35500a9cc0 con 0x7f35601045f0 2024-05-03T14:45:07.189 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:45:07.190 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1849151522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1849151522},{"type":"v1","addr":"172.21.15.60:6835","nonce":1849151522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/523710346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":523710346},{"type":"v1","addr":"172.21.15.60:6837","nonce":523710346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3846205424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3846205424},{"type":"v1","addr":"172.21.15.60:6839","nonce":3846205424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/3033424631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":3033424631},{"type":"v1","addr":"172.21.15.90:6833","nonce":3033424631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/3454021291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3454021291},{"type":"v1","addr":"172.21.15.149:6833","nonce":3454021291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1057049585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1057049585},{"type":"v1","addr":"172.21.15.90:6835","nonce":1057049585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/882239924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":882239924},{"type":"v1","addr":"172.21.15.149:6835","nonce":882239924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/2243702844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":2243702844},{"type":"v1","addr":"172.21.15.90:6837","nonce":2243702844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/2611278690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2611278690},{"type":"v1","addr":"172.21.15.149:6837","nonce":2611278690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6839/2527622212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2527622212},{"type":"v1","addr":"172.21.15.149:6839","nonce":2527622212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24509,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6839/3512381172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3512381172},{"type":"v1","addr":"172.21.15.90:6839","nonce":3512381172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-03T14:44:34.226052+0000","modified":"2024-05-03T14:44:52.471988+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.60:6841/924163182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":924163182},{"type":"v1","addr":"172.21.15.60:6841","nonce":924163182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24511,"qdb_cluster":[24511]},"id":1}]} 2024-05-03T14:45:07.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:07.190+0000 7f3565c00700 1 -- 172.21.15.60:0/2770966727 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f35440b2880 msgr2=0x7f35440b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:07.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:07.190+0000 7f3565c00700 1 --2- 172.21.15.60:0/2770966727 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f35440b2880 0x7f35440b4d40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f354c0098c0 tx=0x7f354c019040 comp rx=0 tx=0).stop 2024-05-03T14:45:07.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:07.190+0000 7f3565c00700 1 -- 172.21.15.60:0/2770966727 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35601045f0 msgr2=0x7f3560102250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:07.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:07.190+0000 7f3565c00700 1 --2- 172.21.15.60:0/2770966727 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35601045f0 0x7f3560102250 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f355000eb30 tx=0x7f355000ee40 comp rx=0 tx=0).stop 2024-05-03T14:45:07.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:07.190+0000 7f3565c00700 1 -- 172.21.15.60:0/2770966727 shutdown_connections 2024-05-03T14:45:07.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:07.190+0000 7f3565c00700 1 --2- 172.21.15.60:0/2770966727 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f35440b2880 0x7f35440b4d40 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:07.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:07.190+0000 7f3565c00700 1 --2- 172.21.15.60:0/2770966727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f356010d9d0 0x7f3560102cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:07.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:07.190+0000 7f3565c00700 1 --2- 172.21.15.60:0/2770966727 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35601045f0 0x7f3560102250 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:07.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:07.190+0000 7f3565c00700 1 --2- 172.21.15.60:0/2770966727 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3560104f10 0x7f3560102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:07.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:07.191+0000 7f3565c00700 1 -- 172.21.15.60:0/2770966727 >> 172.21.15.60:0/2770966727 conn(0x7f35600fddc0 msgr2=0x7f35600fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:45:07.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:07.191+0000 7f3565c00700 1 -- 172.21.15.60:0/2770966727 shutdown_connections 2024-05-03T14:45:07.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:07.191+0000 7f3565c00700 1 -- 172.21.15.60:0/2770966727 wait complete. 2024-05-03T14:45:07.194 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 17 2024-05-03T14:45:07.593 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:07 smithi060 ceph-mon[28873]: pgmap v348: 129 pgs: 1 activating, 3 peering, 125 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:45:07.593 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:07 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/2770966727' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T14:45:07.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:07 smithi090 ceph-mon[33568]: pgmap v348: 129 pgs: 1 activating, 3 peering, 125 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:45:07.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:07 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/2770966727' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T14:45:07.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:07 smithi149 ceph-mon[33332]: pgmap v348: 129 pgs: 1 activating, 3 peering, 125 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:45:07.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:07 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/2770966727' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T14:45:07.977 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-03T14:45:08.463 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:45:09.825 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:09 smithi060 ceph-mon[28873]: pgmap v349: 129 pgs: 1 activating, 3 peering, 125 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:45:09.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:09 smithi090 ceph-mon[33568]: pgmap v349: 129 pgs: 1 activating, 3 peering, 125 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:45:09.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:09 smithi149 ceph-mon[33332]: pgmap v349: 129 pgs: 1 activating, 3 peering, 125 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:45:10.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:10.727+0000 7fd12a200700 1 -- 172.21.15.60:0/3721250144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1241083d0 msgr2=0x7fd12406b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:10.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:10.727+0000 7fd12a200700 1 --2- 172.21.15.60:0/3721250144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1241083d0 0x7fd12406b1b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd114009990 tx=0x7fd114009ca0 comp rx=0 tx=0).stop 2024-05-03T14:45:10.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:10.727+0000 7fd12a200700 1 -- 172.21.15.60:0/3721250144 shutdown_connections 2024-05-03T14:45:10.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:10.727+0000 7fd12a200700 1 --2- 172.21.15.60:0/3721250144 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd124105ee0 0x7fd124106370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:10.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:10.727+0000 7fd12a200700 1 --2- 172.21.15.60:0/3721250144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1241083d0 0x7fd12406b1b0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:10.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:10.727+0000 7fd12a200700 1 --2- 172.21.15.60:0/3721250144 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd12406b6f0 0x7fd12406bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:10.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:10.727+0000 7fd12a200700 1 -- 172.21.15.60:0/3721250144 >> 172.21.15.60:0/3721250144 conn(0x7fd124079350 msgr2=0x7fd124079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:45:10.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:10.727+0000 7fd12a200700 1 -- 172.21.15.60:0/3721250144 shutdown_connections 2024-05-03T14:45:10.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:10.727+0000 7fd12a200700 1 -- 172.21.15.60:0/3721250144 wait complete. 2024-05-03T14:45:10.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:10.728+0000 7fd12a200700 1 Processor -- start 2024-05-03T14:45:10.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:10.728+0000 7fd12a200700 1 -- start start 2024-05-03T14:45:10.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:10.729+0000 7fd12a200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd12406b6f0 0x7fd1241a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:45:10.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:10.729+0000 7fd12a200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd124105ee0 0x7fd1241a0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:45:10.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:10.729+0000 7fd12a200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd1241083d0 0x7fd1241a4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:45:10.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:10.729+0000 7fd12a200700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd124119ec0 con 0x7fd12406b6f0 2024-05-03T14:45:10.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:10.729+0000 7fd12a200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd124119d40 con 0x7fd124105ee0 2024-05-03T14:45:10.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:10.729+0000 7fd12a200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd12411a040 con 0x7fd1241083d0 2024-05-03T14:45:10.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:10.729+0000 7fd123400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd12406b6f0 0x7fd1241a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:45:10.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:10.729+0000 7fd123400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd12406b6f0 0x7fd1241a0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43660/0 (socket says 172.21.15.60:43660) 2024-05-03T14:45:10.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:10.729+0000 7fd123400700 1 -- 172.21.15.60:0/2105803614 learned_addr learned my addr 172.21.15.60:0/2105803614 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:45:10.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:10.729+0000 7fd11aa00700 1 --2- 172.21.15.60:0/2105803614 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd124105ee0 0x7fd1241a0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:45:10.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:10.729+0000 7fd123e00700 1 --2- 172.21.15.60:0/2105803614 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd1241083d0 0x7fd1241a4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:45:10.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:10.730+0000 7fd123400700 1 -- 172.21.15.60:0/2105803614 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd1241083d0 msgr2=0x7fd1241a4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:10.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:10.730+0000 7fd123400700 1 --2- 172.21.15.60:0/2105803614 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd1241083d0 0x7fd1241a4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:10.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:10.730+0000 7fd123400700 1 -- 172.21.15.60:0/2105803614 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd124105ee0 msgr2=0x7fd1241a0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:10.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:10.730+0000 7fd123400700 1 --2- 172.21.15.60:0/2105803614 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd124105ee0 0x7fd1241a0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:10.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:10.730+0000 7fd123400700 1 -- 172.21.15.60:0/2105803614 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1241a5320 con 0x7fd12406b6f0 2024-05-03T14:45:10.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:10.730+0000 7fd123400700 1 --2- 172.21.15.60:0/2105803614 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd12406b6f0 0x7fd1241a0310 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd114000c00 tx=0x7fd1140047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:45:10.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:10.730+0000 7fd120c00700 1 -- 172.21.15.60:0/2105803614 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd114027070 con 0x7fd12406b6f0 2024-05-03T14:45:10.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:10.731+0000 7fd12a200700 1 -- 172.21.15.60:0/2105803614 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1241a55b0 con 0x7fd12406b6f0 2024-05-03T14:45:10.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:10.731+0000 7fd120c00700 1 -- 172.21.15.60:0/2105803614 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7fd114022470 con 0x7fd12406b6f0 2024-05-03T14:45:10.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:10.731+0000 7fd120c00700 1 -- 172.21.15.60:0/2105803614 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd114017650 con 0x7fd12406b6f0 2024-05-03T14:45:10.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:10.731+0000 7fd12a200700 1 -- 172.21.15.60:0/2105803614 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1241ace90 con 0x7fd12406b6f0 2024-05-03T14:45:10.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:10.732+0000 7fd12a200700 1 -- 172.21.15.60:0/2105803614 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1241029c0 con 0x7fd12406b6f0 2024-05-03T14:45:10.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:10.736+0000 7fd120c00700 1 -- 172.21.15.60:0/2105803614 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fd11402a070 con 0x7fd12406b6f0 2024-05-03T14:45:10.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:10.738+0000 7fd120c00700 1 --2- 172.21.15.60:0/2105803614 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fd1080b2880 0x7fd1080b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:45:10.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:10.738+0000 7fd120c00700 1 -- 172.21.15.60:0/2105803614 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fd1140e37d0 con 0x7fd12406b6f0 2024-05-03T14:45:10.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:10.738+0000 7fd11aa00700 1 --2- 172.21.15.60:0/2105803614 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fd1080b2880 0x7fd1080b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:45:10.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:10.739+0000 7fd11aa00700 1 --2- 172.21.15.60:0/2105803614 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fd1080b2880 0x7fd1080b4d40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd12406b410 tx=0x7fd10c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:45:10.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:10.740+0000 7fd120c00700 1 -- 172.21.15.60:0/2105803614 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fd114013070 con 0x7fd12406b6f0 2024-05-03T14:45:10.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:10.979+0000 7fd12a200700 1 -- 172.21.15.60:0/2105803614 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd1241ad060 con 0x7fd12406b6f0 2024-05-03T14:45:10.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:10.983+0000 7fd120c00700 1 -- 172.21.15.60:0/2105803614 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24626 (secure 0 0 0) 0x7fd1140ab1c0 con 0x7fd12406b6f0 2024-05-03T14:45:10.986 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:45:10.986 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":87,"fsid":"285d64b2-095a-11ef-bc97-c7b262605968","created":"2024-05-03T14:34:53.560460+0000","modified":"2024-05-03T14:45:05.402298+0000","last_up_change":"2024-05-03T14:43:14.877043+0000","last_in_change":"2024-05-03T14:42:55.085959+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-03T14:38:54.333714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-03T14:44:25.608599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-03T14:44:30.154009+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-03T14:44:42.549502+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f0797730-c4c1-47ea-9dce-9270d5ebb4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2952930162},{"type":"v1","addr":"172.21.15.60:6803","nonce":2952930162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2952930162},{"type":"v1","addr":"172.21.15.60:6805","nonce":2952930162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":2952930162},{"type":"v1","addr":"172.21.15.60:6809","nonce":2952930162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2952930162},{"type":"v1","addr":"172.21.15.60:6807","nonce":2952930162}]},"public_addr":"172.21.15.60:6803/2952930162","cluster_addr":"172.21.15.60:6805/2952930162","heartbeat_back_addr":"172.21.15.60:6809/2952930162","heartbeat_front_addr":"172.21.15.60:6807/2952930162","state":["exists","up"]},{"osd":1,"uuid":"96d0ad08-cda4-4824-bad4-e64973508c0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":720063242},{"type":"v1","addr":"172.21.15.90:6801","nonce":720063242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":720063242},{"type":"v1","addr":"172.21.15.90:6803","nonce":720063242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":720063242},{"type":"v1","addr":"172.21.15.90:6807","nonce":720063242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":720063242},{"type":"v1","addr":"172.21.15.90:6805","nonce":720063242}]},"public_addr":"172.21.15.90:6801/720063242","cluster_addr":"172.21.15.90:6803/720063242","heartbeat_back_addr":"172.21.15.90:6807/720063242","heartbeat_front_addr":"172.21.15.90:6805/720063242","state":["exists","up"]},{"osd":2,"uuid":"ce95fb7c-1611-4dd1-aee7-97c420d6a466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1359662077},{"type":"v1","addr":"172.21.15.149:6801","nonce":1359662077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1359662077},{"type":"v1","addr":"172.21.15.149:6803","nonce":1359662077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1359662077},{"type":"v1","addr":"172.21.15.149:6807","nonce":1359662077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1359662077},{"type":"v1","addr":"172.21.15.149:6805","nonce":1359662077}]},"public_addr":"172.21.15.149:6801/1359662077","cluster_addr":"172.21.15.149:6803/1359662077","heartbeat_back_addr":"172.21.15.149:6807/1359662077","heartbeat_front_addr":"172.21.15.149:6805/1359662077","state":["exists","up"]},{"osd":3,"uuid":"ef31136a-9b36-4b87-8389-f53d311bc5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3226133334},{"type":"v1","addr":"172.21.15.60:6811","nonce":3226133334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3226133334},{"type":"v1","addr":"172.21.15.60:6813","nonce":3226133334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3226133334},{"type":"v1","addr":"172.21.15.60:6817","nonce":3226133334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3226133334},{"type":"v1","addr":"172.21.15.60:6815","nonce":3226133334}]},"public_addr":"172.21.15.60:6811/3226133334","cluster_addr":"172.21.15.60:6813/3226133334","heartbeat_back_addr":"172.21.15.60:6817/3226133334","heartbeat_front_addr":"172.21.15.60:6815/3226133334","state":["exists","up"]},{"osd":4,"uuid":"73db7408-a2ff-47e9-b13d-d31688a93610","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2201391883},{"type":"v1","addr":"172.21.15.90:6809","nonce":2201391883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":2201391883},{"type":"v1","addr":"172.21.15.90:6811","nonce":2201391883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":2201391883},{"type":"v1","addr":"172.21.15.90:6815","nonce":2201391883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":2201391883},{"type":"v1","addr":"172.21.15.90:6813","nonce":2201391883}]},"public_addr":"172.21.15.90:6809/2201391883","cluster_addr":"172.21.15.90:6811/2201391883","heartbeat_back_addr":"172.21.15.90:6815/2201391883","heartbeat_front_addr":"172.21.15.90:6813/2201391883","state":["exists","up"]},{"osd":5,"uuid":"aefb78a2-522a-4b35-92b8-e5c04fa47b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2139732836},{"type":"v1","addr":"172.21.15.149:6809","nonce":2139732836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":2139732836},{"type":"v1","addr":"172.21.15.149:6811","nonce":2139732836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":2139732836},{"type":"v1","addr":"172.21.15.149:6815","nonce":2139732836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":2139732836},{"type":"v1","addr":"172.21.15.149:6813","nonce":2139732836}]},"public_addr":"172.21.15.149:6809/2139732836","cluster_addr":"172.21.15.149:6811/2139732836","heartbeat_back_addr":"172.21.15.149:6815/2139732836","heartbeat_front_addr":"172.21.15.149:6813/2139732836","state":["exists","up"]},{"osd":6,"uuid":"7c887819-6182-4636-8cfd-d845c30bd13c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3738521002},{"type":"v1","addr":"172.21.15.60:6819","nonce":3738521002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3738521002},{"type":"v1","addr":"172.21.15.60:6821","nonce":3738521002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3738521002},{"type":"v1","addr":"172.21.15.60:6825","nonce":3738521002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3738521002},{"type":"v1","addr":"172.21.15.60:6823","nonce":3738521002}]},"public_addr":"172.21.15.60:6819/3738521002","cluster_addr":"172.21.15.60:6821/3738521002","heartbeat_back_addr":"172.21.15.60:6825/3738521002","heartbeat_front_addr":"172.21.15.60:6823/3738521002","state":["exists","up"]},{"osd":7,"uuid":"ebce6acf-8873-4b35-bd5f-80de2163a307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":1686298661},{"type":"v1","addr":"172.21.15.90:6817","nonce":1686298661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":1686298661},{"type":"v1","addr":"172.21.15.90:6819","nonce":1686298661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":1686298661},{"type":"v1","addr":"172.21.15.90:6823","nonce":1686298661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1686298661},{"type":"v1","addr":"172.21.15.90:6821","nonce":1686298661}]},"public_addr":"172.21.15.90:6817/1686298661","cluster_addr":"172.21.15.90:6819/1686298661","heartbeat_back_addr":"172.21.15.90:6823/1686298661","heartbeat_front_addr":"172.21.15.90:6821/1686298661","state":["exists","up"]},{"osd":8,"uuid":"4ae04b98-e9b2-4956-83d9-17e5ac3c963f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":2608348248},{"type":"v1","addr":"172.21.15.149:6817","nonce":2608348248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2608348248},{"type":"v1","addr":"172.21.15.149:6819","nonce":2608348248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2608348248},{"type":"v1","addr":"172.21.15.149:6823","nonce":2608348248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2608348248},{"type":"v1","addr":"172.21.15.149:6821","nonce":2608348248}]},"public_addr":"172.21.15.149:6817/2608348248","cluster_addr":"172.21.15.149:6819/2608348248","heartbeat_back_addr":"172.21.15.149:6823/2608348248","heartbeat_front_addr":"172.21.15.149:6821/2608348248","state":["exists","up"]},{"osd":9,"uuid":"d0c0a70d-aa76-47e4-91e8-c343c8c41197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":2156130999},{"type":"v1","addr":"172.21.15.60:6827","nonce":2156130999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":2156130999},{"type":"v1","addr":"172.21.15.60:6829","nonce":2156130999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":2156130999},{"type":"v1","addr":"172.21.15.60:6833","nonce":2156130999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":2156130999},{"type":"v1","addr":"172.21.15.60:6831","nonce":2156130999}]},"public_addr":"172.21.15.60:6827/2156130999","cluster_addr":"172.21.15.60:6829/2156130999","heartbeat_back_addr":"172.21.15.60:6833/2156130999","heartbeat_front_addr":"172.21.15.60:6831/2156130999","state":["exists","up"]},{"osd":10,"uuid":"cdc3d7b8-33c7-4cc4-be4e-6887d15a3d8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":2492593590},{"type":"v1","addr":"172.21.15.90:6825","nonce":2492593590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":2492593590},{"type":"v1","addr":"172.21.15.90:6827","nonce":2492593590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":2492593590},{"type":"v1","addr":"172.21.15.90:6831","nonce":2492593590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":2492593590},{"type":"v1","addr":"172.21.15.90:6829","nonce":2492593590}]},"public_addr":"172.21.15.90:6825/2492593590","cluster_addr":"172.21.15.90:6827/2492593590","heartbeat_back_addr":"172.21.15.90:6831/2492593590","heartbeat_front_addr":"172.21.15.90:6829/2492593590","state":["exists","up"]},{"osd":11,"uuid":"8d17ec1a-bdd5-41f1-9b59-c2b6a527bfbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":1063245126},{"type":"v1","addr":"172.21.15.149:6825","nonce":1063245126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":1063245126},{"type":"v1","addr":"172.21.15.149:6827","nonce":1063245126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":1063245126},{"type":"v1","addr":"172.21.15.149:6831","nonce":1063245126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":1063245126},{"type":"v1","addr":"172.21.15.149:6829","nonce":1063245126}]},"public_addr":"172.21.15.149:6825/1063245126","cluster_addr":"172.21.15.149:6827/1063245126","heartbeat_back_addr":"172.21.15.149:6831/1063245126","heartbeat_front_addr":"172.21.15.149:6829/1063245126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:38:02.056563+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:38:27.140677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:38:51.287070+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:39:25.484410+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:39:51.483133+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:40:17.200423+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:40:51.979804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:41:18.742542+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:41:44.922541+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:42:19.778451+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:42:47.438440+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:43:13.429419+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6801/424391417":"2024-05-04T14:36:04.154333+0000","172.21.15.60:6800/424391417":"2024-05-04T14:36:04.154333+0000","172.21.15.60:0/1426532561":"2024-05-04T14:36:04.154333+0000","172.21.15.60:0/3230079926":"2024-05-04T14:35:28.066961+0000","172.21.15.60:0/2334895610":"2024-05-04T14:35:28.066961+0000","172.21.15.60:6800/888447185":"2024-05-04T14:35:28.066961+0000","172.21.15.60:0/3996899937":"2024-05-04T14:35:28.066961+0000","172.21.15.60:0/2779644018":"2024-05-04T14:36:04.154333+0000","172.21.15.60:0/71440838":"2024-05-04T14:36:04.154333+0000","172.21.15.60:6801/888447185":"2024-05-04T14:35:28.066961+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-03T14:45:10.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:10.987+0000 7fd12a200700 1 -- 172.21.15.60:0/2105803614 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fd1080b2880 msgr2=0x7fd1080b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:10.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:10.987+0000 7fd12a200700 1 --2- 172.21.15.60:0/2105803614 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fd1080b2880 0x7fd1080b4d40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd12406b410 tx=0x7fd10c009260 comp rx=0 tx=0).stop 2024-05-03T14:45:10.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:10.987+0000 7fd12a200700 1 -- 172.21.15.60:0/2105803614 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd12406b6f0 msgr2=0x7fd1241a0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:10.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:10.987+0000 7fd12a200700 1 --2- 172.21.15.60:0/2105803614 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd12406b6f0 0x7fd1241a0310 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd114000c00 tx=0x7fd1140047b0 comp rx=0 tx=0).stop 2024-05-03T14:45:10.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:10.987+0000 7fd12a200700 1 -- 172.21.15.60:0/2105803614 shutdown_connections 2024-05-03T14:45:10.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:10.987+0000 7fd12a200700 1 --2- 172.21.15.60:0/2105803614 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fd1080b2880 0x7fd1080b4d40 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:10.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:10.987+0000 7fd12a200700 1 --2- 172.21.15.60:0/2105803614 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd1241083d0 0x7fd1241a4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:10.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:10.987+0000 7fd12a200700 1 --2- 172.21.15.60:0/2105803614 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd12406b6f0 0x7fd1241a0310 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:10.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:10.987+0000 7fd12a200700 1 --2- 172.21.15.60:0/2105803614 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd124105ee0 0x7fd1241a0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:10.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:10.987+0000 7fd12a200700 1 -- 172.21.15.60:0/2105803614 >> 172.21.15.60:0/2105803614 conn(0x7fd124079350 msgr2=0x7fd124101390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:45:10.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:10.987+0000 7fd12a200700 1 -- 172.21.15.60:0/2105803614 shutdown_connections 2024-05-03T14:45:10.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:10.987+0000 7fd12a200700 1 -- 172.21.15.60:0/2105803614 wait complete. 2024-05-03T14:45:11.749 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:11 smithi060 ceph-mon[28873]: pgmap v350: 129 pgs: 1 activating, 1 peering, 127 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:45:11.749 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:11 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/2105803614' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T14:45:11.809 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-03T14:44:34.226052+0000', 'modified': '2024-05-03T14:44:52.471988+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24511}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24511': {'gid': 24511, 'name': 'a', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.60:6841/924163182', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6840', 'nonce': 924163182}, {'type': 'v1', 'addr': '172.21.15.60:6841', 'nonce': 924163182}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24511, 'qdb_cluster': [24511]} 2024-05-03T14:45:11.809 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-03T14:45:11.809 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph tell mds.a status 2024-05-03T14:45:11.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:11 smithi090 ceph-mon[33568]: pgmap v350: 129 pgs: 1 activating, 1 peering, 127 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:45:11.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:11 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/2105803614' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T14:45:11.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:11 smithi149 ceph-mon[33332]: pgmap v350: 129 pgs: 1 activating, 1 peering, 127 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:45:11.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:11 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/2105803614' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T14:45:12.285 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:45:13.810 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:13 smithi060 ceph-mon[28873]: pgmap v351: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:45:13.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:13 smithi090 ceph-mon[33568]: pgmap v351: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:45:13.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:13 smithi149 ceph-mon[33332]: pgmap v351: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:45:14.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.305+0000 7f7cc2a00700 1 -- 172.21.15.60:0/2251628908 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7cbc105f90 msgr2=0x7f7cbc112450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:14.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.305+0000 7f7cc2a00700 1 --2- 172.21.15.60:0/2251628908 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7cbc105f90 0x7f7cbc112450 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7cb400b0d0 tx=0x7f7cb400b3e0 comp rx=0 tx=0).stop 2024-05-03T14:45:14.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.305+0000 7f7cc2a00700 1 -- 172.21.15.60:0/2251628908 shutdown_connections 2024-05-03T14:45:14.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.305+0000 7f7cc2a00700 1 --2- 172.21.15.60:0/2251628908 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7cbc105f90 0x7f7cbc112450 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:14.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.305+0000 7f7cc2a00700 1 --2- 172.21.15.60:0/2251628908 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7cbc10b680 0x7f7cbc10ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:14.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.305+0000 7f7cc2a00700 1 --2- 172.21.15.60:0/2251628908 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7cbc1055d0 0x7f7cbc105a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:14.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.305+0000 7f7cc2a00700 1 -- 172.21.15.60:0/2251628908 >> 172.21.15.60:0/2251628908 conn(0x7f7cbc101080 msgr2=0x7f7cbc1034a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:45:14.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.305+0000 7f7cc2a00700 1 -- 172.21.15.60:0/2251628908 shutdown_connections 2024-05-03T14:45:14.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.305+0000 7f7cc2a00700 1 -- 172.21.15.60:0/2251628908 wait complete. 2024-05-03T14:45:14.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.307+0000 7f7cc2a00700 1 Processor -- start 2024-05-03T14:45:14.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.308+0000 7f7cc2a00700 1 -- start start 2024-05-03T14:45:14.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.308+0000 7f7cc2a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7cbc1055d0 0x7f7cbc1a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:45:14.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.308+0000 7f7cc2a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7cbc105f90 0x7f7cbc1a1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:45:14.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.308+0000 7f7cc2a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7cbc10b680 0x7f7cbc19bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:45:14.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.308+0000 7f7cc2a00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f7cbc115aa0 con 0x7f7cbc1055d0 2024-05-03T14:45:14.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.308+0000 7f7cc2a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7cbc115920 con 0x7f7cbc105f90 2024-05-03T14:45:14.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.308+0000 7f7cc2a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7cbc115c20 con 0x7f7cbc10b680 2024-05-03T14:45:14.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.308+0000 7f7cc0c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7cbc105f90 0x7f7cbc1a1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:45:14.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.309+0000 7f7cc1600700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7cbc1055d0 0x7f7cbc1a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:45:14.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.309+0000 7f7cc0c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7cbc105f90 0x7f7cbc1a1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.60:34180/0 (socket says 172.21.15.60:34180) 2024-05-03T14:45:14.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.309+0000 7f7cc0c00700 1 -- 172.21.15.60:0/4159541839 learned_addr learned my addr 172.21.15.60:0/4159541839 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:45:14.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.309+0000 7f7cc2000700 1 --2- 172.21.15.60:0/4159541839 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7cbc10b680 0x7f7cbc19bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:45:14.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.309+0000 7f7cc0c00700 1 -- 172.21.15.60:0/4159541839 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7cbc10b680 msgr2=0x7f7cbc19bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:14.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.309+0000 7f7cc0c00700 1 --2- 172.21.15.60:0/4159541839 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7cbc10b680 0x7f7cbc19bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:14.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.309+0000 7f7cc0c00700 1 -- 172.21.15.60:0/4159541839 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7cbc1055d0 msgr2=0x7f7cbc1a1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:14.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.309+0000 7f7cc0c00700 1 --2- 172.21.15.60:0/4159541839 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7cbc1055d0 0x7f7cbc1a1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:14.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.309+0000 7f7cc0c00700 1 -- 172.21.15.60:0/4159541839 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cbc19c400 con 0x7f7cbc105f90 2024-05-03T14:45:14.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.309+0000 7f7cc2000700 1 --2- 172.21.15.60:0/4159541839 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7cbc10b680 0x7f7cbc19bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T14:45:14.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.309+0000 7f7cc0c00700 1 --2- 172.21.15.60:0/4159541839 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7cbc105f90 0x7f7cbc1a1f90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7ca800e9a0 tx=0x7f7ca800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:45:14.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.310+0000 7f7cba000700 1 -- 172.21.15.60:0/4159541839 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca800cbd0 con 0x7f7cbc105f90 2024-05-03T14:45:14.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.310+0000 7f7cba000700 1 -- 172.21.15.60:0/4159541839 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f7ca800cd70 con 0x7f7cbc105f90 2024-05-03T14:45:14.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.310+0000 7f7cc2a00700 1 -- 172.21.15.60:0/4159541839 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cbc19c6f0 con 0x7f7cbc105f90 2024-05-03T14:45:14.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.310+0000 7f7cba000700 1 -- 172.21.15.60:0/4159541839 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca8018820 con 0x7f7cbc105f90 2024-05-03T14:45:14.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.310+0000 7f7cc2a00700 1 -- 172.21.15.60:0/4159541839 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cbc1a89d0 con 0x7f7cbc105f90 2024-05-03T14:45:14.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.313+0000 7f7cba000700 1 -- 172.21.15.60:0/4159541839 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f7ca8010430 con 0x7f7cbc105f90 2024-05-03T14:45:14.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.314+0000 7f7ccea54b80 1 Processor -- start 2024-05-03T14:45:14.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.315+0000 7f7ccea54b80 1 -- start start 2024-05-03T14:45:14.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.315+0000 7f7ccea54b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605d8b789f0 0x5605d8b78dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:45:14.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.315+0000 7f7ccea54b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5605d8b79310 0x5605d8b72f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:45:14.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.315+0000 7f7ccea54b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5605d8b73580 0x5605d8b739e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:45:14.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.315+0000 7f7cba000700 1 --2- 172.21.15.60:0/4159541839 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f7ca40b2900 0x7f7ca40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:45:14.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.315+0000 7f7ccea54b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5605d8ac59d0 con 0x5605d8b73580 2024-05-03T14:45:14.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.315+0000 7f7ccea54b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5605d8a63710 con 0x5605d8b79310 2024-05-03T14:45:14.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.315+0000 7f7ccea54b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5605d8ac1990 con 0x5605d8b789f0 2024-05-03T14:45:14.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.315+0000 7f7cba000700 1 -- 172.21.15.60:0/4159541839 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f7ca80e27c0 con 0x7f7cbc105f90 2024-05-03T14:45:14.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.315+0000 7f7cc2000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605d8b789f0 0x5605d8b78dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:45:14.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.315+0000 7f7cc0c00700 1 --2- 172.21.15.60:0/4159541839 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f7ca40b2900 0x7f7ca40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:45:14.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.316+0000 7f7cc2000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605d8b789f0 0x5605d8b78dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.60:42568/0 (socket says 172.21.15.60:42568) 2024-05-03T14:45:14.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.316+0000 7f7cc2000700 1 -- 172.21.15.60:0/2983983676 learned_addr learned my addr 172.21.15.60:0/2983983676 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:45:14.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.316+0000 7f7cc1600700 1 --2- 172.21.15.60:0/2983983676 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5605d8b79310 0x5605d8b72f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:45:14.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.316+0000 7f7cc2000700 1 --2- 172.21.15.60:0/2983983676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5605d8b73580 0x5605d8b739e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:45:14.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.316+0000 7f7cc1600700 1 -- 172.21.15.60:0/2983983676 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605d8b789f0 msgr2=0x5605d8b78dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:45:14.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.316+0000 7f7cc1600700 1 --2- 172.21.15.60:0/2983983676 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605d8b789f0 0x5605d8b78dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:14.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.316+0000 7f7cc1600700 1 -- 172.21.15.60:0/2983983676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5605d8b73580 msgr2=0x5605d8b739e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:45:14.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.316+0000 7f7cc1600700 1 --2- 172.21.15.60:0/2983983676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5605d8b73580 0x5605d8b739e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:14.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.316+0000 7f7cc1600700 1 -- 172.21.15.60:0/2983983676 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605d8a12880 con 0x5605d8b79310 2024-05-03T14:45:14.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.316+0000 7f7cc0c00700 1 --2- 172.21.15.60:0/4159541839 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f7ca40b2900 0x7f7ca40b4dc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8011fd0 tx=0x7f7ca808ff90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:45:14.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.316+0000 7f7cc1600700 1 --2- 172.21.15.60:0/2983983676 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5605d8b79310 0x5605d8b72f60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0009c90 tx=0x7f7cb0011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7b8da22d31212d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:45:14.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.317+0000 7f7caea00700 1 -- 172.21.15.60:0/2983983676 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb0011d20 con 0x5605d8b79310 2024-05-03T14:45:14.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.317+0000 7f7caea00700 1 -- 172.21.15.60:0/2983983676 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f7cb0004550 con 0x5605d8b79310 2024-05-03T14:45:14.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.317+0000 7f7caea00700 1 -- 172.21.15.60:0/2983983676 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb0005310 con 0x5605d8b79310 2024-05-03T14:45:14.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.317+0000 7f7ccea54b80 1 -- 172.21.15.60:0/2983983676 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5605d8b79310 msgr2=0x5605d8b72f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:14.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.317+0000 7f7ccea54b80 1 --2- 172.21.15.60:0/2983983676 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5605d8b79310 0x5605d8b72f60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0009c90 tx=0x7f7cb0011440 comp rx=0 tx=0).stop 2024-05-03T14:45:14.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.318+0000 7f7ccea54b80 1 -- 172.21.15.60:0/2983983676 shutdown_connections 2024-05-03T14:45:14.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.318+0000 7f7ccea54b80 1 --2- 172.21.15.60:0/2983983676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5605d8b73580 0x5605d8b739e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:14.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.318+0000 7f7ccea54b80 1 --2- 172.21.15.60:0/2983983676 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605d8b789f0 0x5605d8b78dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:14.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.318+0000 7f7ccea54b80 1 --2- 172.21.15.60:0/2983983676 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5605d8b79310 0x5605d8b72f60 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:14.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.318+0000 7f7ccea54b80 1 -- 172.21.15.60:0/2983983676 >> 172.21.15.60:0/2983983676 conn(0x5605d8aa1b10 msgr2=0x5605d8b6d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:45:14.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.318+0000 7f7ccea54b80 1 -- 172.21.15.60:0/2983983676 shutdown_connections 2024-05-03T14:45:14.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.318+0000 7f7ccea54b80 1 -- 172.21.15.60:0/2983983676 wait complete. 2024-05-03T14:45:14.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.318+0000 7f7ccea54b80 1 -- start start 2024-05-03T14:45:14.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.319+0000 7f7ccea54b80 1 Processor -- start 2024-05-03T14:45:14.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.319+0000 7f7ccea54b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5605d8b73580 0x5605d8b7efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:45:14.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.319+0000 7f7ccea54b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5605d8b789f0 0x5605d8b7c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:45:14.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.319+0000 7f7ccea54b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605d8b79310 0x5605d8b7c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:45:14.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.319+0000 7f7ccea54b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5605d8b7fd80 con 0x5605d8b73580 2024-05-03T14:45:14.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.319+0000 7f7ccea54b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5605d8a2bad0 con 0x5605d8b789f0 2024-05-03T14:45:14.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.319+0000 7f7ccea54b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5605d8b7ff00 con 0x5605d8b79310 2024-05-03T14:45:14.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.319+0000 7f7ccea54b80 10 client.0 _ll_register_callbacks cb 0x5605d8a6cc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T14:45:14.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.320+0000 7f7cc2000700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5605d8b73580 0x5605d8b7efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:45:14.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.320+0000 7f7cc1600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5605d8b789f0 0x5605d8b7c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:45:14.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.320+0000 7f7cc2000700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5605d8b73580 0x5605d8b7efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40378/0 (socket says 172.21.15.60:40378) 2024-05-03T14:45:14.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.320+0000 7f7cc2000700 1 -- 172.21.15.60:0/3900171288 learned_addr learned my addr 172.21.15.60:0/3900171288 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:45:14.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.320+0000 7f7cc2000700 1 --2- 172.21.15.60:0/3900171288 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605d8b79310 0x5605d8b7c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:45:14.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.320+0000 7f7cc2000700 1 -- 172.21.15.60:0/3900171288 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605d8b79310 msgr2=0x5605d8b7c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:45:14.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.320+0000 7f7cc2000700 1 --2- 172.21.15.60:0/3900171288 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605d8b79310 0x5605d8b7c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:14.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.320+0000 7f7cc2000700 1 -- 172.21.15.60:0/3900171288 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5605d8b789f0 msgr2=0x5605d8b7c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:45:14.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.320+0000 7f7cc2000700 1 --2- 172.21.15.60:0/3900171288 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5605d8b789f0 0x5605d8b7c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:14.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.320+0000 7f7cc2000700 1 -- 172.21.15.60:0/3900171288 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605d8a8b3d0 con 0x5605d8b73580 2024-05-03T14:45:14.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.320+0000 7f7cc2000700 1 --2- 172.21.15.60:0/3900171288 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5605d8b73580 0x5605d8b7efc0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7cb400b770 tx=0x7f7cb400b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb23b701defbb9ba server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:45:14.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.320+0000 7f7ca3400700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-05-03T14:45:14.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.321+0000 7f7ca3400700 1 -- 172.21.15.60:0/3900171288 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb4008b50 con 0x5605d8b73580 2024-05-03T14:45:14.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.321+0000 7f7ca3400700 1 -- 172.21.15.60:0/3900171288 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f7cb4008cf0 con 0x5605d8b73580 2024-05-03T14:45:14.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.321+0000 7f7ccea54b80 1 -- 172.21.15.60:0/3900171288 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5605d8b40180 con 0x5605d8b73580 2024-05-03T14:45:14.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.321+0000 7f7ca3400700 1 -- 172.21.15.60:0/3900171288 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f7cb4003a00 con 0x5605d8b73580 2024-05-03T14:45:14.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.321+0000 7f7ccea54b80 10 client.14505 fetch_fsmap learned FSMap version 17 2024-05-03T14:45:14.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.321+0000 7f7ccea54b80 1 -- 172.21.15.60:0/3900171288 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x5605d8b8a590 con 0x5605d8b73580 2024-05-03T14:45:14.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.321+0000 7f7ca3400700 1 -- 172.21.15.60:0/3900171288 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x5605d8b40180 con 0x5605d8b73580 2024-05-03T14:45:14.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.322+0000 7f7ca3400700 1 -- 172.21.15.60:0/3900171288 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f7cb4011070 con 0x5605d8b73580 2024-05-03T14:45:14.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.322+0000 7f7ccea54b80 10 client.14505 fetch_fsmap finished waiting for FSMap version 17 2024-05-03T14:45:14.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.322+0000 7f7ccea54b80 10 client.14505 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-03T14:45:14.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.322+0000 7f7ccea54b80 20 client.14505 populate_metadata read hostname 'smithi060' 2024-05-03T14:45:14.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.322+0000 7f7ccea54b80 1 --2- 172.21.15.60:0/3900171288 >> [v2:172.21.15.60:6840/924163182,v1:172.21.15.60:6841/924163182] conn(0x5605d8b8cba0 0x5605d8b8e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:45:14.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.322+0000 7f7ccea54b80 4 client.14505 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T14:45:14.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.322+0000 7f7ccea54b80 1 -- 172.21.15.60:0/3900171288 --> [v2:172.21.15.60:6840/924163182,v1:172.21.15.60:6841/924163182] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605d8ad33b0 con 0x5605d8b8cba0 2024-05-03T14:45:14.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.322+0000 7f7cc1600700 1 --2- 172.21.15.60:0/3900171288 >> [v2:172.21.15.60:6840/924163182,v1:172.21.15.60:6841/924163182] conn(0x5605d8b8cba0 0x5605d8b8e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:45:14.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.323+0000 7f7cc1600700 1 --2- 172.21.15.60:0/3900171288 >> [v2:172.21.15.60:6840/924163182,v1:172.21.15.60:6841/924163182] conn(0x5605d8b8cba0 0x5605d8b8e010 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1afee9913f3555e7 server_cookie=155fe788c134602e in_seq=0 out_seq=0 2024-05-03T14:45:14.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.323+0000 7f7ca3400700 10 client.14505 ms_handle_connect on v2:172.21.15.60:6840/924163182 2024-05-03T14:45:14.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.324+0000 7f7ca3400700 1 -- 172.21.15.60:0/3900171288 <== mds.0 v2:172.21.15.60:6840/924163182 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5605d8ad33b0 con 0x5605d8b8cba0 2024-05-03T14:45:14.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.324+0000 7f7ca3400700 10 client.14505 handle_command_reply: tid=0 2024-05-03T14:45:14.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.324+0000 7f7ccea54b80 1 client.14505 shutdown 2024-05-03T14:45:14.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.324+0000 7f7ccea54b80 1 -- 172.21.15.60:0/3900171288 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5605d8b73580 msgr2=0x5605d8b7efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:14.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.324+0000 7f7ccea54b80 1 --2- 172.21.15.60:0/3900171288 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5605d8b73580 0x5605d8b7efc0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7cb400b770 tx=0x7f7cb400b3e0 comp rx=0 tx=0).stop 2024-05-03T14:45:14.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.324+0000 7f7ccea54b80 1 -- 172.21.15.60:0/3900171288 shutdown_connections 2024-05-03T14:45:14.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.324+0000 7f7ccea54b80 1 --2- 172.21.15.60:0/3900171288 >> [v2:172.21.15.60:6840/924163182,v1:172.21.15.60:6841/924163182] conn(0x5605d8b8cba0 0x5605d8b8e010 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:14.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.324+0000 7f7ccea54b80 1 --2- 172.21.15.60:0/3900171288 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605d8b79310 0x5605d8b7c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:14.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.324+0000 7f7ccea54b80 1 --2- 172.21.15.60:0/3900171288 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5605d8b73580 0x5605d8b7efc0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:14.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.324+0000 7f7ccea54b80 1 --2- 172.21.15.60:0/3900171288 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5605d8b789f0 0x5605d8b7c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:14.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.324+0000 7f7ccea54b80 1 -- 172.21.15.60:0/3900171288 >> 172.21.15.60:0/3900171288 conn(0x5605d8aa1b10 msgr2=0x5605d8b6d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:45:14.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.324+0000 7f7ca3400700 0 client.14505 ms_handle_reset on v2:172.21.15.60:6840/924163182 2024-05-03T14:45:14.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.324+0000 7f7ccea54b80 1 -- 172.21.15.60:0/3900171288 shutdown_connections 2024-05-03T14:45:14.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.324+0000 7f7ccea54b80 1 -- 172.21.15.60:0/3900171288 wait complete. 2024-05-03T14:45:14.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.325+0000 7f7ccea54b80 20 client.14505 trim_cache size 0 max 16384 2024-05-03T14:45:14.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.348+0000 7f7ccea54b80 1 Processor -- start 2024-05-03T14:45:14.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.348+0000 7f7ccea54b80 1 -- start start 2024-05-03T14:45:14.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.349+0000 7f7ccea54b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5605d8b73580 0x5605d8b7c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:45:14.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.349+0000 7f7ccea54b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5605d8b789f0 0x5605d8b7c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:45:14.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.349+0000 7f7ccea54b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605d8b79310 0x5605d8b76490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:45:14.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.349+0000 7f7ccea54b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5605d8b8c350 con 0x5605d8b73580 2024-05-03T14:45:14.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.349+0000 7f7ccea54b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5605d8b8ca20 con 0x5605d8b789f0 2024-05-03T14:45:14.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.349+0000 7f7ccea54b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5605d8b8d0f0 con 0x5605d8b79310 2024-05-03T14:45:14.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.349+0000 7f7cc2000700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5605d8b73580 0x5605d8b7c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:45:14.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.349+0000 7f7cc2000700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5605d8b73580 0x5605d8b7c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40392/0 (socket says 172.21.15.60:40392) 2024-05-03T14:45:14.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.349+0000 7f7cc2000700 1 -- 172.21.15.60:0/38109287 learned_addr learned my addr 172.21.15.60:0/38109287 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:45:14.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.349+0000 7f7cc1600700 1 --2- 172.21.15.60:0/38109287 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5605d8b789f0 0x5605d8b7c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:45:14.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.350+0000 7f7cc2000700 1 --2- 172.21.15.60:0/38109287 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605d8b79310 0x5605d8b76490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:45:14.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.350+0000 7f7cc1600700 1 -- 172.21.15.60:0/38109287 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605d8b79310 msgr2=0x5605d8b76490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:45:14.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.350+0000 7f7cc1600700 1 --2- 172.21.15.60:0/38109287 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605d8b79310 0x5605d8b76490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:14.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.350+0000 7f7cc1600700 1 -- 172.21.15.60:0/38109287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5605d8b73580 msgr2=0x5605d8b7c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:45:14.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.350+0000 7f7cc1600700 1 --2- 172.21.15.60:0/38109287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5605d8b73580 0x5605d8b7c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:14.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.350+0000 7f7cc1600700 1 -- 172.21.15.60:0/38109287 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605d8b7a8f0 con 0x5605d8b789f0 2024-05-03T14:45:14.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.350+0000 7f7cc1600700 1 --2- 172.21.15.60:0/38109287 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5605d8b789f0 0x5605d8b7c860 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0009c60 tx=0x7f7cb0009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd0f73ed2ecd2c56 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:45:14.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.350+0000 7f7cae000700 1 -- 172.21.15.60:0/38109287 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb0005ce0 con 0x5605d8b789f0 2024-05-03T14:45:14.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.350+0000 7f7cae000700 1 -- 172.21.15.60:0/38109287 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f7cb0005550 con 0x5605d8b789f0 2024-05-03T14:45:14.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.350+0000 7f7cae000700 1 -- 172.21.15.60:0/38109287 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb002a2a0 con 0x5605d8b789f0 2024-05-03T14:45:14.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.351+0000 7f7ccea54b80 1 -- 172.21.15.60:0/38109287 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5605d8b789f0 msgr2=0x5605d8b7c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:14.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.351+0000 7f7ccea54b80 1 --2- 172.21.15.60:0/38109287 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5605d8b789f0 0x5605d8b7c860 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0009c60 tx=0x7f7cb0009b90 comp rx=0 tx=0).stop 2024-05-03T14:45:14.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.351+0000 7f7ccea54b80 1 -- 172.21.15.60:0/38109287 shutdown_connections 2024-05-03T14:45:14.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.351+0000 7f7ccea54b80 1 --2- 172.21.15.60:0/38109287 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605d8b79310 0x5605d8b76490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:14.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.351+0000 7f7ccea54b80 1 --2- 172.21.15.60:0/38109287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5605d8b73580 0x5605d8b7c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:14.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.351+0000 7f7ccea54b80 1 --2- 172.21.15.60:0/38109287 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5605d8b789f0 0x5605d8b7c860 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:14.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.351+0000 7f7ccea54b80 1 -- 172.21.15.60:0/38109287 >> 172.21.15.60:0/38109287 conn(0x5605d8aa1b10 msgr2=0x5605d8b6f230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:45:14.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.351+0000 7f7ccea54b80 1 -- 172.21.15.60:0/38109287 shutdown_connections 2024-05-03T14:45:14.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.351+0000 7f7ccea54b80 1 -- 172.21.15.60:0/38109287 wait complete. 2024-05-03T14:45:14.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.352+0000 7f7ccea54b80 1 -- start start 2024-05-03T14:45:14.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.352+0000 7f7ccea54b80 1 Processor -- start 2024-05-03T14:45:14.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.353+0000 7f7ccea54b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5605d8b73580 0x5605d8b97ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:45:14.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.353+0000 7f7ccea54b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605d8b789f0 0x5605d8b71f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:45:14.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.353+0000 7f7ccea54b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5605d8b79310 0x5605d8b72560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:45:14.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.353+0000 7f7ccea54b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5605d8b8de90 con 0x5605d8b73580 2024-05-03T14:45:14.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.353+0000 7f7ccea54b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5605d8b8d7c0 con 0x5605d8b79310 2024-05-03T14:45:14.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.353+0000 7f7ccea54b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5605d8b8e560 con 0x5605d8b789f0 2024-05-03T14:45:14.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.353+0000 7f7ccea54b80 10 client.0 _ll_register_callbacks cb 0x5605d8a6cc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T14:45:14.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.353+0000 7f7cc1600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605d8b789f0 0x5605d8b71f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:45:14.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.353+0000 7f7cc2000700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5605d8b73580 0x5605d8b97ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:45:14.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.353+0000 7f7cc1600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605d8b789f0 0x5605d8b71f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.60:42598/0 (socket says 172.21.15.60:42598) 2024-05-03T14:45:14.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.353+0000 7f7cc1600700 1 -- 172.21.15.60:0/653331915 learned_addr learned my addr 172.21.15.60:0/653331915 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:45:14.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.353+0000 7f7cc2000700 1 --2- 172.21.15.60:0/653331915 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5605d8b79310 0x5605d8b72560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:45:14.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.354+0000 7f7cc1600700 1 -- 172.21.15.60:0/653331915 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5605d8b79310 msgr2=0x5605d8b72560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:45:14.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.354+0000 7f7cc1600700 1 --2- 172.21.15.60:0/653331915 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5605d8b79310 0x5605d8b72560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:14.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.354+0000 7f7cc1600700 1 -- 172.21.15.60:0/653331915 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5605d8b73580 msgr2=0x5605d8b97ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:45:14.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.354+0000 7f7cc1600700 1 --2- 172.21.15.60:0/653331915 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5605d8b73580 0x5605d8b97ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:14.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.354+0000 7f7cc1600700 1 -- 172.21.15.60:0/653331915 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605d8b72d70 con 0x5605d8b789f0 2024-05-03T14:45:14.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.354+0000 7f7cc2000700 1 --2- 172.21.15.60:0/653331915 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5605d8b79310 0x5605d8b72560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T14:45:14.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.354+0000 7f7cc1600700 1 --2- 172.21.15.60:0/653331915 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605d8b789f0 0x5605d8b71f50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0011790 tx=0x7f7cb00092e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=796105a1bbef4b94 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:45:14.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.354+0000 7f7ca3400700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-05-03T14:45:14.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.354+0000 7f7ca3400700 1 -- 172.21.15.60:0/653331915 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb0004910 con 0x5605d8b789f0 2024-05-03T14:45:14.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.354+0000 7f7ccea54b80 1 -- 172.21.15.60:0/653331915 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cb0005550 con 0x5605d8b789f0 2024-05-03T14:45:14.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.354+0000 7f7ca3400700 1 -- 172.21.15.60:0/653331915 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f7cb0004b90 con 0x5605d8b789f0 2024-05-03T14:45:14.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.355+0000 7f7ca3400700 1 -- 172.21.15.60:0/653331915 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f7cb003e050 con 0x5605d8b789f0 2024-05-03T14:45:14.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.355+0000 7f7ccea54b80 10 client.24619 fetch_fsmap learned FSMap version 17 2024-05-03T14:45:14.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.355+0000 7f7ccea54b80 1 -- 172.21.15.60:0/653331915 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x5605d8bf8120 con 0x5605d8b789f0 2024-05-03T14:45:14.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.355+0000 7f7ca3400700 1 -- 172.21.15.60:0/653331915 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f7cb0005550 con 0x5605d8b789f0 2024-05-03T14:45:14.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.355+0000 7f7ca3400700 1 -- 172.21.15.60:0/653331915 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f7cb0018070 con 0x5605d8b789f0 2024-05-03T14:45:14.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.355+0000 7f7ccea54b80 10 client.24619 fetch_fsmap finished waiting for FSMap version 17 2024-05-03T14:45:14.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.355+0000 7f7ccea54b80 10 client.24619 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-03T14:45:14.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.356+0000 7f7ccea54b80 20 client.24619 populate_metadata read hostname 'smithi060' 2024-05-03T14:45:14.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.356+0000 7f7ccea54b80 1 --2- 172.21.15.60:0/653331915 >> [v2:172.21.15.60:6840/924163182,v1:172.21.15.60:6841/924163182] conn(0x5605d8bfc7a0 0x5605d8bfb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:45:14.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.356+0000 7f7ccea54b80 4 client.24619 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-03T14:45:14.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.356+0000 7f7ccea54b80 1 -- 172.21.15.60:0/653331915 --> [v2:172.21.15.60:6840/924163182,v1:172.21.15.60:6841/924163182] -- command(tid 0: {"prefix": "status"}) -- 0x5605d8acd450 con 0x5605d8bfc7a0 2024-05-03T14:45:14.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.356+0000 7f7cc2000700 1 --2- 172.21.15.60:0/653331915 >> [v2:172.21.15.60:6840/924163182,v1:172.21.15.60:6841/924163182] conn(0x5605d8bfc7a0 0x5605d8bfb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:45:14.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.356+0000 7f7cc2000700 1 --2- 172.21.15.60:0/653331915 >> [v2:172.21.15.60:6840/924163182,v1:172.21.15.60:6841/924163182] conn(0x5605d8bfc7a0 0x5605d8bfb4b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cff03456064e894 server_cookie=314cbf2d86044185 in_seq=0 out_seq=0 2024-05-03T14:45:14.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.356+0000 7f7ca3400700 10 client.24619 ms_handle_connect on v2:172.21.15.60:6840/924163182 2024-05-03T14:45:14.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.357+0000 7f7ca3400700 1 -- 172.21.15.60:0/653331915 <== mds.0 v2:172.21.15.60:6840/924163182 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x5605d8acd450 con 0x5605d8bfc7a0 2024-05-03T14:45:14.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.357+0000 7f7ca3400700 10 client.24619 handle_command_reply: tid=0 2024-05-03T14:45:14.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.357+0000 7f7ccea54b80 1 client.24619 shutdown 2024-05-03T14:45:14.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.357+0000 7f7ccea54b80 1 -- 172.21.15.60:0/653331915 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605d8b789f0 msgr2=0x5605d8b71f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:14.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.357+0000 7f7ccea54b80 1 --2- 172.21.15.60:0/653331915 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605d8b789f0 0x5605d8b71f50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0011790 tx=0x7f7cb00092e0 comp rx=0 tx=0).stop 2024-05-03T14:45:14.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.357+0000 7f7ccea54b80 1 -- 172.21.15.60:0/653331915 shutdown_connections 2024-05-03T14:45:14.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.357+0000 7f7ccea54b80 1 --2- 172.21.15.60:0/653331915 >> [v2:172.21.15.60:6840/924163182,v1:172.21.15.60:6841/924163182] conn(0x5605d8bfc7a0 0x5605d8bfb4b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:14.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.357+0000 7f7ccea54b80 1 --2- 172.21.15.60:0/653331915 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5605d8b79310 0x5605d8b72560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:14.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.357+0000 7f7ccea54b80 1 --2- 172.21.15.60:0/653331915 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5605d8b73580 0x5605d8b97ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:14.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.357+0000 7f7ccea54b80 1 --2- 172.21.15.60:0/653331915 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605d8b789f0 0x5605d8b71f50 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:14.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.357+0000 7f7ca3400700 0 client.24619 ms_handle_reset on v2:172.21.15.60:6840/924163182 2024-05-03T14:45:14.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.357+0000 7f7ccea54b80 1 -- 172.21.15.60:0/653331915 >> 172.21.15.60:0/653331915 conn(0x5605d8aa1b10 msgr2=0x5605d8b7c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:45:14.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.357+0000 7f7ccea54b80 1 -- 172.21.15.60:0/653331915 shutdown_connections 2024-05-03T14:45:14.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.357+0000 7f7ccea54b80 1 -- 172.21.15.60:0/653331915 wait complete. 2024-05-03T14:45:14.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.357+0000 7f7ccea54b80 20 client.24619 trim_cache size 0 max 16384 2024-05-03T14:45:14.362 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-05-03T14:45:14.362 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "285d64b2-095a-11ef-bc97-c7b262605968", 2024-05-03T14:45:14.362 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2024-05-03T14:45:14.363 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 24511, 2024-05-03T14:45:14.363 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-05-03T14:45:14.363 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-05-03T14:45:14.363 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-05-03T14:45:14.363 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 39.893803458999997, 2024-05-03T14:45:14.363 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 17, 2024-05-03T14:45:14.363 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 87, 2024-05-03T14:45:14.363 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 85, 2024-05-03T14:45:14.363 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 62.842576158 2024-05-03T14:45:14.363 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-05-03T14:45:14.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.358+0000 7f7cc2a00700 1 -- 172.21.15.60:0/4159541839 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f7ca40b2900 msgr2=0x7f7ca40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:14.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.358+0000 7f7cc2a00700 1 --2- 172.21.15.60:0/4159541839 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f7ca40b2900 0x7f7ca40b4dc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8011fd0 tx=0x7f7ca808ff90 comp rx=0 tx=0).stop 2024-05-03T14:45:14.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.358+0000 7f7cc2a00700 1 -- 172.21.15.60:0/4159541839 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7cbc105f90 msgr2=0x7f7cbc1a1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:14.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.358+0000 7f7cc2a00700 1 --2- 172.21.15.60:0/4159541839 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7cbc105f90 0x7f7cbc1a1f90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7ca800e9a0 tx=0x7f7ca800ecb0 comp rx=0 tx=0).stop 2024-05-03T14:45:14.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.358+0000 7f7cc2a00700 1 -- 172.21.15.60:0/4159541839 shutdown_connections 2024-05-03T14:45:14.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.358+0000 7f7cc2a00700 1 --2- 172.21.15.60:0/4159541839 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f7ca40b2900 0x7f7ca40b4dc0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:14.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.358+0000 7f7cc2a00700 1 --2- 172.21.15.60:0/4159541839 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7cbc10b680 0x7f7cbc19bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:14.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.358+0000 7f7cc2a00700 1 --2- 172.21.15.60:0/4159541839 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7cbc1055d0 0x7f7cbc1a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:14.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.358+0000 7f7cc2a00700 1 --2- 172.21.15.60:0/4159541839 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7cbc105f90 0x7f7cbc1a1f90 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:14.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.358+0000 7f7cc2a00700 1 -- 172.21.15.60:0/4159541839 >> 172.21.15.60:0/4159541839 conn(0x7f7cbc101080 msgr2=0x7f7cbc1023b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:45:14.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.358+0000 7f7cc2a00700 1 -- 172.21.15.60:0/4159541839 shutdown_connections 2024-05-03T14:45:14.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:14.359+0000 7f7cc2a00700 1 -- 172.21.15.60:0/4159541839 wait complete. 2024-05-03T14:45:15.159 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-03T14:45:15.159 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-03T14:45:15.160 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph fs ls 2024-05-03T14:45:15.654 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:45:15.808 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:15 smithi060 ceph-mon[28873]: pgmap v352: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:45:15.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:15 smithi090 ceph-mon[33568]: pgmap v352: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:45:15.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:15 smithi149 ceph-mon[33332]: pgmap v352: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:45:17.725 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:17 smithi149 ceph-mon[33332]: pgmap v353: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:45:17.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:17.753+0000 7f39f5400700 1 -- 172.21.15.60:0/3639255538 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f39f0075ff0 msgr2=0x7f39f0114130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:17.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:17.753+0000 7f39f5400700 1 --2- 172.21.15.60:0/3639255538 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f39f0075ff0 0x7f39f0114130 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f39dc009990 tx=0x7f39dc009ca0 comp rx=0 tx=0).stop 2024-05-03T14:45:17.755 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:17 smithi060 ceph-mon[28873]: pgmap v353: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:45:17.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:17.753+0000 7f39f5400700 1 -- 172.21.15.60:0/3639255538 shutdown_connections 2024-05-03T14:45:17.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:17.753+0000 7f39f5400700 1 --2- 172.21.15.60:0/3639255538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f39f0114670 0x7f39f0116a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:17.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:17.753+0000 7f39f5400700 1 --2- 172.21.15.60:0/3639255538 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39f00756d0 0x7f39f0075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:17.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:17.753+0000 7f39f5400700 1 --2- 172.21.15.60:0/3639255538 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f39f0075ff0 0x7f39f0114130 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:17.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:17.753+0000 7f39f5400700 1 -- 172.21.15.60:0/3639255538 >> 172.21.15.60:0/3639255538 conn(0x7f39f007a600 msgr2=0x7f39f007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:45:17.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:17.753+0000 7f39f5400700 1 -- 172.21.15.60:0/3639255538 shutdown_connections 2024-05-03T14:45:17.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:17.753+0000 7f39f5400700 1 -- 172.21.15.60:0/3639255538 wait complete. 2024-05-03T14:45:17.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:17.754+0000 7f39f5400700 1 Processor -- start 2024-05-03T14:45:17.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:17.754+0000 7f39f5400700 1 -- start start 2024-05-03T14:45:17.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:17.755+0000 7f39f5400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f39f00756d0 0x7f39f01a0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:45:17.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:17.755+0000 7f39f5400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f39f0075ff0 0x7f39f01a0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:45:17.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:17.755+0000 7f39f5400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39f0114670 0x7f39f01a4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:45:17.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:17.755+0000 7f39f5400700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f39f0119ef0 con 0x7f39f0114670 2024-05-03T14:45:17.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:17.755+0000 7f39f5400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f39f0119d70 con 0x7f39f0075ff0 2024-05-03T14:45:17.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:17.755+0000 7f39f5400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f39f011a070 con 0x7f39f00756d0 2024-05-03T14:45:17.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:17.755+0000 7f39ee800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f39f00756d0 0x7f39f01a0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:45:17.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:17.755+0000 7f39ee800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f39f00756d0 0x7f39f01a0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.60:42618/0 (socket says 172.21.15.60:42618) 2024-05-03T14:45:17.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:17.755+0000 7f39ee800700 1 -- 172.21.15.60:0/3195084075 learned_addr learned my addr 172.21.15.60:0/3195084075 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:45:17.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:17.755+0000 7f39ef200700 1 --2- 172.21.15.60:0/3195084075 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39f0114670 0x7f39f01a4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:45:17.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:17.755+0000 7f39ede00700 1 --2- 172.21.15.60:0/3195084075 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f39f0075ff0 0x7f39f01a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:45:17.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:17.755+0000 7f39ee800700 1 -- 172.21.15.60:0/3195084075 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f39f0075ff0 msgr2=0x7f39f01a0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:17.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:17.755+0000 7f39ee800700 1 --2- 172.21.15.60:0/3195084075 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f39f0075ff0 0x7f39f01a0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:17.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:17.755+0000 7f39ee800700 1 -- 172.21.15.60:0/3195084075 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39f0114670 msgr2=0x7f39f01a4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:17.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:17.755+0000 7f39ee800700 1 --2- 172.21.15.60:0/3195084075 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39f0114670 0x7f39f01a4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:17.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:17.755+0000 7f39ee800700 1 -- 172.21.15.60:0/3195084075 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39f01a5310 con 0x7f39f00756d0 2024-05-03T14:45:17.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:17.756+0000 7f39ee800700 1 --2- 172.21.15.60:0/3195084075 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f39f00756d0 0x7f39f01a0300 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f39e000eb30 tx=0x7f39e000ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:45:17.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:17.756+0000 7f39e7400700 1 -- 172.21.15.60:0/3195084075 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e000cbd0 con 0x7f39f00756d0 2024-05-03T14:45:17.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:17.756+0000 7f39f5400700 1 -- 172.21.15.60:0/3195084075 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39f01a5600 con 0x7f39f00756d0 2024-05-03T14:45:17.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:17.756+0000 7f39e7400700 1 -- 172.21.15.60:0/3195084075 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f39e0004520 con 0x7f39f00756d0 2024-05-03T14:45:17.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:17.756+0000 7f39e7400700 1 -- 172.21.15.60:0/3195084075 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e0010430 con 0x7f39f00756d0 2024-05-03T14:45:17.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:17.756+0000 7f39f5400700 1 -- 172.21.15.60:0/3195084075 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39f01ace80 con 0x7f39f00756d0 2024-05-03T14:45:17.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:17.757+0000 7f39f5400700 1 -- 172.21.15.60:0/3195084075 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39f006a190 con 0x7f39f00756d0 2024-05-03T14:45:17.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:17.760+0000 7f39e7400700 1 -- 172.21.15.60:0/3195084075 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f39e0003f70 con 0x7f39f00756d0 2024-05-03T14:45:17.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:17.761+0000 7f39e7400700 1 --2- 172.21.15.60:0/3195084075 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f39d40b2880 0x7f39d40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:45:17.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:17.761+0000 7f39e7400700 1 -- 172.21.15.60:0/3195084075 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f39e00e29f0 con 0x7f39f00756d0 2024-05-03T14:45:17.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:17.761+0000 7f39ede00700 1 --2- 172.21.15.60:0/3195084075 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f39d40b2880 0x7f39d40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:45:17.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:17.762+0000 7f39ede00700 1 --2- 172.21.15.60:0/3195084075 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f39d40b2880 0x7f39d40b4d40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f39dc00b2e0 tx=0x7f39dc005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:45:17.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:17.763+0000 7f39e7400700 1 -- 172.21.15.60:0/3195084075 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f39e00aa3e0 con 0x7f39f00756d0 2024-05-03T14:45:17.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:17 smithi090 ceph-mon[33568]: pgmap v353: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:45:18.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:18.051+0000 7f39f5400700 1 -- 172.21.15.60:0/3195084075 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f39f01ad080 con 0x7f39f00756d0 2024-05-03T14:45:18.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:18.052+0000 7f39e7400700 1 -- 172.21.15.60:0/3195084075 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7f39e00a9ce0 con 0x7f39f00756d0 2024-05-03T14:45:18.055 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-03T14:45:18.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:18.055+0000 7f39f5400700 1 -- 172.21.15.60:0/3195084075 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f39d40b2880 msgr2=0x7f39d40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:18.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:18.055+0000 7f39f5400700 1 --2- 172.21.15.60:0/3195084075 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f39d40b2880 0x7f39d40b4d40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f39dc00b2e0 tx=0x7f39dc005960 comp rx=0 tx=0).stop 2024-05-03T14:45:18.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:18.055+0000 7f39f5400700 1 -- 172.21.15.60:0/3195084075 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f39f00756d0 msgr2=0x7f39f01a0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:18.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:18.055+0000 7f39f5400700 1 --2- 172.21.15.60:0/3195084075 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f39f00756d0 0x7f39f01a0300 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f39e000eb30 tx=0x7f39e000ee40 comp rx=0 tx=0).stop 2024-05-03T14:45:18.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:18.056+0000 7f39f5400700 1 -- 172.21.15.60:0/3195084075 shutdown_connections 2024-05-03T14:45:18.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:18.056+0000 7f39f5400700 1 --2- 172.21.15.60:0/3195084075 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f39d40b2880 0x7f39d40b4d40 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:18.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:18.056+0000 7f39f5400700 1 --2- 172.21.15.60:0/3195084075 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39f0114670 0x7f39f01a4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:18.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:18.056+0000 7f39f5400700 1 --2- 172.21.15.60:0/3195084075 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f39f00756d0 0x7f39f01a0300 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:18.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:18.056+0000 7f39f5400700 1 --2- 172.21.15.60:0/3195084075 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f39f0075ff0 0x7f39f01a0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:18.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:18.056+0000 7f39f5400700 1 -- 172.21.15.60:0/3195084075 >> 172.21.15.60:0/3195084075 conn(0x7f39f007a600 msgr2=0x7f39f01133c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:45:18.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:18.056+0000 7f39f5400700 1 -- 172.21.15.60:0/3195084075 shutdown_connections 2024-05-03T14:45:18.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:18.056+0000 7f39f5400700 1 -- 172.21.15.60:0/3195084075 wait complete. 2024-05-03T14:45:18.789 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:18 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/3195084075' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-03T14:45:18.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:18 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/3195084075' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-03T14:45:18.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:18 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/3195084075' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-03T14:45:18.945 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-03T14:45:18.945 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-03T14:45:18.945 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2024-05-03T14:45:18.946 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-05-03T14:45:18.946 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-03T14:45:18.946 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-03T14:45:18.946 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-03T14:45:18.946 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-03T14:45:18.946 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-05-03T14:45:18.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T14:45:18.946 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2024-05-03T14:45:19.002 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-03T14:45:19.002 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-03T14:45:19.002 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2024-05-03T14:45:19.002 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-05-03T14:45:19.002 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2024-05-03T14:45:19.003 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-05-03T14:45:19.003 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-03T14:45:19.003 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2024-05-03T14:45:19.003 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2024-05-03T14:45:19.003 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-03T14:45:19.003 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2024-05-03T14:45:19.003 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2024-05-03T14:45:19.003 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-03T14:45:19.003 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2024-05-03T14:45:19.003 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2024-05-03T14:45:19.003 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-05-03T14:45:19.003 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 42008sec preferred_lft 42008sec 2024-05-03T14:45:19.003 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link noprefixroute 2024-05-03T14:45:19.004 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-05-03T14:45:19.004 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-03T14:45:19.004 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2024-05-03T14:45:19.004 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2024-05-03T14:45:19.004 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-03T14:45:19.004 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-03T14:45:19.004 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-05-03T14:45:19.004 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link add name ceph-brx type bridge 2024-05-03T14:45:19.005 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip addr flush dev ceph-brx 2024-05-03T14:45:19.005 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set ceph-brx up 2024-05-03T14:45:19.005 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-03T14:45:19.005 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-05-03T14:45:19.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-03T14:45:19.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-03T14:45:19.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T14:45:19.504 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-03T14:45:19.570 INFO:teuthology.orchestra.run.smithi060.stdout:1 2024-05-03T14:45:19.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T14:45:19.575 DEBUG:teuthology.orchestra.run.smithi060:> route 2024-05-03T14:45:19.623 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2024-05-03T14:45:19.623 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-03T14:45:19.624 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-03T14:45:19.624 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-03T14:45:19.624 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-03T14:45:19.624 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-03T14:45:19.624 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-05-03T14:45:19.624 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-03T14:45:19.624 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-03T14:45:19.624 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-03T14:45:19.624 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-05-03T14:45:19.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-03T14:45:19.738 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:19 smithi060 ceph-mon[28873]: pgmap v354: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:45:19.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:19 smithi090 ceph-mon[33568]: pgmap v354: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:45:19.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:19 smithi149 ceph-mon[33332]: pgmap v354: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:45:20.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-03T14:45:20.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T14:45:20.398 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-05-03T14:45:20.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T14:45:20.454 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list-id 2024-05-03T14:45:20.514 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-03T14:45:20.514 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-05-03T14:45:20.515 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-03T14:45:20.515 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-05-03T14:45:20.515 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-05-03T14:45:20.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-03T14:45:20.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-03T14:45:20.714 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-05-03T14:45:20.714 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-03T14:45:20.714 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-05-03T14:45:20.714 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-05-03T14:45:20.715 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-03T14:45:20.715 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-05-03T14:45:20.715 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-05-03T14:45:20.715 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-05-03T14:45:20.715 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-05-03T14:45:20.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-03T14:45:21.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-03T14:45:21.044 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-03T14:45:21.044 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-05-03T14:45:21.044 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2024-05-03T14:45:21.044 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set dev brx.0 master ceph-brx 2024-05-03T14:45:21.044 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-05-03T14:45:21.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-03T14:45:21.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-03T14:45:21.263 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-05-03T14:45:21.263 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-03T14:45:21.263 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-05-03T14:45:21.315 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-05-03T14:45:21.316 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-03T14:45:21.316 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-05-03T14:45:21.363 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2024-05-03T14:45:21.498 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2024-05-03T14:45:21.553 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2024-05-03T14:45:21.553 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2024-05-03T14:45:21.553 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2024-05-03T14:45:21.553 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2024-05-03T14:45:21.553 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2024-05-03T14:45:21.553 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2024-05-03T14:45:21.553 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2024-05-03T14:45:21.553 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2024-05-03T14:45:21.553 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2024-05-03T14:45:21.553 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2024-05-03T14:45:21.554 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2024-05-03T14:45:21.554 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/selinux 2024-05-03T14:45:21.554 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-05-03T14:45:21.554 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2024-05-03T14:45:21.554 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2024-05-03T14:45:21.554 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2024-05-03T14:45:21.554 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2024-05-03T14:45:21.554 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2024-05-03T14:45:21.554 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2024-05-03T14:45:21.554 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysctl.service 2024-05-03T14:45:21.554 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-03T14:45:21.554 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-03T14:45:21.554 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/nfs/rpc_pipefs 2024-05-03T14:45:21.555 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2024-05-03T14:45:21.555 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2024-05-03T14:45:21.555 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay 2024-05-03T14:45:21.555 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/f4b8c91fe61ac2c5dafc12c16692bd1c5054403eb9d515f17b45ba72b4156bfd/merged 2024-05-03T14:45:21.555 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/dea08ab99e3bf96db4bc2f63c160ebae60b11767639c127bfae1c456d17f32aa/merged 2024-05-03T14:45:21.555 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/0 2024-05-03T14:45:21.555 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-05-03T14:45:21.555 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/3e4ea8217380437f36fe4647025f68f0e1818033342f929370b7bb225389ad6f/merged 2024-05-03T14:45:21.555 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/bf2b6c66ad61369062b087f0df7b8bb7d621d568cdfac56fa3405020a5a5ba63/merged 2024-05-03T14:45:21.555 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/4692a39ddfa49d1f338957ca3ed68c3cee888d4c71af0cfe37be9753c637d423/merged 2024-05-03T14:45:21.555 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/88a9a78949f679d5568e794dfe6657078fd73d80888268b584b0713c29dc84f5/merged 2024-05-03T14:45:21.555 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/7e08d9f619de1c51d467f005c4b6c0fa7faa13812e99dd0c9415b0c131a64246/merged 2024-05-03T14:45:21.555 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/9247b1cb15d8fd63928dd40868da17af5ab7d30aadbf68a9b948c5355a208663/merged 2024-05-03T14:45:21.555 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/0143fe0762d0f6789c37355a0e5b1b3bc3fc8f234ae5c669c1b8ca255a30e207/merged 2024-05-03T14:45:21.555 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/2b8479a9d59d62149de5e6ea2ec39ddc4dda3242e6570a070bc5e23c3b3e99d2/merged 2024-05-03T14:45:21.556 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2024-05-03T14:45:21.556 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-03T14:45:21.556 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-03T14:45:21.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T14:45:21.557 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2024-05-03T14:45:21.606 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-03T14:45:21.606 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-05-03T14:45:21.654 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2024-05-03T14:45:21.717 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:21 smithi060 ceph-mon[28873]: pgmap v355: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:45:21.731 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2024-05-03T14:45:21.779 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2024-05-03T14:45:21.780 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2024-05-03T14:45:21.780 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2024-05-03T14:45:21.780 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2024-05-03T14:45:21.780 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2024-05-03T14:45:21.780 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2024-05-03T14:45:21.780 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2024-05-03T14:45:21.780 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2024-05-03T14:45:21.780 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2024-05-03T14:45:21.780 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2024-05-03T14:45:21.780 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2024-05-03T14:45:21.780 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/selinux 2024-05-03T14:45:21.781 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-05-03T14:45:21.781 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2024-05-03T14:45:21.781 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2024-05-03T14:45:21.781 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2024-05-03T14:45:21.781 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2024-05-03T14:45:21.781 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2024-05-03T14:45:21.781 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2024-05-03T14:45:21.781 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysctl.service 2024-05-03T14:45:21.781 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-03T14:45:21.781 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-03T14:45:21.781 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/nfs/rpc_pipefs 2024-05-03T14:45:21.781 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2024-05-03T14:45:21.781 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2024-05-03T14:45:21.781 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay 2024-05-03T14:45:21.781 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/f4b8c91fe61ac2c5dafc12c16692bd1c5054403eb9d515f17b45ba72b4156bfd/merged 2024-05-03T14:45:21.782 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/dea08ab99e3bf96db4bc2f63c160ebae60b11767639c127bfae1c456d17f32aa/merged 2024-05-03T14:45:21.782 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/0 2024-05-03T14:45:21.782 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-05-03T14:45:21.782 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/3e4ea8217380437f36fe4647025f68f0e1818033342f929370b7bb225389ad6f/merged 2024-05-03T14:45:21.782 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/bf2b6c66ad61369062b087f0df7b8bb7d621d568cdfac56fa3405020a5a5ba63/merged 2024-05-03T14:45:21.782 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/4692a39ddfa49d1f338957ca3ed68c3cee888d4c71af0cfe37be9753c637d423/merged 2024-05-03T14:45:21.782 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/88a9a78949f679d5568e794dfe6657078fd73d80888268b584b0713c29dc84f5/merged 2024-05-03T14:45:21.782 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/7e08d9f619de1c51d467f005c4b6c0fa7faa13812e99dd0c9415b0c131a64246/merged 2024-05-03T14:45:21.782 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/9247b1cb15d8fd63928dd40868da17af5ab7d30aadbf68a9b948c5355a208663/merged 2024-05-03T14:45:21.782 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/0143fe0762d0f6789c37355a0e5b1b3bc3fc8f234ae5c669c1b8ca255a30e207/merged 2024-05-03T14:45:21.782 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/2b8479a9d59d62149de5e6ea2ec39ddc4dda3242e6570a070bc5e23c3b3e99d2/merged 2024-05-03T14:45:21.782 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2024-05-03T14:45:21.782 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-03T14:45:21.782 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-03T14:45:21.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T14:45:21.783 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2024-05-03T14:45:21.863 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi060.stderr:2024-05-03T14:45:21.861+0000 7f47b2eb2480 -1 init, newargv = 0x55ebc3b566c0 newargc=15 2024-05-03T14:45:21.863 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi060.stderr:ceph-fuse[72103]: starting ceph client 2024-05-03T14:45:21.874 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi060.stderr:ceph-fuse[72103]: starting fuse 2024-05-03T14:45:21.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:21 smithi090 ceph-mon[33568]: pgmap v355: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:45:21.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:21 smithi149 ceph-mon[33332]: pgmap v355: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:45:22.927 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2024-05-03T14:45:23.027 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2024-05-03T14:45:23.085 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2024-05-03T14:45:23.085 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2024-05-03T14:45:23.085 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2024-05-03T14:45:23.085 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2024-05-03T14:45:23.085 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2024-05-03T14:45:23.085 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2024-05-03T14:45:23.085 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2024-05-03T14:45:23.085 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2024-05-03T14:45:23.086 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2024-05-03T14:45:23.086 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2024-05-03T14:45:23.086 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2024-05-03T14:45:23.086 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/selinux 2024-05-03T14:45:23.086 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-05-03T14:45:23.086 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2024-05-03T14:45:23.086 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2024-05-03T14:45:23.086 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2024-05-03T14:45:23.086 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2024-05-03T14:45:23.086 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2024-05-03T14:45:23.086 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2024-05-03T14:45:23.086 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysctl.service 2024-05-03T14:45:23.086 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-03T14:45:23.087 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-03T14:45:23.087 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/nfs/rpc_pipefs 2024-05-03T14:45:23.087 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2024-05-03T14:45:23.087 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2024-05-03T14:45:23.087 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay 2024-05-03T14:45:23.087 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/f4b8c91fe61ac2c5dafc12c16692bd1c5054403eb9d515f17b45ba72b4156bfd/merged 2024-05-03T14:45:23.087 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/dea08ab99e3bf96db4bc2f63c160ebae60b11767639c127bfae1c456d17f32aa/merged 2024-05-03T14:45:23.087 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/0 2024-05-03T14:45:23.087 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-05-03T14:45:23.087 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/3e4ea8217380437f36fe4647025f68f0e1818033342f929370b7bb225389ad6f/merged 2024-05-03T14:45:23.087 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/bf2b6c66ad61369062b087f0df7b8bb7d621d568cdfac56fa3405020a5a5ba63/merged 2024-05-03T14:45:23.087 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/4692a39ddfa49d1f338957ca3ed68c3cee888d4c71af0cfe37be9753c637d423/merged 2024-05-03T14:45:23.087 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/88a9a78949f679d5568e794dfe6657078fd73d80888268b584b0713c29dc84f5/merged 2024-05-03T14:45:23.088 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/7e08d9f619de1c51d467f005c4b6c0fa7faa13812e99dd0c9415b0c131a64246/merged 2024-05-03T14:45:23.088 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/9247b1cb15d8fd63928dd40868da17af5ab7d30aadbf68a9b948c5355a208663/merged 2024-05-03T14:45:23.088 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/0143fe0762d0f6789c37355a0e5b1b3bc3fc8f234ae5c669c1b8ca255a30e207/merged 2024-05-03T14:45:23.088 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/2b8479a9d59d62149de5e6ea2ec39ddc4dda3242e6570a070bc5e23c3b3e99d2/merged 2024-05-03T14:45:23.088 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2024-05-03T14:45:23.088 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-03T14:45:23.088 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-03T14:45:23.088 INFO:teuthology.orchestra.run.smithi060.stdout:/home/ubuntu/cephtest/mnt.admin 2024-05-03T14:45:23.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T14:45:23.089 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2024-05-03T14:45:23.142 INFO:teuthology.orchestra.run.smithi060.stdout:103 2024-05-03T14:45:23.142 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [103] 2024-05-03T14:45:23.142 DEBUG:teuthology.orchestra.run.smithi060:> sudo stdin-killer -- python3 -c ' 2024-05-03T14:45:23.142 DEBUG:teuthology.orchestra.run.smithi060:> import glob 2024-05-03T14:45:23.142 DEBUG:teuthology.orchestra.run.smithi060:> import re 2024-05-03T14:45:23.142 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-05-03T14:45:23.143 DEBUG:teuthology.orchestra.run.smithi060:> import subprocess 2024-05-03T14:45:23.143 DEBUG:teuthology.orchestra.run.smithi060:> 2024-05-03T14:45:23.143 DEBUG:teuthology.orchestra.run.smithi060:> def _find_admin_socket(client_name): 2024-05-03T14:45:23.143 DEBUG:teuthology.orchestra.run.smithi060:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-05-03T14:45:23.143 DEBUG:teuthology.orchestra.run.smithi060:> files = glob.glob(asok_path) 2024-05-03T14:45:23.143 DEBUG:teuthology.orchestra.run.smithi060:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-05-03T14:45:23.143 DEBUG:teuthology.orchestra.run.smithi060:> 2024-05-03T14:45:23.143 DEBUG:teuthology.orchestra.run.smithi060:> # Given a non-glob path, it better be there 2024-05-03T14:45:23.143 DEBUG:teuthology.orchestra.run.smithi060:> if "*" not in asok_path: 2024-05-03T14:45:23.143 DEBUG:teuthology.orchestra.run.smithi060:> assert(len(files) == 1) 2024-05-03T14:45:23.143 DEBUG:teuthology.orchestra.run.smithi060:> return files[0] 2024-05-03T14:45:23.143 DEBUG:teuthology.orchestra.run.smithi060:> 2024-05-03T14:45:23.143 DEBUG:teuthology.orchestra.run.smithi060:> for f in files: 2024-05-03T14:45:23.143 DEBUG:teuthology.orchestra.run.smithi060:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-03T14:45:23.143 DEBUG:teuthology.orchestra.run.smithi060:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-03T14:45:23.143 DEBUG:teuthology.orchestra.run.smithi060:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-03T14:45:23.143 DEBUG:teuthology.orchestra.run.smithi060:> contents = proc_f.read() 2024-05-03T14:45:23.144 DEBUG:teuthology.orchestra.run.smithi060:> if mountpoint in contents: 2024-05-03T14:45:23.144 DEBUG:teuthology.orchestra.run.smithi060:> return f 2024-05-03T14:45:23.144 DEBUG:teuthology.orchestra.run.smithi060:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-03T14:45:23.144 DEBUG:teuthology.orchestra.run.smithi060:> 2024-05-03T14:45:23.144 DEBUG:teuthology.orchestra.run.smithi060:> print(_find_admin_socket("client.admin")) 2024-05-03T14:45:23.144 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-05-03T14:45:23.320 INFO:teuthology.orchestra.run.smithi060.stdout:/var/run/ceph/ceph-client.admin.72103.asok 2024-05-03T14:45:23.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-03T14:45:23.339 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.72103.asok 2024-05-03T14:45:23.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T14:45:23.339 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.72103.asok status 2024-05-03T14:45:23.580 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-05-03T14:45:23.580 INFO:teuthology.orchestra.run.smithi060.stdout: "metadata": { 2024-05-03T14:45:23.580 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_sha1": "9a047fe7599a549042d06a3fb5b6813a203eabea", 2024-05-03T14:45:23.580 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_version": "ceph version 19.0.0-3450-g9a047fe7 (9a047fe7599a549042d06a3fb5b6813a203eabea) squid (dev)", 2024-05-03T14:45:23.580 INFO:teuthology.orchestra.run.smithi060.stdout: "entity_id": "admin", 2024-05-03T14:45:23.580 INFO:teuthology.orchestra.run.smithi060.stdout: "hostname": "smithi060", 2024-05-03T14:45:23.580 INFO:teuthology.orchestra.run.smithi060.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-05-03T14:45:23.580 INFO:teuthology.orchestra.run.smithi060.stdout: "pid": "72103", 2024-05-03T14:45:23.580 INFO:teuthology.orchestra.run.smithi060.stdout: "root": "/" 2024-05-03T14:45:23.580 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-05-03T14:45:23.580 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_count": 0, 2024-05-03T14:45:23.581 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_pinned_count": 0, 2024-05-03T14:45:23.581 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 24620, 2024-05-03T14:45:23.581 INFO:teuthology.orchestra.run.smithi060.stdout: "inst": { 2024-05-03T14:45:23.581 INFO:teuthology.orchestra.run.smithi060.stdout: "name": { 2024-05-03T14:45:23.581 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "client", 2024-05-03T14:45:23.581 INFO:teuthology.orchestra.run.smithi060.stdout: "num": 24620 2024-05-03T14:45:23.581 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-05-03T14:45:23.581 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2024-05-03T14:45:23.581 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-05-03T14:45:23.581 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:0", 2024-05-03T14:45:23.581 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 882953782 2024-05-03T14:45:23.581 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-05-03T14:45:23.581 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-05-03T14:45:23.581 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2024-05-03T14:45:23.581 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-05-03T14:45:23.582 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:0", 2024-05-03T14:45:23.582 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 882953782 2024-05-03T14:45:23.582 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-05-03T14:45:23.582 INFO:teuthology.orchestra.run.smithi060.stdout: "inst_str": "client.24620 172.21.15.60:0/882953782", 2024-05-03T14:45:23.582 INFO:teuthology.orchestra.run.smithi060.stdout: "addr_str": "172.21.15.60:0/882953782", 2024-05-03T14:45:23.582 INFO:teuthology.orchestra.run.smithi060.stdout: "inode_count": 1, 2024-05-03T14:45:23.582 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_epoch": 17, 2024-05-03T14:45:23.582 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch": 87, 2024-05-03T14:45:23.582 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch_barrier": 0, 2024-05-03T14:45:23.582 INFO:teuthology.orchestra.run.smithi060.stdout: "blocklisted": false, 2024-05-03T14:45:23.582 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs" 2024-05-03T14:45:23.582 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-05-03T14:45:23.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:23 smithi060 ceph-mon[28873]: pgmap v356: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:45:23.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T14:45:23.596 DEBUG:teuthology.orchestra.run.smithi060:> stat --file-system '--printf=%T 2024-05-03T14:45:23.596 DEBUG:teuthology.orchestra.run.smithi060:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-05-03T14:45:23.654 INFO:teuthology.orchestra.run.smithi060.stdout:fuseblk 2024-05-03T14:45:23.654 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-05-03T14:45:23.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T14:45:23.654 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-05-03T14:45:23.745 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-05-03T14:45:23.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:23 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-03T14:45:23.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:23 smithi090 ceph-mon[33568]: pgmap v356: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:45:23.894 INFO:teuthology.orchestra.run.smithi060.stdout:# file: . 2024-05-03T14:45:23.894 INFO:teuthology.orchestra.run.smithi060.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-05-03T14:45:23.894 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:45:23.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-03T14:45:23.905 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-05-03T14:45:23.905 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-03T14:45:23.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:23 smithi149 ceph-mon[33332]: pgmap v356: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:45:23.964 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph osd blocklist ls 2024-05-03T14:45:24.431 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:45:25.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:25 smithi060 ceph-mon[28873]: pgmap v357: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:45:25.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:25 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:45:25.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:25 smithi090 ceph-mon[33568]: pgmap v357: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:45:25.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:25 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:45:25.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:25 smithi149 ceph-mon[33332]: pgmap v357: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:45:25.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:25 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:45:26.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:26.559+0000 7fe226000700 1 -- 172.21.15.60:0/3953106122 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe220103550 msgr2=0x7fe2201039b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:26.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:26.559+0000 7fe226000700 1 --2- 172.21.15.60:0/3953106122 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe220103550 0x7fe2201039b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe208009990 tx=0x7fe208009ca0 comp rx=0 tx=0).stop 2024-05-03T14:45:26.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:26.559+0000 7fe226000700 1 -- 172.21.15.60:0/3953106122 shutdown_connections 2024-05-03T14:45:26.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:26.559+0000 7fe226000700 1 --2- 172.21.15.60:0/3953106122 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe220103ef0 0x7fe2201103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:26.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:26.559+0000 7fe226000700 1 --2- 172.21.15.60:0/3953106122 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe220108460 0x7fe220108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:26.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:26.559+0000 7fe226000700 1 --2- 172.21.15.60:0/3953106122 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe220103550 0x7fe2201039b0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:26.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:26.559+0000 7fe226000700 1 -- 172.21.15.60:0/3953106122 >> 172.21.15.60:0/3953106122 conn(0x7fe22007a600 msgr2=0x7fe22007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:45:26.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:26.559+0000 7fe226000700 1 -- 172.21.15.60:0/3953106122 shutdown_connections 2024-05-03T14:45:26.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:26.559+0000 7fe226000700 1 -- 172.21.15.60:0/3953106122 wait complete. 2024-05-03T14:45:26.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:26.560+0000 7fe226000700 1 Processor -- start 2024-05-03T14:45:26.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:26.561+0000 7fe226000700 1 -- start start 2024-05-03T14:45:26.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:26.561+0000 7fe226000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe220103550 0x7fe22019c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:45:26.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:26.561+0000 7fe226000700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe220103ef0 0x7fe22019c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:45:26.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:26.561+0000 7fe226000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe220108460 0x7fe2201a08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:45:26.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:26.561+0000 7fe226000700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe220113970 con 0x7fe220103ef0 2024-05-03T14:45:26.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:26.561+0000 7fe226000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe2201137f0 con 0x7fe220108460 2024-05-03T14:45:26.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:26.561+0000 7fe226000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe220113af0 con 0x7fe220103550 2024-05-03T14:45:26.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:26.561+0000 7fe21f400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe220103550 0x7fe22019c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:45:26.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:26.561+0000 7fe217e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe220103ef0 0x7fe22019c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:45:26.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:26.561+0000 7fe21f400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe220103550 0x7fe22019c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.60:55866/0 (socket says 172.21.15.60:55866) 2024-05-03T14:45:26.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:26.561+0000 7fe21f400700 1 -- 172.21.15.60:0/2771967144 learned_addr learned my addr 172.21.15.60:0/2771967144 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:45:26.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:26.561+0000 7fe21fe00700 1 --2- 172.21.15.60:0/2771967144 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe220108460 0x7fe2201a08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:45:26.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:26.562+0000 7fe21f400700 1 -- 172.21.15.60:0/2771967144 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe220108460 msgr2=0x7fe2201a08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:26.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:26.562+0000 7fe21f400700 1 --2- 172.21.15.60:0/2771967144 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe220108460 0x7fe2201a08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:26.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:26.562+0000 7fe21f400700 1 -- 172.21.15.60:0/2771967144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe220103ef0 msgr2=0x7fe22019c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:26.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:26.562+0000 7fe21f400700 1 --2- 172.21.15.60:0/2771967144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe220103ef0 0x7fe22019c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:26.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:26.562+0000 7fe21f400700 1 -- 172.21.15.60:0/2771967144 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2201a1010 con 0x7fe220103550 2024-05-03T14:45:26.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:26.562+0000 7fe21f400700 1 --2- 172.21.15.60:0/2771967144 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe220103550 0x7fe22019c000 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe21000e970 tx=0x7fe21000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:45:26.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:26.563+0000 7fe21cc00700 1 -- 172.21.15.60:0/2771967144 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe210004220 con 0x7fe220103550 2024-05-03T14:45:26.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:26.563+0000 7fe226000700 1 -- 172.21.15.60:0/2771967144 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2201a1300 con 0x7fe220103550 2024-05-03T14:45:26.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:26.563+0000 7fe21cc00700 1 -- 172.21.15.60:0/2771967144 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7fe21000cce0 con 0x7fe220103550 2024-05-03T14:45:26.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:26.563+0000 7fe21cc00700 1 -- 172.21.15.60:0/2771967144 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe210003e60 con 0x7fe220103550 2024-05-03T14:45:26.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:26.563+0000 7fe226000700 1 -- 172.21.15.60:0/2771967144 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2201a8b80 con 0x7fe220103550 2024-05-03T14:45:26.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:26.564+0000 7fe226000700 1 -- 172.21.15.60:0/2771967144 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe22006a190 con 0x7fe220103550 2024-05-03T14:45:26.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:26.566+0000 7fe21cc00700 1 -- 172.21.15.60:0/2771967144 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fe21000c7e0 con 0x7fe220103550 2024-05-03T14:45:26.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:26.568+0000 7fe21cc00700 1 --2- 172.21.15.60:0/2771967144 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fe2040b2880 0x7fe2040b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:45:26.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:26.568+0000 7fe21cc00700 1 -- 172.21.15.60:0/2771967144 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fe2100e27d0 con 0x7fe220103550 2024-05-03T14:45:26.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:26.568+0000 7fe217e00700 1 --2- 172.21.15.60:0/2771967144 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fe2040b2880 0x7fe2040b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:45:26.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:26.569+0000 7fe217e00700 1 --2- 172.21.15.60:0/2771967144 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fe2040b2880 0x7fe2040b4d40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe220110780 tx=0x7fe20800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:45:26.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:26.569+0000 7fe21cc00700 1 -- 172.21.15.60:0/2771967144 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fe2100aa1c0 con 0x7fe220103550 2024-05-03T14:45:26.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:26.835+0000 7fe226000700 1 -- 172.21.15.60:0/2771967144 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe2201a8e10 con 0x7fe220103550 2024-05-03T14:45:26.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:26.836+0000 7fe21cc00700 1 -- 172.21.15.60:0/2771967144 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v87) ==== 81+0+586 (secure 0 0 0) 0x7fe2100a9ac0 con 0x7fe220103550 2024-05-03T14:45:26.838 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6801/424391417 2024-05-04T14:36:04.154333+0000 2024-05-03T14:45:26.838 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6800/424391417 2024-05-04T14:36:04.154333+0000 2024-05-03T14:45:26.838 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/1426532561 2024-05-04T14:36:04.154333+0000 2024-05-03T14:45:26.838 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/3230079926 2024-05-04T14:35:28.066961+0000 2024-05-03T14:45:26.838 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/2334895610 2024-05-04T14:35:28.066961+0000 2024-05-03T14:45:26.838 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6800/888447185 2024-05-04T14:35:28.066961+0000 2024-05-03T14:45:26.838 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/3996899937 2024-05-04T14:35:28.066961+0000 2024-05-03T14:45:26.838 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/2779644018 2024-05-04T14:36:04.154333+0000 2024-05-03T14:45:26.838 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/71440838 2024-05-04T14:36:04.154333+0000 2024-05-03T14:45:26.838 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6801/888447185 2024-05-04T14:35:28.066961+0000 2024-05-03T14:45:26.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:26.839+0000 7fe226000700 1 -- 172.21.15.60:0/2771967144 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fe2040b2880 msgr2=0x7fe2040b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:26.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:26.839+0000 7fe226000700 1 --2- 172.21.15.60:0/2771967144 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fe2040b2880 0x7fe2040b4d40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe220110780 tx=0x7fe20800b260 comp rx=0 tx=0).stop 2024-05-03T14:45:26.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:26.839+0000 7fe226000700 1 -- 172.21.15.60:0/2771967144 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe220103550 msgr2=0x7fe22019c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:26.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:26.839+0000 7fe226000700 1 --2- 172.21.15.60:0/2771967144 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe220103550 0x7fe22019c000 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe21000e970 tx=0x7fe21000ec80 comp rx=0 tx=0).stop 2024-05-03T14:45:26.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:26.839+0000 7fe226000700 1 -- 172.21.15.60:0/2771967144 shutdown_connections 2024-05-03T14:45:26.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:26.839+0000 7fe226000700 1 --2- 172.21.15.60:0/2771967144 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fe2040b2880 0x7fe2040b4d40 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:26.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:26.839+0000 7fe226000700 1 --2- 172.21.15.60:0/2771967144 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe220108460 0x7fe2201a08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:26.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:26.839+0000 7fe226000700 1 --2- 172.21.15.60:0/2771967144 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe220103550 0x7fe22019c000 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:26.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:26.839+0000 7fe226000700 1 --2- 172.21.15.60:0/2771967144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe220103ef0 0x7fe22019c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:26.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:26.839+0000 7fe226000700 1 -- 172.21.15.60:0/2771967144 >> 172.21.15.60:0/2771967144 conn(0x7fe22007a600 msgr2=0x7fe220101420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:45:26.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:26.839+0000 7fe226000700 1 -- 172.21.15.60:0/2771967144 shutdown_connections 2024-05-03T14:45:26.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:26.839+0000 7fe226000700 1 -- 172.21.15.60:0/2771967144 wait complete. 2024-05-03T14:45:26.842 INFO:teuthology.orchestra.run.smithi060.stderr:listed 10 entries 2024-05-03T14:45:27.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:27 smithi060 ceph-mon[28873]: pgmap v358: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-03T14:45:27.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:27 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/2771967144' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-03T14:45:27.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:27 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:45:27.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:27 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:45:27.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:27 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:45:27.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:27 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:45:27.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:27 smithi090 ceph-mon[33568]: pgmap v358: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-03T14:45:27.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:27 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/2771967144' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-03T14:45:27.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:27 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:45:27.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:27 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:45:27.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:27 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:45:27.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:27 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:45:27.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:27 smithi149 ceph-mon[33332]: pgmap v358: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-03T14:45:27.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:27 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/2771967144' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-03T14:45:27.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:27 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:45:27.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:27 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:45:27.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:27 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:45:27.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:27 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:45:28.245 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-05-03T14:45:28.245 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-03T14:45:28.295 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph osd blocklist ls 2024-05-03T14:45:28.755 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:45:29.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:29 smithi060 ceph-mon[28873]: pgmap v359: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-03T14:45:29.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:29 smithi090 ceph-mon[33568]: pgmap v359: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-03T14:45:29.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:29 smithi149 ceph-mon[33332]: pgmap v359: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-03T14:45:30.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:30.915+0000 7f406d600700 1 -- 172.21.15.60:0/3881955291 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4068106070 msgr2=0x7f4068112580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:30.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:30.915+0000 7f406d600700 1 --2- 172.21.15.60:0/3881955291 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4068106070 0x7f4068112580 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f406000b0a0 tx=0x7f406000b3b0 comp rx=0 tx=0).stop 2024-05-03T14:45:30.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:30.916+0000 7f406d600700 1 -- 172.21.15.60:0/3881955291 shutdown_connections 2024-05-03T14:45:30.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:30.916+0000 7f406d600700 1 --2- 172.21.15.60:0/3881955291 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4068106070 0x7f4068112580 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:30.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:30.916+0000 7f406d600700 1 --2- 172.21.15.60:0/3881955291 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f406810b6b0 0x7f406810ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:30.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:30.916+0000 7f406d600700 1 --2- 172.21.15.60:0/3881955291 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f40681056b0 0x7f4068105b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:30.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:30.916+0000 7f406d600700 1 -- 172.21.15.60:0/3881955291 >> 172.21.15.60:0/3881955291 conn(0x7f40681011a0 msgr2=0x7f40681035c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:45:30.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:30.916+0000 7f406d600700 1 -- 172.21.15.60:0/3881955291 shutdown_connections 2024-05-03T14:45:30.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:30.916+0000 7f406d600700 1 -- 172.21.15.60:0/3881955291 wait complete. 2024-05-03T14:45:30.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:30.917+0000 7f406d600700 1 Processor -- start 2024-05-03T14:45:30.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:30.917+0000 7f406d600700 1 -- start start 2024-05-03T14:45:30.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:30.918+0000 7f406d600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f40681056b0 0x7f406819beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:45:30.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:30.918+0000 7f406d600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4068106070 0x7f406819c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:45:30.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:30.918+0000 7f406d600700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f406810b6b0 0x7f40681a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:45:30.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:30.918+0000 7f406d600700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f4068115b60 con 0x7f406810b6b0 2024-05-03T14:45:30.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:30.918+0000 7f406d600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f40681159e0 con 0x7f40681056b0 2024-05-03T14:45:30.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:30.918+0000 7f406d600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4068115ce0 con 0x7f4068106070 2024-05-03T14:45:30.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:30.918+0000 7f4066000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4068106070 0x7f406819c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:45:30.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:30.918+0000 7f4066000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4068106070 0x7f406819c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.60:55888/0 (socket says 172.21.15.60:55888) 2024-05-03T14:45:30.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:30.918+0000 7f4066000700 1 -- 172.21.15.60:0/243807511 learned_addr learned my addr 172.21.15.60:0/243807511 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:45:30.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:30.918+0000 7f4067400700 1 --2- 172.21.15.60:0/243807511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f406810b6b0 0x7f40681a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:45:30.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:30.918+0000 7f4066a00700 1 --2- 172.21.15.60:0/243807511 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f40681056b0 0x7f406819beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:45:30.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:30.919+0000 7f4066000700 1 -- 172.21.15.60:0/243807511 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f40681056b0 msgr2=0x7f406819beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:30.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:30.919+0000 7f4066000700 1 --2- 172.21.15.60:0/243807511 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f40681056b0 0x7f406819beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:30.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:30.919+0000 7f4066000700 1 -- 172.21.15.60:0/243807511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f406810b6b0 msgr2=0x7f40681a0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:30.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:30.919+0000 7f4066000700 1 --2- 172.21.15.60:0/243807511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f406810b6b0 0x7f40681a0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:30.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:30.919+0000 7f4066000700 1 -- 172.21.15.60:0/243807511 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40681a0ec0 con 0x7f4068106070 2024-05-03T14:45:30.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:30.919+0000 7f4066000700 1 --2- 172.21.15.60:0/243807511 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4068106070 0x7f406819c3f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f405400d980 tx=0x7f405400dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:45:30.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:30.920+0000 7f405f400700 1 -- 172.21.15.60:0/243807511 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4054016070 con 0x7f4068106070 2024-05-03T14:45:30.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:30.920+0000 7f405f400700 1 -- 172.21.15.60:0/243807511 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f4054004d30 con 0x7f4068106070 2024-05-03T14:45:30.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:30.920+0000 7f405f400700 1 -- 172.21.15.60:0/243807511 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40540056c0 con 0x7f4068106070 2024-05-03T14:45:30.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:30.920+0000 7f406d600700 1 -- 172.21.15.60:0/243807511 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40681a11b0 con 0x7f4068106070 2024-05-03T14:45:30.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:30.920+0000 7f406d600700 1 -- 172.21.15.60:0/243807511 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40681a8a30 con 0x7f4068106070 2024-05-03T14:45:30.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:30.921+0000 7f406d600700 1 -- 172.21.15.60:0/243807511 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f406806a190 con 0x7f4068106070 2024-05-03T14:45:30.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:30.925+0000 7f405f400700 1 -- 172.21.15.60:0/243807511 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f4054004750 con 0x7f4068106070 2024-05-03T14:45:30.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:30.926+0000 7f405f400700 1 --2- 172.21.15.60:0/243807511 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f404c0b2900 0x7f404c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:45:30.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:30.926+0000 7f405f400700 1 -- 172.21.15.60:0/243807511 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f40540e2780 con 0x7f4068106070 2024-05-03T14:45:30.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:30.927+0000 7f4066a00700 1 --2- 172.21.15.60:0/243807511 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f404c0b2900 0x7f404c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:45:30.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:30.927+0000 7f405f400700 1 -- 172.21.15.60:0/243807511 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f405401f020 con 0x7f4068106070 2024-05-03T14:45:30.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:30.928+0000 7f4066a00700 1 --2- 172.21.15.60:0/243807511 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f404c0b2900 0x7f404c0b4dc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4058009be0 tx=0x7f4058009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:45:31.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:30 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:45:31.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:30 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:45:31.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:30 smithi060 ceph-mon[28873]: pgmap v360: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-03T14:45:31.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:31.206+0000 7f406d600700 1 -- 172.21.15.60:0/243807511 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f406819d210 con 0x7f4068106070 2024-05-03T14:45:31.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:31.206+0000 7f405f400700 1 -- 172.21.15.60:0/243807511 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v87) ==== 81+0+586 (secure 0 0 0) 0x7f406819d210 con 0x7f4068106070 2024-05-03T14:45:31.209 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6801/424391417 2024-05-04T14:36:04.154333+0000 2024-05-03T14:45:31.209 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6800/424391417 2024-05-04T14:36:04.154333+0000 2024-05-03T14:45:31.209 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/1426532561 2024-05-04T14:36:04.154333+0000 2024-05-03T14:45:31.209 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/3230079926 2024-05-04T14:35:28.066961+0000 2024-05-03T14:45:31.209 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/2334895610 2024-05-04T14:35:28.066961+0000 2024-05-03T14:45:31.209 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6800/888447185 2024-05-04T14:35:28.066961+0000 2024-05-03T14:45:31.209 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/3996899937 2024-05-04T14:35:28.066961+0000 2024-05-03T14:45:31.209 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/2779644018 2024-05-04T14:36:04.154333+0000 2024-05-03T14:45:31.209 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/71440838 2024-05-04T14:36:04.154333+0000 2024-05-03T14:45:31.209 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6801/888447185 2024-05-04T14:35:28.066961+0000 2024-05-03T14:45:31.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:31.210+0000 7f406d600700 1 -- 172.21.15.60:0/243807511 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f404c0b2900 msgr2=0x7f404c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:31.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:31.210+0000 7f406d600700 1 --2- 172.21.15.60:0/243807511 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f404c0b2900 0x7f404c0b4dc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4058009be0 tx=0x7f4058009260 comp rx=0 tx=0).stop 2024-05-03T14:45:31.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:31.210+0000 7f406d600700 1 -- 172.21.15.60:0/243807511 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4068106070 msgr2=0x7f406819c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:31.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:31.210+0000 7f406d600700 1 --2- 172.21.15.60:0/243807511 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4068106070 0x7f406819c3f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f405400d980 tx=0x7f405400dc90 comp rx=0 tx=0).stop 2024-05-03T14:45:31.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:31.210+0000 7f406d600700 1 -- 172.21.15.60:0/243807511 shutdown_connections 2024-05-03T14:45:31.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:31.210+0000 7f406d600700 1 --2- 172.21.15.60:0/243807511 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f404c0b2900 0x7f404c0b4dc0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:31.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:31.210+0000 7f406d600700 1 --2- 172.21.15.60:0/243807511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f406810b6b0 0x7f40681a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:31.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:31.210+0000 7f406d600700 1 --2- 172.21.15.60:0/243807511 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f40681056b0 0x7f406819beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:31.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:31.210+0000 7f406d600700 1 --2- 172.21.15.60:0/243807511 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4068106070 0x7f406819c3f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:31.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:31.210+0000 7f406d600700 1 -- 172.21.15.60:0/243807511 >> 172.21.15.60:0/243807511 conn(0x7f40681011a0 msgr2=0x7f40681015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:45:31.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:31.210+0000 7f406d600700 1 -- 172.21.15.60:0/243807511 shutdown_connections 2024-05-03T14:45:31.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:31.210+0000 7f406d600700 1 -- 172.21.15.60:0/243807511 wait complete. 2024-05-03T14:45:31.213 INFO:teuthology.orchestra.run.smithi060.stderr:listed 10 entries 2024-05-03T14:45:31.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:30 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:45:31.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:30 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:45:31.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:30 smithi090 ceph-mon[33568]: pgmap v360: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-03T14:45:31.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:30 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:45:31.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:30 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:45:31.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:30 smithi149 ceph-mon[33332]: pgmap v360: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-03T14:45:32.021 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi060.front.sepia.ceph.com... 2024-05-03T14:45:32.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T14:45:32.022 DEBUG:teuthology.orchestra.run.smithi060:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-05-03T14:45:32.111 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi060.stderr:ceph-fuse[72103]: fuse finished with error 0 and tester_r 0 2024-05-03T14:45:32.118 INFO:teuthology.orchestra.run:waiting for 300 2024-05-03T14:45:32.253 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:31 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/243807511' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-03T14:45:32.253 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:31 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:45:32.253 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:31 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:45:32.253 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:31 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:45:32.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:31 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/243807511' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-03T14:45:32.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:31 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:45:32.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:31 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:45:32.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:31 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:45:32.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:31 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/243807511' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-03T14:45:32.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:31 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:45:32.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:31 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:45:32.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:31 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:45:33.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:32 smithi060 ceph-mon[28873]: pgmap v361: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-03T14:45:33.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:32 smithi090 ceph-mon[33568]: pgmap v361: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-03T14:45:33.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:32 smithi149 ceph-mon[33332]: pgmap v361: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-03T14:45:35.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:35 smithi149 ceph-mon[33332]: pgmap v362: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-03T14:45:35.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:35 smithi060 ceph-mon[28873]: pgmap v362: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-03T14:45:35.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:35 smithi090 ceph-mon[33568]: pgmap v362: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-03T14:45:37.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:37 smithi149 ceph-mon[33332]: pgmap v363: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-03T14:45:37.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:37 smithi060 ceph-mon[28873]: pgmap v363: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-03T14:45:37.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:37 smithi090 ceph-mon[33568]: pgmap v363: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-03T14:45:38.220 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2024-05-03T14:45:38.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T14:45:38.220 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-05-03T14:45:38.299 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-03T14:45:38.751 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:45:39.502 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:39 smithi060 ceph-mon[28873]: pgmap v364: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-03T14:45:39.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:39 smithi149 ceph-mon[33332]: pgmap v364: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-03T14:45:39.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:39 smithi090 ceph-mon[33568]: pgmap v364: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-03T14:45:41.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:41.157+0000 7f755a400700 1 -- 172.21.15.60:0/2030987390 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f755410fea0 msgr2=0x7f7554112290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:41.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:41.157+0000 7f755a400700 1 --2- 172.21.15.60:0/2030987390 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f755410fea0 0x7f7554112290 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f754c00b0a0 tx=0x7f754c00b3b0 comp rx=0 tx=0).stop 2024-05-03T14:45:41.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:41.157+0000 7f755a400700 1 -- 172.21.15.60:0/2030987390 shutdown_connections 2024-05-03T14:45:41.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:41.157+0000 7f755a400700 1 --2- 172.21.15.60:0/2030987390 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f755410fea0 0x7f7554112290 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:41.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:41.157+0000 7f755a400700 1 --2- 172.21.15.60:0/2030987390 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75540756d0 0x7f7554075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:41.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:41.157+0000 7f755a400700 1 --2- 172.21.15.60:0/2030987390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7554075ff0 0x7f755410f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:41.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:41.157+0000 7f755a400700 1 -- 172.21.15.60:0/2030987390 >> 172.21.15.60:0/2030987390 conn(0x7f755407a600 msgr2=0x7f755407aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:45:41.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:41.157+0000 7f755a400700 1 -- 172.21.15.60:0/2030987390 shutdown_connections 2024-05-03T14:45:41.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:41.157+0000 7f755a400700 1 -- 172.21.15.60:0/2030987390 wait complete. 2024-05-03T14:45:41.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:41.159+0000 7f755a400700 1 Processor -- start 2024-05-03T14:45:41.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:41.159+0000 7f755a400700 1 -- start start 2024-05-03T14:45:41.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:41.159+0000 7f755a400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f75540756d0 0x7f755419bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:45:41.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:41.159+0000 7f755a400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7554075ff0 0x7f755419c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:45:41.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:41.159+0000 7f755a400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f755410fea0 0x7f75541a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:45:41.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:41.159+0000 7f755a400700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f7554115b30 con 0x7f75540756d0 2024-05-03T14:45:41.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:41.159+0000 7f755a400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f75541159b0 con 0x7f7554075ff0 2024-05-03T14:45:41.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:41.159+0000 7f755a400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7554115cb0 con 0x7f755410fea0 2024-05-03T14:45:41.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:41.160+0000 7f7552a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7554075ff0 0x7f755419c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:45:41.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:41.160+0000 7f7553400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f75540756d0 0x7f755419bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:45:41.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:41.160+0000 7f7552a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7554075ff0 0x7f755419c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.60:34990/0 (socket says 172.21.15.60:34990) 2024-05-03T14:45:41.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:41.160+0000 7f7552a00700 1 -- 172.21.15.60:0/1372126912 learned_addr learned my addr 172.21.15.60:0/1372126912 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:45:41.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:41.160+0000 7f7553e00700 1 --2- 172.21.15.60:0/1372126912 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f755410fea0 0x7f75541a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:45:41.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:41.160+0000 7f7552a00700 1 -- 172.21.15.60:0/1372126912 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f755410fea0 msgr2=0x7f75541a0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:41.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:41.160+0000 7f7552a00700 1 --2- 172.21.15.60:0/1372126912 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f755410fea0 0x7f75541a0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:41.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:41.160+0000 7f7552a00700 1 -- 172.21.15.60:0/1372126912 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f75540756d0 msgr2=0x7f755419bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:41.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:41.160+0000 7f7552a00700 1 --2- 172.21.15.60:0/1372126912 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f75540756d0 0x7f755419bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:41.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:41.160+0000 7f7552a00700 1 -- 172.21.15.60:0/1372126912 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75541a0eb0 con 0x7f7554075ff0 2024-05-03T14:45:41.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:41.160+0000 7f7552a00700 1 --2- 172.21.15.60:0/1372126912 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7554075ff0 0x7f755419c3e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f754000e970 tx=0x7f754000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:45:41.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:41.161+0000 7f754be00700 1 -- 172.21.15.60:0/1372126912 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7540004220 con 0x7f7554075ff0 2024-05-03T14:45:41.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:41.161+0000 7f754be00700 1 -- 172.21.15.60:0/1372126912 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f754000cce0 con 0x7f7554075ff0 2024-05-03T14:45:41.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:41.161+0000 7f755a400700 1 -- 172.21.15.60:0/1372126912 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75541a11a0 con 0x7f7554075ff0 2024-05-03T14:45:41.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:41.161+0000 7f754be00700 1 -- 172.21.15.60:0/1372126912 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7540003e60 con 0x7f7554075ff0 2024-05-03T14:45:41.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:41.161+0000 7f755a400700 1 -- 172.21.15.60:0/1372126912 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75541a8a20 con 0x7f7554075ff0 2024-05-03T14:45:41.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:41.162+0000 7f755a400700 1 -- 172.21.15.60:0/1372126912 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f755406a190 con 0x7f7554075ff0 2024-05-03T14:45:41.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:41.165+0000 7f754be00700 1 -- 172.21.15.60:0/1372126912 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f754000c7e0 con 0x7f7554075ff0 2024-05-03T14:45:41.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:41.167+0000 7f754be00700 1 --2- 172.21.15.60:0/1372126912 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f75380b2900 0x7f75380b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:45:41.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:41.167+0000 7f754be00700 1 -- 172.21.15.60:0/1372126912 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f75400e2730 con 0x7f7554075ff0 2024-05-03T14:45:41.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:41.167+0000 7f7553400700 1 --2- 172.21.15.60:0/1372126912 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f75380b2900 0x7f75380b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:45:41.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:41.168+0000 7f7553400700 1 --2- 172.21.15.60:0/1372126912 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f75380b2900 0x7f75380b4dc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7544009680 tx=0x7f7544009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:45:41.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:41.171+0000 7f754be00700 1 -- 172.21.15.60:0/1372126912 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f75400aa090 con 0x7f7554075ff0 2024-05-03T14:45:41.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:41.410+0000 7f755a400700 1 -- 172.21.15.60:0/1372126912 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f755419d200 con 0x7f7554075ff0 2024-05-03T14:45:41.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:41.415+0000 7f754be00700 1 -- 172.21.15.60:0/1372126912 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24626 (secure 0 0 0) 0x7f755419d200 con 0x7f7554075ff0 2024-05-03T14:45:41.417 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:45:41.417 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":87,"fsid":"285d64b2-095a-11ef-bc97-c7b262605968","created":"2024-05-03T14:34:53.560460+0000","modified":"2024-05-03T14:45:05.402298+0000","last_up_change":"2024-05-03T14:43:14.877043+0000","last_in_change":"2024-05-03T14:42:55.085959+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-03T14:38:54.333714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-03T14:44:25.608599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-03T14:44:30.154009+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-03T14:44:42.549502+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f0797730-c4c1-47ea-9dce-9270d5ebb4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2952930162},{"type":"v1","addr":"172.21.15.60:6803","nonce":2952930162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2952930162},{"type":"v1","addr":"172.21.15.60:6805","nonce":2952930162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":2952930162},{"type":"v1","addr":"172.21.15.60:6809","nonce":2952930162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2952930162},{"type":"v1","addr":"172.21.15.60:6807","nonce":2952930162}]},"public_addr":"172.21.15.60:6803/2952930162","cluster_addr":"172.21.15.60:6805/2952930162","heartbeat_back_addr":"172.21.15.60:6809/2952930162","heartbeat_front_addr":"172.21.15.60:6807/2952930162","state":["exists","up"]},{"osd":1,"uuid":"96d0ad08-cda4-4824-bad4-e64973508c0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":720063242},{"type":"v1","addr":"172.21.15.90:6801","nonce":720063242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":720063242},{"type":"v1","addr":"172.21.15.90:6803","nonce":720063242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":720063242},{"type":"v1","addr":"172.21.15.90:6807","nonce":720063242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":720063242},{"type":"v1","addr":"172.21.15.90:6805","nonce":720063242}]},"public_addr":"172.21.15.90:6801/720063242","cluster_addr":"172.21.15.90:6803/720063242","heartbeat_back_addr":"172.21.15.90:6807/720063242","heartbeat_front_addr":"172.21.15.90:6805/720063242","state":["exists","up"]},{"osd":2,"uuid":"ce95fb7c-1611-4dd1-aee7-97c420d6a466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1359662077},{"type":"v1","addr":"172.21.15.149:6801","nonce":1359662077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1359662077},{"type":"v1","addr":"172.21.15.149:6803","nonce":1359662077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1359662077},{"type":"v1","addr":"172.21.15.149:6807","nonce":1359662077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1359662077},{"type":"v1","addr":"172.21.15.149:6805","nonce":1359662077}]},"public_addr":"172.21.15.149:6801/1359662077","cluster_addr":"172.21.15.149:6803/1359662077","heartbeat_back_addr":"172.21.15.149:6807/1359662077","heartbeat_front_addr":"172.21.15.149:6805/1359662077","state":["exists","up"]},{"osd":3,"uuid":"ef31136a-9b36-4b87-8389-f53d311bc5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3226133334},{"type":"v1","addr":"172.21.15.60:6811","nonce":3226133334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3226133334},{"type":"v1","addr":"172.21.15.60:6813","nonce":3226133334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3226133334},{"type":"v1","addr":"172.21.15.60:6817","nonce":3226133334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3226133334},{"type":"v1","addr":"172.21.15.60:6815","nonce":3226133334}]},"public_addr":"172.21.15.60:6811/3226133334","cluster_addr":"172.21.15.60:6813/3226133334","heartbeat_back_addr":"172.21.15.60:6817/3226133334","heartbeat_front_addr":"172.21.15.60:6815/3226133334","state":["exists","up"]},{"osd":4,"uuid":"73db7408-a2ff-47e9-b13d-d31688a93610","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2201391883},{"type":"v1","addr":"172.21.15.90:6809","nonce":2201391883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":2201391883},{"type":"v1","addr":"172.21.15.90:6811","nonce":2201391883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":2201391883},{"type":"v1","addr":"172.21.15.90:6815","nonce":2201391883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":2201391883},{"type":"v1","addr":"172.21.15.90:6813","nonce":2201391883}]},"public_addr":"172.21.15.90:6809/2201391883","cluster_addr":"172.21.15.90:6811/2201391883","heartbeat_back_addr":"172.21.15.90:6815/2201391883","heartbeat_front_addr":"172.21.15.90:6813/2201391883","state":["exists","up"]},{"osd":5,"uuid":"aefb78a2-522a-4b35-92b8-e5c04fa47b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2139732836},{"type":"v1","addr":"172.21.15.149:6809","nonce":2139732836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":2139732836},{"type":"v1","addr":"172.21.15.149:6811","nonce":2139732836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":2139732836},{"type":"v1","addr":"172.21.15.149:6815","nonce":2139732836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":2139732836},{"type":"v1","addr":"172.21.15.149:6813","nonce":2139732836}]},"public_addr":"172.21.15.149:6809/2139732836","cluster_addr":"172.21.15.149:6811/2139732836","heartbeat_back_addr":"172.21.15.149:6815/2139732836","heartbeat_front_addr":"172.21.15.149:6813/2139732836","state":["exists","up"]},{"osd":6,"uuid":"7c887819-6182-4636-8cfd-d845c30bd13c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3738521002},{"type":"v1","addr":"172.21.15.60:6819","nonce":3738521002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3738521002},{"type":"v1","addr":"172.21.15.60:6821","nonce":3738521002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3738521002},{"type":"v1","addr":"172.21.15.60:6825","nonce":3738521002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3738521002},{"type":"v1","addr":"172.21.15.60:6823","nonce":3738521002}]},"public_addr":"172.21.15.60:6819/3738521002","cluster_addr":"172.21.15.60:6821/3738521002","heartbeat_back_addr":"172.21.15.60:6825/3738521002","heartbeat_front_addr":"172.21.15.60:6823/3738521002","state":["exists","up"]},{"osd":7,"uuid":"ebce6acf-8873-4b35-bd5f-80de2163a307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":1686298661},{"type":"v1","addr":"172.21.15.90:6817","nonce":1686298661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":1686298661},{"type":"v1","addr":"172.21.15.90:6819","nonce":1686298661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":1686298661},{"type":"v1","addr":"172.21.15.90:6823","nonce":1686298661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1686298661},{"type":"v1","addr":"172.21.15.90:6821","nonce":1686298661}]},"public_addr":"172.21.15.90:6817/1686298661","cluster_addr":"172.21.15.90:6819/1686298661","heartbeat_back_addr":"172.21.15.90:6823/1686298661","heartbeat_front_addr":"172.21.15.90:6821/1686298661","state":["exists","up"]},{"osd":8,"uuid":"4ae04b98-e9b2-4956-83d9-17e5ac3c963f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":2608348248},{"type":"v1","addr":"172.21.15.149:6817","nonce":2608348248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2608348248},{"type":"v1","addr":"172.21.15.149:6819","nonce":2608348248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2608348248},{"type":"v1","addr":"172.21.15.149:6823","nonce":2608348248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2608348248},{"type":"v1","addr":"172.21.15.149:6821","nonce":2608348248}]},"public_addr":"172.21.15.149:6817/2608348248","cluster_addr":"172.21.15.149:6819/2608348248","heartbeat_back_addr":"172.21.15.149:6823/2608348248","heartbeat_front_addr":"172.21.15.149:6821/2608348248","state":["exists","up"]},{"osd":9,"uuid":"d0c0a70d-aa76-47e4-91e8-c343c8c41197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":2156130999},{"type":"v1","addr":"172.21.15.60:6827","nonce":2156130999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":2156130999},{"type":"v1","addr":"172.21.15.60:6829","nonce":2156130999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":2156130999},{"type":"v1","addr":"172.21.15.60:6833","nonce":2156130999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":2156130999},{"type":"v1","addr":"172.21.15.60:6831","nonce":2156130999}]},"public_addr":"172.21.15.60:6827/2156130999","cluster_addr":"172.21.15.60:6829/2156130999","heartbeat_back_addr":"172.21.15.60:6833/2156130999","heartbeat_front_addr":"172.21.15.60:6831/2156130999","state":["exists","up"]},{"osd":10,"uuid":"cdc3d7b8-33c7-4cc4-be4e-6887d15a3d8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":2492593590},{"type":"v1","addr":"172.21.15.90:6825","nonce":2492593590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":2492593590},{"type":"v1","addr":"172.21.15.90:6827","nonce":2492593590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":2492593590},{"type":"v1","addr":"172.21.15.90:6831","nonce":2492593590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":2492593590},{"type":"v1","addr":"172.21.15.90:6829","nonce":2492593590}]},"public_addr":"172.21.15.90:6825/2492593590","cluster_addr":"172.21.15.90:6827/2492593590","heartbeat_back_addr":"172.21.15.90:6831/2492593590","heartbeat_front_addr":"172.21.15.90:6829/2492593590","state":["exists","up"]},{"osd":11,"uuid":"8d17ec1a-bdd5-41f1-9b59-c2b6a527bfbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":1063245126},{"type":"v1","addr":"172.21.15.149:6825","nonce":1063245126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":1063245126},{"type":"v1","addr":"172.21.15.149:6827","nonce":1063245126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":1063245126},{"type":"v1","addr":"172.21.15.149:6831","nonce":1063245126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":1063245126},{"type":"v1","addr":"172.21.15.149:6829","nonce":1063245126}]},"public_addr":"172.21.15.149:6825/1063245126","cluster_addr":"172.21.15.149:6827/1063245126","heartbeat_back_addr":"172.21.15.149:6831/1063245126","heartbeat_front_addr":"172.21.15.149:6829/1063245126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:38:02.056563+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:38:27.140677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:38:51.287070+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:39:25.484410+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:39:51.483133+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:40:17.200423+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:40:51.979804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:41:18.742542+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:41:44.922541+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:42:19.778451+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:42:47.438440+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:43:13.429419+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6801/424391417":"2024-05-04T14:36:04.154333+0000","172.21.15.60:6800/424391417":"2024-05-04T14:36:04.154333+0000","172.21.15.60:0/1426532561":"2024-05-04T14:36:04.154333+0000","172.21.15.60:0/3230079926":"2024-05-04T14:35:28.066961+0000","172.21.15.60:0/2334895610":"2024-05-04T14:35:28.066961+0000","172.21.15.60:6800/888447185":"2024-05-04T14:35:28.066961+0000","172.21.15.60:0/3996899937":"2024-05-04T14:35:28.066961+0000","172.21.15.60:0/2779644018":"2024-05-04T14:36:04.154333+0000","172.21.15.60:0/71440838":"2024-05-04T14:36:04.154333+0000","172.21.15.60:6801/888447185":"2024-05-04T14:35:28.066961+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-03T14:45:41.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:41.418+0000 7f755a400700 1 -- 172.21.15.60:0/1372126912 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f75380b2900 msgr2=0x7f75380b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:41.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:41.418+0000 7f755a400700 1 --2- 172.21.15.60:0/1372126912 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f75380b2900 0x7f75380b4dc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7544009680 tx=0x7f7544009260 comp rx=0 tx=0).stop 2024-05-03T14:45:41.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:41.418+0000 7f755a400700 1 -- 172.21.15.60:0/1372126912 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7554075ff0 msgr2=0x7f755419c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:41.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:41.418+0000 7f755a400700 1 --2- 172.21.15.60:0/1372126912 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7554075ff0 0x7f755419c3e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f754000e970 tx=0x7f754000ec80 comp rx=0 tx=0).stop 2024-05-03T14:45:41.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:41.419+0000 7f755a400700 1 -- 172.21.15.60:0/1372126912 shutdown_connections 2024-05-03T14:45:41.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:41.419+0000 7f755a400700 1 --2- 172.21.15.60:0/1372126912 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f75380b2900 0x7f75380b4dc0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:41.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:41.419+0000 7f755a400700 1 --2- 172.21.15.60:0/1372126912 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f755410fea0 0x7f75541a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:41.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:41.419+0000 7f755a400700 1 --2- 172.21.15.60:0/1372126912 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f75540756d0 0x7f755419bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:41.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:41.419+0000 7f755a400700 1 --2- 172.21.15.60:0/1372126912 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7554075ff0 0x7f755419c3e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:41.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:41.419+0000 7f755a400700 1 -- 172.21.15.60:0/1372126912 >> 172.21.15.60:0/1372126912 conn(0x7f755407a600 msgr2=0x7f7554100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:45:41.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:41.419+0000 7f755a400700 1 -- 172.21.15.60:0/1372126912 shutdown_connections 2024-05-03T14:45:41.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:41.419+0000 7f755a400700 1 -- 172.21.15.60:0/1372126912 wait complete. 2024-05-03T14:45:41.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:41 smithi149 ceph-mon[33332]: pgmap v365: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-03T14:45:41.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:41 smithi060 ceph-mon[28873]: pgmap v365: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-03T14:45:41.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:41 smithi090 ceph-mon[33568]: pgmap v365: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-03T14:45:42.476 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-03T14:45:42.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:42 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/1372126912' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T14:45:42.836 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:42 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/1372126912' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T14:45:42.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:42 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/1372126912' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T14:45:42.954 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:45:43.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:43 smithi149 ceph-mon[33332]: pgmap v366: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-03T14:45:43.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:43 smithi060 ceph-mon[28873]: pgmap v366: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-03T14:45:43.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:43 smithi090 ceph-mon[33568]: pgmap v366: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-03T14:45:45.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:45.081+0000 7f3071e00700 1 -- 172.21.15.60:0/3396930837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f306c10ff00 msgr2=0x7f306c1122f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:45.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:45.081+0000 7f3071e00700 1 --2- 172.21.15.60:0/3396930837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f306c10ff00 0x7f306c1122f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f306400b0a0 tx=0x7f306400b3b0 comp rx=0 tx=0).stop 2024-05-03T14:45:45.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:45.081+0000 7f3071e00700 1 -- 172.21.15.60:0/3396930837 shutdown_connections 2024-05-03T14:45:45.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:45.081+0000 7f3071e00700 1 --2- 172.21.15.60:0/3396930837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f306c10ff00 0x7f306c1122f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:45.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:45.081+0000 7f3071e00700 1 --2- 172.21.15.60:0/3396930837 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f306c0756d0 0x7f306c075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:45.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:45.081+0000 7f3071e00700 1 --2- 172.21.15.60:0/3396930837 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f306c075ff0 0x7f306c10f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:45.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:45.081+0000 7f3071e00700 1 -- 172.21.15.60:0/3396930837 >> 172.21.15.60:0/3396930837 conn(0x7f306c07a600 msgr2=0x7f306c07aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:45:45.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:45.081+0000 7f3071e00700 1 -- 172.21.15.60:0/3396930837 shutdown_connections 2024-05-03T14:45:45.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:45.081+0000 7f3071e00700 1 -- 172.21.15.60:0/3396930837 wait complete. 2024-05-03T14:45:45.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:45.083+0000 7f3071e00700 1 Processor -- start 2024-05-03T14:45:45.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:45.083+0000 7f3071e00700 1 -- start start 2024-05-03T14:45:45.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:45.084+0000 7f3071e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f306c0756d0 0x7f306c19bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:45:45.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:45.084+0000 7f3071e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f306c075ff0 0x7f306c19c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:45:45.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:45.084+0000 7f3071e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f306c10ff00 0x7f306c1a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:45:45.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:45.084+0000 7f3071e00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f306c115b60 con 0x7f306c075ff0 2024-05-03T14:45:45.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:45.084+0000 7f3071e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f306c1159e0 con 0x7f306c0756d0 2024-05-03T14:45:45.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:45.084+0000 7f3071e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f306c115ce0 con 0x7f306c10ff00 2024-05-03T14:45:45.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:45.084+0000 7f306a000700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f306c075ff0 0x7f306c19c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:45:45.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:45.084+0000 7f306aa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f306c0756d0 0x7f306c19bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:45:45.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:45.084+0000 7f306aa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f306c0756d0 0x7f306c19bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.60:44348/0 (socket says 172.21.15.60:44348) 2024-05-03T14:45:45.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:45.084+0000 7f306aa00700 1 -- 172.21.15.60:0/4267381562 learned_addr learned my addr 172.21.15.60:0/4267381562 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:45:45.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:45.084+0000 7f306b400700 1 --2- 172.21.15.60:0/4267381562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f306c10ff00 0x7f306c1a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:45:45.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:45.085+0000 7f306aa00700 1 -- 172.21.15.60:0/4267381562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f306c10ff00 msgr2=0x7f306c1a0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:45.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:45.085+0000 7f306aa00700 1 --2- 172.21.15.60:0/4267381562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f306c10ff00 0x7f306c1a0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:45.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:45.085+0000 7f306aa00700 1 -- 172.21.15.60:0/4267381562 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f306c075ff0 msgr2=0x7f306c19c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:45.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:45.085+0000 7f306aa00700 1 --2- 172.21.15.60:0/4267381562 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f306c075ff0 0x7f306c19c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:45.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:45.085+0000 7f306aa00700 1 -- 172.21.15.60:0/4267381562 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f306c1a0e90 con 0x7f306c0756d0 2024-05-03T14:45:45.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:45.085+0000 7f306a000700 1 --2- 172.21.15.60:0/4267381562 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f306c075ff0 0x7f306c19c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:45:45.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:45.085+0000 7f306aa00700 1 --2- 172.21.15.60:0/4267381562 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f306c0756d0 0x7f306c19bed0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f305c00e9a0 tx=0x7f305c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:45:45.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:45.085+0000 7f3070a00700 1 -- 172.21.15.60:0/4267381562 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305c00ca10 con 0x7f306c0756d0 2024-05-03T14:45:45.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:45.085+0000 7f3070a00700 1 -- 172.21.15.60:0/4267381562 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f305c00cbb0 con 0x7f306c0756d0 2024-05-03T14:45:45.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:45.085+0000 7f3071e00700 1 -- 172.21.15.60:0/4267381562 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f306c1a1180 con 0x7f306c0756d0 2024-05-03T14:45:45.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:45.085+0000 7f3070a00700 1 -- 172.21.15.60:0/4267381562 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305c0187a0 con 0x7f306c0756d0 2024-05-03T14:45:45.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:45.086+0000 7f3071e00700 1 -- 172.21.15.60:0/4267381562 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f306c1a8a00 con 0x7f306c0756d0 2024-05-03T14:45:45.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:45.087+0000 7f3071e00700 1 -- 172.21.15.60:0/4267381562 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f306c06a190 con 0x7f306c0756d0 2024-05-03T14:45:45.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:45.089+0000 7f3070a00700 1 -- 172.21.15.60:0/4267381562 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f305c010430 con 0x7f306c0756d0 2024-05-03T14:45:45.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:45.091+0000 7f3070a00700 1 --2- 172.21.15.60:0/4267381562 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f30500b2900 0x7f30500b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:45:45.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:45.091+0000 7f3070a00700 1 -- 172.21.15.60:0/4267381562 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f305c0e2940 con 0x7f306c0756d0 2024-05-03T14:45:45.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:45.091+0000 7f306a000700 1 --2- 172.21.15.60:0/4267381562 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f30500b2900 0x7f30500b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:45:45.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:45.092+0000 7f306a000700 1 --2- 172.21.15.60:0/4267381562 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f30500b2900 0x7f30500b4dc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f306c07b070 tx=0x7f3058009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:45:45.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:45.093+0000 7f3070a00700 1 -- 172.21.15.60:0/4267381562 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f305c0aa2a0 con 0x7f306c0756d0 2024-05-03T14:45:45.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:45.334+0000 7f3071e00700 1 -- 172.21.15.60:0/4267381562 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f306c1a8c30 con 0x7f306c0756d0 2024-05-03T14:45:45.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:45.336+0000 7f3070a00700 1 -- 172.21.15.60:0/4267381562 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24626 (secure 0 0 0) 0x7f305c0b2320 con 0x7f306c0756d0 2024-05-03T14:45:45.338 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:45:45.339 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":87,"fsid":"285d64b2-095a-11ef-bc97-c7b262605968","created":"2024-05-03T14:34:53.560460+0000","modified":"2024-05-03T14:45:05.402298+0000","last_up_change":"2024-05-03T14:43:14.877043+0000","last_in_change":"2024-05-03T14:42:55.085959+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-03T14:38:54.333714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-03T14:44:25.608599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-03T14:44:30.154009+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-03T14:44:42.549502+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f0797730-c4c1-47ea-9dce-9270d5ebb4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2952930162},{"type":"v1","addr":"172.21.15.60:6803","nonce":2952930162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2952930162},{"type":"v1","addr":"172.21.15.60:6805","nonce":2952930162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":2952930162},{"type":"v1","addr":"172.21.15.60:6809","nonce":2952930162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2952930162},{"type":"v1","addr":"172.21.15.60:6807","nonce":2952930162}]},"public_addr":"172.21.15.60:6803/2952930162","cluster_addr":"172.21.15.60:6805/2952930162","heartbeat_back_addr":"172.21.15.60:6809/2952930162","heartbeat_front_addr":"172.21.15.60:6807/2952930162","state":["exists","up"]},{"osd":1,"uuid":"96d0ad08-cda4-4824-bad4-e64973508c0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":720063242},{"type":"v1","addr":"172.21.15.90:6801","nonce":720063242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":720063242},{"type":"v1","addr":"172.21.15.90:6803","nonce":720063242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":720063242},{"type":"v1","addr":"172.21.15.90:6807","nonce":720063242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":720063242},{"type":"v1","addr":"172.21.15.90:6805","nonce":720063242}]},"public_addr":"172.21.15.90:6801/720063242","cluster_addr":"172.21.15.90:6803/720063242","heartbeat_back_addr":"172.21.15.90:6807/720063242","heartbeat_front_addr":"172.21.15.90:6805/720063242","state":["exists","up"]},{"osd":2,"uuid":"ce95fb7c-1611-4dd1-aee7-97c420d6a466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1359662077},{"type":"v1","addr":"172.21.15.149:6801","nonce":1359662077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1359662077},{"type":"v1","addr":"172.21.15.149:6803","nonce":1359662077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1359662077},{"type":"v1","addr":"172.21.15.149:6807","nonce":1359662077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1359662077},{"type":"v1","addr":"172.21.15.149:6805","nonce":1359662077}]},"public_addr":"172.21.15.149:6801/1359662077","cluster_addr":"172.21.15.149:6803/1359662077","heartbeat_back_addr":"172.21.15.149:6807/1359662077","heartbeat_front_addr":"172.21.15.149:6805/1359662077","state":["exists","up"]},{"osd":3,"uuid":"ef31136a-9b36-4b87-8389-f53d311bc5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3226133334},{"type":"v1","addr":"172.21.15.60:6811","nonce":3226133334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3226133334},{"type":"v1","addr":"172.21.15.60:6813","nonce":3226133334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3226133334},{"type":"v1","addr":"172.21.15.60:6817","nonce":3226133334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3226133334},{"type":"v1","addr":"172.21.15.60:6815","nonce":3226133334}]},"public_addr":"172.21.15.60:6811/3226133334","cluster_addr":"172.21.15.60:6813/3226133334","heartbeat_back_addr":"172.21.15.60:6817/3226133334","heartbeat_front_addr":"172.21.15.60:6815/3226133334","state":["exists","up"]},{"osd":4,"uuid":"73db7408-a2ff-47e9-b13d-d31688a93610","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2201391883},{"type":"v1","addr":"172.21.15.90:6809","nonce":2201391883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":2201391883},{"type":"v1","addr":"172.21.15.90:6811","nonce":2201391883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":2201391883},{"type":"v1","addr":"172.21.15.90:6815","nonce":2201391883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":2201391883},{"type":"v1","addr":"172.21.15.90:6813","nonce":2201391883}]},"public_addr":"172.21.15.90:6809/2201391883","cluster_addr":"172.21.15.90:6811/2201391883","heartbeat_back_addr":"172.21.15.90:6815/2201391883","heartbeat_front_addr":"172.21.15.90:6813/2201391883","state":["exists","up"]},{"osd":5,"uuid":"aefb78a2-522a-4b35-92b8-e5c04fa47b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2139732836},{"type":"v1","addr":"172.21.15.149:6809","nonce":2139732836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":2139732836},{"type":"v1","addr":"172.21.15.149:6811","nonce":2139732836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":2139732836},{"type":"v1","addr":"172.21.15.149:6815","nonce":2139732836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":2139732836},{"type":"v1","addr":"172.21.15.149:6813","nonce":2139732836}]},"public_addr":"172.21.15.149:6809/2139732836","cluster_addr":"172.21.15.149:6811/2139732836","heartbeat_back_addr":"172.21.15.149:6815/2139732836","heartbeat_front_addr":"172.21.15.149:6813/2139732836","state":["exists","up"]},{"osd":6,"uuid":"7c887819-6182-4636-8cfd-d845c30bd13c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3738521002},{"type":"v1","addr":"172.21.15.60:6819","nonce":3738521002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3738521002},{"type":"v1","addr":"172.21.15.60:6821","nonce":3738521002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3738521002},{"type":"v1","addr":"172.21.15.60:6825","nonce":3738521002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3738521002},{"type":"v1","addr":"172.21.15.60:6823","nonce":3738521002}]},"public_addr":"172.21.15.60:6819/3738521002","cluster_addr":"172.21.15.60:6821/3738521002","heartbeat_back_addr":"172.21.15.60:6825/3738521002","heartbeat_front_addr":"172.21.15.60:6823/3738521002","state":["exists","up"]},{"osd":7,"uuid":"ebce6acf-8873-4b35-bd5f-80de2163a307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":1686298661},{"type":"v1","addr":"172.21.15.90:6817","nonce":1686298661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":1686298661},{"type":"v1","addr":"172.21.15.90:6819","nonce":1686298661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":1686298661},{"type":"v1","addr":"172.21.15.90:6823","nonce":1686298661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1686298661},{"type":"v1","addr":"172.21.15.90:6821","nonce":1686298661}]},"public_addr":"172.21.15.90:6817/1686298661","cluster_addr":"172.21.15.90:6819/1686298661","heartbeat_back_addr":"172.21.15.90:6823/1686298661","heartbeat_front_addr":"172.21.15.90:6821/1686298661","state":["exists","up"]},{"osd":8,"uuid":"4ae04b98-e9b2-4956-83d9-17e5ac3c963f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":2608348248},{"type":"v1","addr":"172.21.15.149:6817","nonce":2608348248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2608348248},{"type":"v1","addr":"172.21.15.149:6819","nonce":2608348248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2608348248},{"type":"v1","addr":"172.21.15.149:6823","nonce":2608348248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2608348248},{"type":"v1","addr":"172.21.15.149:6821","nonce":2608348248}]},"public_addr":"172.21.15.149:6817/2608348248","cluster_addr":"172.21.15.149:6819/2608348248","heartbeat_back_addr":"172.21.15.149:6823/2608348248","heartbeat_front_addr":"172.21.15.149:6821/2608348248","state":["exists","up"]},{"osd":9,"uuid":"d0c0a70d-aa76-47e4-91e8-c343c8c41197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":2156130999},{"type":"v1","addr":"172.21.15.60:6827","nonce":2156130999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":2156130999},{"type":"v1","addr":"172.21.15.60:6829","nonce":2156130999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":2156130999},{"type":"v1","addr":"172.21.15.60:6833","nonce":2156130999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":2156130999},{"type":"v1","addr":"172.21.15.60:6831","nonce":2156130999}]},"public_addr":"172.21.15.60:6827/2156130999","cluster_addr":"172.21.15.60:6829/2156130999","heartbeat_back_addr":"172.21.15.60:6833/2156130999","heartbeat_front_addr":"172.21.15.60:6831/2156130999","state":["exists","up"]},{"osd":10,"uuid":"cdc3d7b8-33c7-4cc4-be4e-6887d15a3d8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":2492593590},{"type":"v1","addr":"172.21.15.90:6825","nonce":2492593590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":2492593590},{"type":"v1","addr":"172.21.15.90:6827","nonce":2492593590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":2492593590},{"type":"v1","addr":"172.21.15.90:6831","nonce":2492593590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":2492593590},{"type":"v1","addr":"172.21.15.90:6829","nonce":2492593590}]},"public_addr":"172.21.15.90:6825/2492593590","cluster_addr":"172.21.15.90:6827/2492593590","heartbeat_back_addr":"172.21.15.90:6831/2492593590","heartbeat_front_addr":"172.21.15.90:6829/2492593590","state":["exists","up"]},{"osd":11,"uuid":"8d17ec1a-bdd5-41f1-9b59-c2b6a527bfbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":1063245126},{"type":"v1","addr":"172.21.15.149:6825","nonce":1063245126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":1063245126},{"type":"v1","addr":"172.21.15.149:6827","nonce":1063245126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":1063245126},{"type":"v1","addr":"172.21.15.149:6831","nonce":1063245126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":1063245126},{"type":"v1","addr":"172.21.15.149:6829","nonce":1063245126}]},"public_addr":"172.21.15.149:6825/1063245126","cluster_addr":"172.21.15.149:6827/1063245126","heartbeat_back_addr":"172.21.15.149:6831/1063245126","heartbeat_front_addr":"172.21.15.149:6829/1063245126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:38:02.056563+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:38:27.140677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:38:51.287070+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:39:25.484410+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:39:51.483133+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:40:17.200423+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:40:51.979804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:41:18.742542+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:41:44.922541+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:42:19.778451+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:42:47.438440+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:43:13.429419+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6801/424391417":"2024-05-04T14:36:04.154333+0000","172.21.15.60:6800/424391417":"2024-05-04T14:36:04.154333+0000","172.21.15.60:0/1426532561":"2024-05-04T14:36:04.154333+0000","172.21.15.60:0/3230079926":"2024-05-04T14:35:28.066961+0000","172.21.15.60:0/2334895610":"2024-05-04T14:35:28.066961+0000","172.21.15.60:6800/888447185":"2024-05-04T14:35:28.066961+0000","172.21.15.60:0/3996899937":"2024-05-04T14:35:28.066961+0000","172.21.15.60:0/2779644018":"2024-05-04T14:36:04.154333+0000","172.21.15.60:0/71440838":"2024-05-04T14:36:04.154333+0000","172.21.15.60:6801/888447185":"2024-05-04T14:35:28.066961+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-03T14:45:45.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:45.339+0000 7f3071e00700 1 -- 172.21.15.60:0/4267381562 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f30500b2900 msgr2=0x7f30500b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:45.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:45.339+0000 7f3071e00700 1 --2- 172.21.15.60:0/4267381562 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f30500b2900 0x7f30500b4dc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f306c07b070 tx=0x7f3058009260 comp rx=0 tx=0).stop 2024-05-03T14:45:45.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:45.339+0000 7f3071e00700 1 -- 172.21.15.60:0/4267381562 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f306c0756d0 msgr2=0x7f306c19bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:45.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:45.339+0000 7f3071e00700 1 --2- 172.21.15.60:0/4267381562 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f306c0756d0 0x7f306c19bed0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f305c00e9a0 tx=0x7f305c00ecb0 comp rx=0 tx=0).stop 2024-05-03T14:45:45.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:45.339+0000 7f3071e00700 1 -- 172.21.15.60:0/4267381562 shutdown_connections 2024-05-03T14:45:45.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:45.339+0000 7f3071e00700 1 --2- 172.21.15.60:0/4267381562 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f30500b2900 0x7f30500b4dc0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:45.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:45.339+0000 7f3071e00700 1 --2- 172.21.15.60:0/4267381562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f306c10ff00 0x7f306c1a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:45.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:45.339+0000 7f3071e00700 1 --2- 172.21.15.60:0/4267381562 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f306c0756d0 0x7f306c19bed0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:45.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:45.339+0000 7f3071e00700 1 --2- 172.21.15.60:0/4267381562 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f306c075ff0 0x7f306c19c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:45.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:45.340+0000 7f3071e00700 1 -- 172.21.15.60:0/4267381562 >> 172.21.15.60:0/4267381562 conn(0x7f306c07a600 msgr2=0x7f306c100570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:45:45.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:45.340+0000 7f3071e00700 1 -- 172.21.15.60:0/4267381562 shutdown_connections 2024-05-03T14:45:45.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:45.340+0000 7f3071e00700 1 -- 172.21.15.60:0/4267381562 wait complete. 2024-05-03T14:45:45.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:45 smithi060 ceph-mon[28873]: pgmap v367: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-03T14:45:45.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:45 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/4267381562' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T14:45:45.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:45 smithi090 ceph-mon[33568]: pgmap v367: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-03T14:45:45.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:45 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/4267381562' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T14:45:45.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:45 smithi149 ceph-mon[33332]: pgmap v367: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-03T14:45:45.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:45 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/4267381562' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T14:45:46.232 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-05-03T14:45:46.713 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:45:47.692 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:47 smithi060 ceph-mon[28873]: pgmap v368: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-03T14:45:47.726 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:47 smithi149 ceph-mon[33332]: pgmap v368: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-03T14:45:47.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:47 smithi090 ceph-mon[33568]: pgmap v368: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-03T14:45:48.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:48.806+0000 7f1536600700 1 -- 172.21.15.60:0/4049030408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1530104eb0 msgr2=0x7f153010d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:48.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:48.806+0000 7f1536600700 1 --2- 172.21.15.60:0/4049030408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1530104eb0 0x7f153010d3a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f151c009990 tx=0x7f151c009ca0 comp rx=0 tx=0).stop 2024-05-03T14:45:48.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:48.806+0000 7f1536600700 1 -- 172.21.15.60:0/4049030408 shutdown_connections 2024-05-03T14:45:48.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:48.806+0000 7f1536600700 1 --2- 172.21.15.60:0/4049030408 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f153010d9c0 0x7f153010fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:48.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:48.806+0000 7f1536600700 1 --2- 172.21.15.60:0/4049030408 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1530104590 0x7f1530104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:48.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:48.806+0000 7f1536600700 1 --2- 172.21.15.60:0/4049030408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1530104eb0 0x7f153010d3a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:48.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:48.806+0000 7f1536600700 1 -- 172.21.15.60:0/4049030408 >> 172.21.15.60:0/4049030408 conn(0x7f15300fddc0 msgr2=0x7f15301001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:45:48.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:48.806+0000 7f1536600700 1 -- 172.21.15.60:0/4049030408 shutdown_connections 2024-05-03T14:45:48.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:48.806+0000 7f1536600700 1 -- 172.21.15.60:0/4049030408 wait complete. 2024-05-03T14:45:48.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:48.808+0000 7f1536600700 1 Processor -- start 2024-05-03T14:45:48.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:48.808+0000 7f1536600700 1 -- start start 2024-05-03T14:45:48.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:48.809+0000 7f1536600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1530104590 0x7f153019bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:45:48.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:48.809+0000 7f1536600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1530104eb0 0x7f153019c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:45:48.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:48.809+0000 7f1536600700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f153010d9c0 0x7f15301a0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:45:48.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:48.809+0000 7f1536600700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f1530113870 con 0x7f153010d9c0 2024-05-03T14:45:48.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:48.809+0000 7f1536600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f15301136f0 con 0x7f1530104590 2024-05-03T14:45:48.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:48.809+0000 7f1536600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f15301139f0 con 0x7f1530104eb0 2024-05-03T14:45:48.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:48.809+0000 7f152fe00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1530104eb0 0x7f153019c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:45:48.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:48.809+0000 7f152fe00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1530104eb0 0x7f153019c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.60:44672/0 (socket says 172.21.15.60:44672) 2024-05-03T14:45:48.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:48.809+0000 7f152fe00700 1 -- 172.21.15.60:0/3752751035 learned_addr learned my addr 172.21.15.60:0/3752751035 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:45:48.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:48.809+0000 7f1535200700 1 --2- 172.21.15.60:0/3752751035 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1530104590 0x7f153019bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:45:48.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:48.809+0000 7f1535c00700 1 --2- 172.21.15.60:0/3752751035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f153010d9c0 0x7f15301a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:45:48.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:48.809+0000 7f152fe00700 1 -- 172.21.15.60:0/3752751035 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1530104590 msgr2=0x7f153019bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:48.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:48.809+0000 7f152fe00700 1 --2- 172.21.15.60:0/3752751035 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1530104590 0x7f153019bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:48.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:48.809+0000 7f152fe00700 1 -- 172.21.15.60:0/3752751035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f153010d9c0 msgr2=0x7f15301a0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:48.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:48.809+0000 7f152fe00700 1 --2- 172.21.15.60:0/3752751035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f153010d9c0 0x7f15301a0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:48.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:48.809+0000 7f152fe00700 1 -- 172.21.15.60:0/3752751035 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15301a0f50 con 0x7f1530104eb0 2024-05-03T14:45:48.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:48.810+0000 7f152fe00700 1 --2- 172.21.15.60:0/3752751035 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1530104eb0 0x7f153019c480 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f151c000c00 tx=0x7f151c005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:45:48.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:48.810+0000 7f152d600700 1 -- 172.21.15.60:0/3752751035 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151c027070 con 0x7f1530104eb0 2024-05-03T14:45:48.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:48.810+0000 7f152d600700 1 -- 172.21.15.60:0/3752751035 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f151c00fcf0 con 0x7f1530104eb0 2024-05-03T14:45:48.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:48.810+0000 7f1536600700 1 -- 172.21.15.60:0/3752751035 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15301a11e0 con 0x7f1530104eb0 2024-05-03T14:45:48.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:48.810+0000 7f152d600700 1 -- 172.21.15.60:0/3752751035 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151c0173f0 con 0x7f1530104eb0 2024-05-03T14:45:48.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:48.810+0000 7f1536600700 1 -- 172.21.15.60:0/3752751035 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15301a8ac0 con 0x7f1530104eb0 2024-05-03T14:45:48.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:48.811+0000 7f1536600700 1 -- 172.21.15.60:0/3752751035 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f153006a190 con 0x7f1530104eb0 2024-05-03T14:45:48.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:48.814+0000 7f152d600700 1 -- 172.21.15.60:0/3752751035 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f151c02a070 con 0x7f1530104eb0 2024-05-03T14:45:48.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:48.815+0000 7f152d600700 1 --2- 172.21.15.60:0/3752751035 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f15180b2880 0x7f15180b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:45:48.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:48.815+0000 7f152d600700 1 -- 172.21.15.60:0/3752751035 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f151c0e3230 con 0x7f1530104eb0 2024-05-03T14:45:48.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:48.815+0000 7f1535200700 1 --2- 172.21.15.60:0/3752751035 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f15180b2880 0x7f15180b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:45:48.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:48.816+0000 7f1535200700 1 --2- 172.21.15.60:0/3752751035 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f15180b2880 0x7f15180b4d40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1524009c60 tx=0x7f1524009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:45:48.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:48.817+0000 7f152d600700 1 -- 172.21.15.60:0/3752751035 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f151c013070 con 0x7f1530104eb0 2024-05-03T14:45:49.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:49.106+0000 7f1536600700 1 -- 172.21.15.60:0/3752751035 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f15301a8c90 con 0x7f1530104eb0 2024-05-03T14:45:49.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:49.439+0000 7f152d600700 1 -- 172.21.15.60:0/3752751035 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f151c0aac20 con 0x7f1530104eb0 2024-05-03T14:45:49.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:49.446+0000 7f1536600700 1 -- 172.21.15.60:0/3752751035 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f15180b2880 msgr2=0x7f15180b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:49.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:49.446+0000 7f1536600700 1 --2- 172.21.15.60:0/3752751035 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f15180b2880 0x7f15180b4d40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1524009c60 tx=0x7f1524009260 comp rx=0 tx=0).stop 2024-05-03T14:45:49.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:49.446+0000 7f1536600700 1 -- 172.21.15.60:0/3752751035 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1530104eb0 msgr2=0x7f153019c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:49.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:49.446+0000 7f1536600700 1 --2- 172.21.15.60:0/3752751035 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1530104eb0 0x7f153019c480 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f151c000c00 tx=0x7f151c005ef0 comp rx=0 tx=0).stop 2024-05-03T14:45:49.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:49.446+0000 7f1536600700 1 -- 172.21.15.60:0/3752751035 shutdown_connections 2024-05-03T14:45:49.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:49.446+0000 7f1536600700 1 --2- 172.21.15.60:0/3752751035 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f15180b2880 0x7f15180b4d40 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:49.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:49.446+0000 7f1536600700 1 --2- 172.21.15.60:0/3752751035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f153010d9c0 0x7f15301a0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:49.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:49.446+0000 7f1536600700 1 --2- 172.21.15.60:0/3752751035 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1530104590 0x7f153019bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:49.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:49.446+0000 7f1536600700 1 --2- 172.21.15.60:0/3752751035 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1530104eb0 0x7f153019c480 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:49.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:49.446+0000 7f1536600700 1 -- 172.21.15.60:0/3752751035 >> 172.21.15.60:0/3752751035 conn(0x7f15300fddc0 msgr2=0x7f153010fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:45:49.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:49.447+0000 7f1536600700 1 -- 172.21.15.60:0/3752751035 shutdown_connections 2024-05-03T14:45:49.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:49.447+0000 7f1536600700 1 -- 172.21.15.60:0/3752751035 wait complete. 2024-05-03T14:45:49.824 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:49 smithi060 ceph-mon[28873]: pgmap v369: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:45:49.824 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:49 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/3752751035' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-03T14:45:49.824 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:49 smithi060 ceph-mon[28873]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-03T14:45:49.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:49 smithi090 ceph-mon[33568]: pgmap v369: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:45:49.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:49 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/3752751035' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-03T14:45:49.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:49 smithi090 ceph-mon[33568]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-03T14:45:49.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:49 smithi149 ceph-mon[33332]: pgmap v369: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:45:49.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:49 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/3752751035' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-03T14:45:49.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:49 smithi149 ceph-mon[33332]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-03T14:45:50.354 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-05-03T14:45:50.354 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-05-03T14:45:50.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:50 smithi060 ceph-mon[28873]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-03T14:45:50.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:50 smithi060 ceph-mon[28873]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2024-05-03T14:45:50.820 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:45:50.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:50 smithi090 ceph-mon[33568]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-03T14:45:50.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:50 smithi090 ceph-mon[33568]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2024-05-03T14:45:50.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:50 smithi149 ceph-mon[33332]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-03T14:45:50.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:50 smithi149 ceph-mon[33332]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2024-05-03T14:45:51.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:51 smithi060 ceph-mon[28873]: pgmap v370: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:45:51.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:51 smithi090 ceph-mon[33568]: pgmap v370: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:45:51.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:51 smithi149 ceph-mon[33332]: pgmap v370: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:45:53.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:53.057+0000 7f6af6800700 1 -- 172.21.15.60:0/677389899 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6af010b6c0 msgr2=0x7f6af010baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:53.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:53.057+0000 7f6af6800700 1 --2- 172.21.15.60:0/677389899 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6af010b6c0 0x7f6af010baa0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6ae00098f0 tx=0x7f6ae0009c00 comp rx=0 tx=0).stop 2024-05-03T14:45:53.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:53.058+0000 7f6af6800700 1 -- 172.21.15.60:0/677389899 shutdown_connections 2024-05-03T14:45:53.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:53.058+0000 7f6af6800700 1 --2- 172.21.15.60:0/677389899 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6af0106080 0x7f6af0112590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:53.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:53.058+0000 7f6af6800700 1 --2- 172.21.15.60:0/677389899 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6af010b6c0 0x7f6af010baa0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:53.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:53.058+0000 7f6af6800700 1 --2- 172.21.15.60:0/677389899 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6af01056c0 0x7f6af0105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:53.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:53.058+0000 7f6af6800700 1 -- 172.21.15.60:0/677389899 >> 172.21.15.60:0/677389899 conn(0x7f6af01011b0 msgr2=0x7f6af01035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:45:53.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:53.058+0000 7f6af6800700 1 -- 172.21.15.60:0/677389899 shutdown_connections 2024-05-03T14:45:53.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:53.058+0000 7f6af6800700 1 -- 172.21.15.60:0/677389899 wait complete. 2024-05-03T14:45:53.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:53.059+0000 7f6af6800700 1 Processor -- start 2024-05-03T14:45:53.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:53.059+0000 7f6af6800700 1 -- start start 2024-05-03T14:45:53.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:53.059+0000 7f6af6800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6af01056c0 0x7f6af019bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:45:53.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:53.059+0000 7f6af6800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6af0106080 0x7f6af019c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:45:53.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:53.059+0000 7f6af6800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6af010b6c0 0x7f6af01a0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:45:53.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:53.059+0000 7f6af6800700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f6af0115b20 con 0x7f6af0106080 2024-05-03T14:45:53.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:53.059+0000 7f6af6800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6af01159a0 con 0x7f6af01056c0 2024-05-03T14:45:53.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:53.059+0000 7f6af6800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6af0115ca0 con 0x7f6af010b6c0 2024-05-03T14:45:53.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:53.060+0000 7f6aef400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6af01056c0 0x7f6af019bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:45:53.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:53.060+0000 7f6aeea00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6af0106080 0x7f6af019c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:45:53.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:53.060+0000 7f6aef400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6af01056c0 0x7f6af019bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.60:59610/0 (socket says 172.21.15.60:59610) 2024-05-03T14:45:53.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:53.060+0000 7f6aef400700 1 -- 172.21.15.60:0/2806936173 learned_addr learned my addr 172.21.15.60:0/2806936173 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:45:53.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:53.060+0000 7f6aefe00700 1 --2- 172.21.15.60:0/2806936173 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6af010b6c0 0x7f6af01a0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:45:53.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:53.060+0000 7f6aef400700 1 -- 172.21.15.60:0/2806936173 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6af010b6c0 msgr2=0x7f6af01a0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:53.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:53.060+0000 7f6aef400700 1 --2- 172.21.15.60:0/2806936173 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6af010b6c0 0x7f6af01a0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:53.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:53.060+0000 7f6aef400700 1 -- 172.21.15.60:0/2806936173 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6af0106080 msgr2=0x7f6af019c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:53.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:53.060+0000 7f6aef400700 1 --2- 172.21.15.60:0/2806936173 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6af0106080 0x7f6af019c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:53.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:53.060+0000 7f6aef400700 1 -- 172.21.15.60:0/2806936173 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6af01a0f40 con 0x7f6af01056c0 2024-05-03T14:45:53.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:53.061+0000 7f6aef400700 1 --2- 172.21.15.60:0/2806936173 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6af01056c0 0x7f6af019bf30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0009230 tx=0x7f6ae0010f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:45:53.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:53.061+0000 7f6ae7e00700 1 -- 172.21.15.60:0/2806936173 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0027070 con 0x7f6af01056c0 2024-05-03T14:45:53.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:53.061+0000 7f6af6800700 1 -- 172.21.15.60:0/2806936173 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6af01a11d0 con 0x7f6af01056c0 2024-05-03T14:45:53.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:53.061+0000 7f6ae7e00700 1 -- 172.21.15.60:0/2806936173 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f6ae000bc40 con 0x7f6af01056c0 2024-05-03T14:45:53.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:53.061+0000 7f6ae7e00700 1 -- 172.21.15.60:0/2806936173 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0004a50 con 0x7f6af01056c0 2024-05-03T14:45:53.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:53.061+0000 7f6af6800700 1 -- 172.21.15.60:0/2806936173 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6af01a8ab0 con 0x7f6af01056c0 2024-05-03T14:45:53.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:53.062+0000 7f6af6800700 1 -- 172.21.15.60:0/2806936173 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6af006a190 con 0x7f6af01056c0 2024-05-03T14:45:53.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:53.065+0000 7f6ae7e00700 1 -- 172.21.15.60:0/2806936173 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f6ae002a070 con 0x7f6af01056c0 2024-05-03T14:45:53.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:53.066+0000 7f6ae7e00700 1 --2- 172.21.15.60:0/2806936173 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f6ad40b2900 0x7f6ad40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:45:53.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:53.066+0000 7f6ae7e00700 1 -- 172.21.15.60:0/2806936173 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f6ae00e2fe0 con 0x7f6af01056c0 2024-05-03T14:45:53.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:53.066+0000 7f6aeea00700 1 --2- 172.21.15.60:0/2806936173 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f6ad40b2900 0x7f6ad40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:45:53.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:53.067+0000 7f6aeea00700 1 --2- 172.21.15.60:0/2806936173 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f6ad40b2900 0x7f6ad40b4dc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6af010ac80 tx=0x7f6adc009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:45:53.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:53.068+0000 7f6ae7e00700 1 -- 172.21.15.60:0/2806936173 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f6ae0013030 con 0x7f6af01056c0 2024-05-03T14:45:53.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:53.359+0000 7f6af6800700 1 -- 172.21.15.60:0/2806936173 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f6af019d290 con 0x7f6af01056c0 2024-05-03T14:45:53.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:53.467+0000 7f6ae7e00700 1 -- 172.21.15.60:0/2806936173 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f6ae00aa9f0 con 0x7f6af01056c0 2024-05-03T14:45:53.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:53.471+0000 7f6af6800700 1 -- 172.21.15.60:0/2806936173 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f6ad40b2900 msgr2=0x7f6ad40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:53.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:53.471+0000 7f6af6800700 1 --2- 172.21.15.60:0/2806936173 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f6ad40b2900 0x7f6ad40b4dc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6af010ac80 tx=0x7f6adc009310 comp rx=0 tx=0).stop 2024-05-03T14:45:53.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:53.471+0000 7f6af6800700 1 -- 172.21.15.60:0/2806936173 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6af01056c0 msgr2=0x7f6af019bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:53.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:53.471+0000 7f6af6800700 1 --2- 172.21.15.60:0/2806936173 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6af01056c0 0x7f6af019bf30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0009230 tx=0x7f6ae0010f80 comp rx=0 tx=0).stop 2024-05-03T14:45:53.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:53.471+0000 7f6af6800700 1 -- 172.21.15.60:0/2806936173 shutdown_connections 2024-05-03T14:45:53.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:53.471+0000 7f6af6800700 1 --2- 172.21.15.60:0/2806936173 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f6ad40b2900 0x7f6ad40b4dc0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:53.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:53.471+0000 7f6af6800700 1 --2- 172.21.15.60:0/2806936173 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6af010b6c0 0x7f6af01a0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:53.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:53.471+0000 7f6af6800700 1 --2- 172.21.15.60:0/2806936173 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6af01056c0 0x7f6af019bf30 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:53.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:53.471+0000 7f6af6800700 1 --2- 172.21.15.60:0/2806936173 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6af0106080 0x7f6af019c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:53.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:53.471+0000 7f6af6800700 1 -- 172.21.15.60:0/2806936173 >> 172.21.15.60:0/2806936173 conn(0x7f6af01011b0 msgr2=0x7f6af01016d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:45:53.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:53.472+0000 7f6af6800700 1 -- 172.21.15.60:0/2806936173 shutdown_connections 2024-05-03T14:45:53.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:53.472+0000 7f6af6800700 1 -- 172.21.15.60:0/2806936173 wait complete. 2024-05-03T14:45:53.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:53 smithi060 ceph-mon[28873]: pgmap v371: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:45:53.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:53 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/2806936173' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-03T14:45:53.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:53 smithi060 ceph-mon[28873]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-03T14:45:53.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:53 smithi090 ceph-mon[33568]: pgmap v371: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:45:53.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:53 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/2806936173' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-03T14:45:53.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:53 smithi090 ceph-mon[33568]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-03T14:45:53.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:53 smithi149 ceph-mon[33332]: pgmap v371: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T14:45:53.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:53 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/2806936173' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-03T14:45:53.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:53 smithi149 ceph-mon[33332]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-03T14:45:54.340 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-05-03T14:45:54.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:54 smithi060 ceph-mon[28873]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-03T14:45:54.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:54 smithi060 ceph-mon[28873]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-03T14:45:54.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:54 smithi060 ceph-mon[28873]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-03T14:45:54.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:54 smithi060 ceph-mon[28873]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2024-05-03T14:45:54.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:54 smithi060 ceph-mon[28873]: fsmap cephfs:2 {0=a=up:active,1=b=up:creating} 10 up:standby 2024-05-03T14:45:54.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:54 smithi060 ceph-mon[28873]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-05-03T14:45:54.827 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:45:54.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:54 smithi090 ceph-mon[33568]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-03T14:45:54.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:54 smithi090 ceph-mon[33568]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-03T14:45:54.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:54 smithi090 ceph-mon[33568]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-03T14:45:54.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:54 smithi090 ceph-mon[33568]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2024-05-03T14:45:54.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:54 smithi090 ceph-mon[33568]: fsmap cephfs:2 {0=a=up:active,1=b=up:creating} 10 up:standby 2024-05-03T14:45:54.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:54 smithi090 ceph-mon[33568]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-05-03T14:45:54.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:54 smithi149 ceph-mon[33332]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-03T14:45:54.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:54 smithi149 ceph-mon[33332]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-03T14:45:54.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:54 smithi149 ceph-mon[33332]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-03T14:45:54.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:54 smithi149 ceph-mon[33332]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2024-05-03T14:45:54.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:54 smithi149 ceph-mon[33332]: fsmap cephfs:2 {0=a=up:active,1=b=up:creating} 10 up:standby 2024-05-03T14:45:54.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:54 smithi149 ceph-mon[33332]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-05-03T14:45:55.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:55 smithi060 ceph-mon[28873]: pgmap v372: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-05-03T14:45:55.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:55 smithi060 ceph-mon[28873]: mds.? [v2:172.21.15.90:6838/3512381172,v1:172.21.15.90:6839/3512381172] up:active 2024-05-03T14:45:55.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:55 smithi060 ceph-mon[28873]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-03T14:45:55.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:55 smithi060 ceph-mon[28873]: fsmap cephfs:2 {0=a=up:active,1=b=up:active} 10 up:standby 2024-05-03T14:45:55.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:55 smithi060 ceph-mon[28873]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:creating} 9 up:standby 2024-05-03T14:45:55.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:55 smithi060 ceph-mon[28873]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-05-03T14:45:55.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:55 smithi090 ceph-mon[33568]: pgmap v372: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-05-03T14:45:55.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:55 smithi090 ceph-mon[33568]: mds.? [v2:172.21.15.90:6838/3512381172,v1:172.21.15.90:6839/3512381172] up:active 2024-05-03T14:45:55.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:55 smithi090 ceph-mon[33568]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-03T14:45:55.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:55 smithi090 ceph-mon[33568]: fsmap cephfs:2 {0=a=up:active,1=b=up:active} 10 up:standby 2024-05-03T14:45:55.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:55 smithi090 ceph-mon[33568]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:creating} 9 up:standby 2024-05-03T14:45:55.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:55 smithi090 ceph-mon[33568]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-05-03T14:45:55.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:55 smithi149 ceph-mon[33332]: pgmap v372: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-05-03T14:45:55.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:55 smithi149 ceph-mon[33332]: mds.? [v2:172.21.15.90:6838/3512381172,v1:172.21.15.90:6839/3512381172] up:active 2024-05-03T14:45:55.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:55 smithi149 ceph-mon[33332]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-03T14:45:55.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:55 smithi149 ceph-mon[33332]: fsmap cephfs:2 {0=a=up:active,1=b=up:active} 10 up:standby 2024-05-03T14:45:55.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:55 smithi149 ceph-mon[33332]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:creating} 9 up:standby 2024-05-03T14:45:55.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:55 smithi149 ceph-mon[33332]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-05-03T14:45:56.777 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:56 smithi090 ceph-mon[33568]: mds.? [v2:172.21.15.149:6838/2527622212,v1:172.21.15.149:6839/2527622212] up:active 2024-05-03T14:45:56.777 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:56 smithi090 ceph-mon[33568]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-03T14:45:56.777 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:56 smithi090 ceph-mon[33568]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 9 up:standby 2024-05-03T14:45:56.777 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:56 smithi090 ceph-mon[33568]: fsmap cephfs:4 {0=a=up:active,1=b=up:active,2=i=up:active,3=c=up:creating} 8 up:standby 2024-05-03T14:45:56.777 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:56 smithi090 ceph-mon[33568]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-05-03T14:45:56.827 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:56 smithi060 ceph-mon[28873]: mds.? [v2:172.21.15.149:6838/2527622212,v1:172.21.15.149:6839/2527622212] up:active 2024-05-03T14:45:56.827 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:56 smithi060 ceph-mon[28873]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-03T14:45:56.827 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:56 smithi060 ceph-mon[28873]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 9 up:standby 2024-05-03T14:45:56.827 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:56 smithi060 ceph-mon[28873]: fsmap cephfs:4 {0=a=up:active,1=b=up:active,2=i=up:active,3=c=up:creating} 8 up:standby 2024-05-03T14:45:56.827 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:56 smithi060 ceph-mon[28873]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-05-03T14:45:56.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:56 smithi149 ceph-mon[33332]: mds.? [v2:172.21.15.149:6838/2527622212,v1:172.21.15.149:6839/2527622212] up:active 2024-05-03T14:45:56.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:56 smithi149 ceph-mon[33332]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-03T14:45:56.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:56 smithi149 ceph-mon[33332]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 9 up:standby 2024-05-03T14:45:56.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:56 smithi149 ceph-mon[33332]: fsmap cephfs:4 {0=a=up:active,1=b=up:active,2=i=up:active,3=c=up:creating} 8 up:standby 2024-05-03T14:45:56.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:56 smithi149 ceph-mon[33332]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-05-03T14:45:56.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:56.989+0000 7f0dcd200700 1 -- 172.21.15.60:0/210450004 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0dc8075ff0 msgr2=0x7f0dc810f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:56.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:56.989+0000 7f0dcd200700 1 --2- 172.21.15.60:0/210450004 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0dc8075ff0 0x7f0dc810f960 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0db4009990 tx=0x7f0db4009ca0 comp rx=0 tx=0).stop 2024-05-03T14:45:56.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:56.990+0000 7f0dcd200700 1 -- 172.21.15.60:0/210450004 shutdown_connections 2024-05-03T14:45:56.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:56.990+0000 7f0dcd200700 1 --2- 172.21.15.60:0/210450004 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0dc810fea0 0x7f0dc8112290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:56.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:56.990+0000 7f0dcd200700 1 --2- 172.21.15.60:0/210450004 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0dc80756d0 0x7f0dc8075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:56.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:56.990+0000 7f0dcd200700 1 --2- 172.21.15.60:0/210450004 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0dc8075ff0 0x7f0dc810f960 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:56.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:56.990+0000 7f0dcd200700 1 -- 172.21.15.60:0/210450004 >> 172.21.15.60:0/210450004 conn(0x7f0dc807a600 msgr2=0x7f0dc807aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:45:56.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:56.990+0000 7f0dcd200700 1 -- 172.21.15.60:0/210450004 shutdown_connections 2024-05-03T14:45:56.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:56.990+0000 7f0dcd200700 1 -- 172.21.15.60:0/210450004 wait complete. 2024-05-03T14:45:56.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:56.991+0000 7f0dcd200700 1 Processor -- start 2024-05-03T14:45:56.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:56.991+0000 7f0dcd200700 1 -- start start 2024-05-03T14:45:56.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:56.992+0000 7f0dcd200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0dc80756d0 0x7f0dc819bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:45:56.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:56.992+0000 7f0dcd200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0dc8075ff0 0x7f0dc819c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:45:56.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:56.992+0000 7f0dcd200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0dc810fea0 0x7f0dc81a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:45:56.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:56.992+0000 7f0dcd200700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc8115b70 con 0x7f0dc810fea0 2024-05-03T14:45:56.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:56.992+0000 7f0dcd200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc81159f0 con 0x7f0dc80756d0 2024-05-03T14:45:56.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:56.992+0000 7f0dcd200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc8115cf0 con 0x7f0dc8075ff0 2024-05-03T14:45:56.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:56.992+0000 7f0dc6800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0dc810fea0 0x7f0dc81a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:45:56.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:56.992+0000 7f0dc6800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0dc810fea0 0x7f0dc81a07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50658/0 (socket says 172.21.15.60:50658) 2024-05-03T14:45:56.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:56.993+0000 7f0dc6800700 1 -- 172.21.15.60:0/693123346 learned_addr learned my addr 172.21.15.60:0/693123346 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:45:56.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:56.993+0000 7f0dc5e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0dc80756d0 0x7f0dc819bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:45:56.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:56.993+0000 7f0dc5400700 1 --2- 172.21.15.60:0/693123346 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0dc8075ff0 0x7f0dc819c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:45:56.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:56.993+0000 7f0dc5e00700 1 -- 172.21.15.60:0/693123346 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0dc8075ff0 msgr2=0x7f0dc819c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:56.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:56.993+0000 7f0dc5e00700 1 --2- 172.21.15.60:0/693123346 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0dc8075ff0 0x7f0dc819c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:56.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:56.993+0000 7f0dc5e00700 1 -- 172.21.15.60:0/693123346 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0dc810fea0 msgr2=0x7f0dc81a07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:56.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:56.993+0000 7f0dc5e00700 1 --2- 172.21.15.60:0/693123346 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0dc810fea0 0x7f0dc81a07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:56.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:56.993+0000 7f0dc5e00700 1 -- 172.21.15.60:0/693123346 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dc81a0f00 con 0x7f0dc80756d0 2024-05-03T14:45:56.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:56.993+0000 7f0dc6800700 1 --2- 172.21.15.60:0/693123346 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0dc810fea0 0x7f0dc81a07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:45:56.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:56.993+0000 7f0dc5e00700 1 --2- 172.21.15.60:0/693123346 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0dc80756d0 0x7f0dc819bf40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0db800e9f0 tx=0x7f0db800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:45:56.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:56.994+0000 7f0dbea00700 1 -- 172.21.15.60:0/693123346 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db800c9c0 con 0x7f0dc80756d0 2024-05-03T14:45:56.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:56.994+0000 7f0dbea00700 1 -- 172.21.15.60:0/693123346 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f0db800cb60 con 0x7f0dc80756d0 2024-05-03T14:45:56.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:56.994+0000 7f0dbea00700 1 -- 172.21.15.60:0/693123346 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db8018680 con 0x7f0dc80756d0 2024-05-03T14:45:56.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:56.994+0000 7f0dcd200700 1 -- 172.21.15.60:0/693123346 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dc81a11f0 con 0x7f0dc80756d0 2024-05-03T14:45:56.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:56.994+0000 7f0dcd200700 1 -- 172.21.15.60:0/693123346 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dc81a8a70 con 0x7f0dc80756d0 2024-05-03T14:45:56.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:56.995+0000 7f0dcd200700 1 -- 172.21.15.60:0/693123346 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dc806a190 con 0x7f0dc80756d0 2024-05-03T14:45:56.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:56.997+0000 7f0dbea00700 1 -- 172.21.15.60:0/693123346 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f0db800cd10 con 0x7f0dc80756d0 2024-05-03T14:45:57.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:56.999+0000 7f0dbea00700 1 --2- 172.21.15.60:0/693123346 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f0dac0b2880 0x7f0dac0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:45:57.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:56.999+0000 7f0dbea00700 1 -- 172.21.15.60:0/693123346 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f0db80e2ba0 con 0x7f0dc80756d0 2024-05-03T14:45:57.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:56.999+0000 7f0dc5400700 1 --2- 172.21.15.60:0/693123346 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f0dac0b2880 0x7f0dac0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:45:57.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:57.000+0000 7f0dc5400700 1 --2- 172.21.15.60:0/693123346 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f0dac0b2880 0x7f0dac0b4d40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8111b60 tx=0x7f0db4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:45:57.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:57.001+0000 7f0dbea00700 1 -- 172.21.15.60:0/693123346 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f0db80aa500 con 0x7f0dc80756d0 2024-05-03T14:45:57.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:57.291+0000 7f0dcd200700 1 -- 172.21.15.60:0/693123346 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f0dc81a8c40 con 0x7f0dc80756d0 2024-05-03T14:45:57.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:57.534+0000 7f0dbea00700 1 -- 172.21.15.60:0/693123346 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v27) ==== 121+0+0 (secure 0 0 0) 0x7f0db80a9e00 con 0x7f0dc80756d0 2024-05-03T14:45:57.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:57.538+0000 7f0dcd200700 1 -- 172.21.15.60:0/693123346 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f0dac0b2880 msgr2=0x7f0dac0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:57.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:57.538+0000 7f0dcd200700 1 --2- 172.21.15.60:0/693123346 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f0dac0b2880 0x7f0dac0b4d40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8111b60 tx=0x7f0db4005fd0 comp rx=0 tx=0).stop 2024-05-03T14:45:57.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:57.538+0000 7f0dcd200700 1 -- 172.21.15.60:0/693123346 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0dc80756d0 msgr2=0x7f0dc819bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:45:57.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:57.538+0000 7f0dcd200700 1 --2- 172.21.15.60:0/693123346 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0dc80756d0 0x7f0dc819bf40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0db800e9f0 tx=0x7f0db800ed00 comp rx=0 tx=0).stop 2024-05-03T14:45:57.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:57.538+0000 7f0dcd200700 1 -- 172.21.15.60:0/693123346 shutdown_connections 2024-05-03T14:45:57.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:57.538+0000 7f0dcd200700 1 --2- 172.21.15.60:0/693123346 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f0dac0b2880 0x7f0dac0b4d40 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:57.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:57.538+0000 7f0dcd200700 1 --2- 172.21.15.60:0/693123346 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0dc810fea0 0x7f0dc81a07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:57.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:57.538+0000 7f0dcd200700 1 --2- 172.21.15.60:0/693123346 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0dc80756d0 0x7f0dc819bf40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:57.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:57.538+0000 7f0dcd200700 1 --2- 172.21.15.60:0/693123346 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0dc8075ff0 0x7f0dc819c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:45:57.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:57.538+0000 7f0dcd200700 1 -- 172.21.15.60:0/693123346 >> 172.21.15.60:0/693123346 conn(0x7f0dc807a600 msgr2=0x7f0dc81005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:45:57.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:57.538+0000 7f0dcd200700 1 -- 172.21.15.60:0/693123346 shutdown_connections 2024-05-03T14:45:57.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:45:57.538+0000 7f0dcd200700 1 -- 172.21.15.60:0/693123346 wait complete. 2024-05-03T14:45:57.560 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:57 smithi060 ceph-mon[28873]: pgmap v373: 129 pgs: 129 active+clean; 582 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 3 op/s 2024-05-03T14:45:57.560 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:57 smithi060 ceph-mon[28873]: mds.? [v2:172.21.15.149:6836/2611278690,v1:172.21.15.149:6837/2611278690] up:active 2024-05-03T14:45:57.560 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:57 smithi060 ceph-mon[28873]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-03T14:45:57.560 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:57 smithi060 ceph-mon[28873]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-03T14:45:57.560 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:57 smithi060 ceph-mon[28873]: Cluster is now healthy 2024-05-03T14:45:57.560 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:57 smithi060 ceph-mon[28873]: fsmap cephfs:4 {0=a=up:active,1=b=up:active,2=i=up:active,3=c=up:active} 8 up:standby 2024-05-03T14:45:57.560 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:57 smithi060 ceph-mon[28873]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-05-03T14:45:57.560 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:57 smithi060 ceph-mon[28873]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-05-03T14:45:57.560 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:57 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/693123346' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-03T14:45:57.560 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:57 smithi060 ceph-mon[28873]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-03T14:45:57.560 INFO:journalctl@ceph.mds.a.smithi060.stdout:May 03 14:45:57 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-a[68311]: 2024-05-03T14:45:57.387+0000 7ff15dc00700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-05-03T14:45:57.561 INFO:journalctl@ceph.mds.a.smithi060.stdout:May 03 14:45:57 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-a[68311]: 2024-05-03T14:45:57.387+0000 7ff15dc00700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-05-03T14:45:57.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:57 smithi090 ceph-mon[33568]: pgmap v373: 129 pgs: 129 active+clean; 582 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 3 op/s 2024-05-03T14:45:57.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:57 smithi090 ceph-mon[33568]: mds.? [v2:172.21.15.149:6836/2611278690,v1:172.21.15.149:6837/2611278690] up:active 2024-05-03T14:45:57.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:57 smithi090 ceph-mon[33568]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-03T14:45:57.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:57 smithi090 ceph-mon[33568]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-03T14:45:57.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:57 smithi090 ceph-mon[33568]: Cluster is now healthy 2024-05-03T14:45:57.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:57 smithi090 ceph-mon[33568]: fsmap cephfs:4 {0=a=up:active,1=b=up:active,2=i=up:active,3=c=up:active} 8 up:standby 2024-05-03T14:45:57.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:57 smithi090 ceph-mon[33568]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-05-03T14:45:57.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:57 smithi090 ceph-mon[33568]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-05-03T14:45:57.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:57 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/693123346' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-03T14:45:57.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:57 smithi090 ceph-mon[33568]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-03T14:45:57.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:57 smithi149 ceph-mon[33332]: pgmap v373: 129 pgs: 129 active+clean; 582 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 3 op/s 2024-05-03T14:45:57.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:57 smithi149 ceph-mon[33332]: mds.? [v2:172.21.15.149:6836/2611278690,v1:172.21.15.149:6837/2611278690] up:active 2024-05-03T14:45:57.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:57 smithi149 ceph-mon[33332]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-03T14:45:57.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:57 smithi149 ceph-mon[33332]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-03T14:45:57.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:57 smithi149 ceph-mon[33332]: Cluster is now healthy 2024-05-03T14:45:57.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:57 smithi149 ceph-mon[33332]: fsmap cephfs:4 {0=a=up:active,1=b=up:active,2=i=up:active,3=c=up:active} 8 up:standby 2024-05-03T14:45:57.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:57 smithi149 ceph-mon[33332]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-05-03T14:45:57.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:57 smithi149 ceph-mon[33332]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-05-03T14:45:57.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:57 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/693123346' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-03T14:45:57.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:57 smithi149 ceph-mon[33332]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-03T14:45:58.476 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-05-03T14:45:58.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:58 smithi060 ceph-mon[28873]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-03T14:45:58.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:58 smithi060 ceph-mon[28873]: mds.? [v2:172.21.15.90:6836/2243702844,v1:172.21.15.90:6837/2243702844] up:active 2024-05-03T14:45:58.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:58 smithi060 ceph-mon[28873]: fsmap cephfs:5 7 up:standby 5 up:active 2024-05-03T14:45:58.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:58 smithi060 ceph-mon[28873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-03T14:45:58.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:58 smithi090 ceph-mon[33568]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-03T14:45:58.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:58 smithi090 ceph-mon[33568]: mds.? [v2:172.21.15.90:6836/2243702844,v1:172.21.15.90:6837/2243702844] up:active 2024-05-03T14:45:58.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:58 smithi090 ceph-mon[33568]: fsmap cephfs:5 7 up:standby 5 up:active 2024-05-03T14:45:58.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:58 smithi090 ceph-mon[33568]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-03T14:45:58.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:58 smithi149 ceph-mon[33332]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-03T14:45:58.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:58 smithi149 ceph-mon[33332]: mds.? [v2:172.21.15.90:6836/2243702844,v1:172.21.15.90:6837/2243702844] up:active 2024-05-03T14:45:58.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:58 smithi149 ceph-mon[33332]: fsmap cephfs:5 7 up:standby 5 up:active 2024-05-03T14:45:58.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:58 smithi149 ceph-mon[33332]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-03T14:45:58.924 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:45:59.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:45:59 smithi060 ceph-mon[28873]: pgmap v374: 129 pgs: 129 active+clean; 583 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 7 op/s 2024-05-03T14:45:59.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:45:59 smithi090 ceph-mon[33568]: pgmap v374: 129 pgs: 129 active+clean; 583 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 7 op/s 2024-05-03T14:45:59.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:45:59 smithi149 ceph-mon[33332]: pgmap v374: 129 pgs: 129 active+clean; 583 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 7 op/s 2024-05-03T14:46:01.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:01.062+0000 7f6432600700 1 -- 172.21.15.60:0/2004285840 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f642c10e8b0 msgr2=0x7f642c10ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:01.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:01.062+0000 7f6432600700 1 --2- 172.21.15.60:0/2004285840 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f642c10e8b0 0x7f642c10ec90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f641c009990 tx=0x7f641c009ca0 comp rx=0 tx=0).stop 2024-05-03T14:46:01.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:01.062+0000 7f6432600700 1 -- 172.21.15.60:0/2004285840 shutdown_connections 2024-05-03T14:46:01.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:01.062+0000 7f6432600700 1 --2- 172.21.15.60:0/2004285840 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f642c10a3a0 0x7f642c116840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:01.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:01.062+0000 7f6432600700 1 --2- 172.21.15.60:0/2004285840 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f642c10e8b0 0x7f642c10ec90 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:01.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:01.062+0000 7f6432600700 1 --2- 172.21.15.60:0/2004285840 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f642c1099e0 0x7f642c109e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:01.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:01.062+0000 7f6432600700 1 -- 172.21.15.60:0/2004285840 >> 172.21.15.60:0/2004285840 conn(0x7f642c079230 msgr2=0x7f642c1082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:46:01.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:01.062+0000 7f6432600700 1 -- 172.21.15.60:0/2004285840 shutdown_connections 2024-05-03T14:46:01.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:01.062+0000 7f6432600700 1 -- 172.21.15.60:0/2004285840 wait complete. 2024-05-03T14:46:01.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:01.063+0000 7f6432600700 1 Processor -- start 2024-05-03T14:46:01.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:01.064+0000 7f6432600700 1 -- start start 2024-05-03T14:46:01.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:01.064+0000 7f6432600700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f642c1099e0 0x7f642c1a0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:46:01.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:01.064+0000 7f6432600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f642c10a3a0 0x7f642c1a07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:46:01.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:01.064+0000 7f6432600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f642c10e8b0 0x7f642c1a4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:46:01.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:01.064+0000 7f6432600700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f642c119e60 con 0x7f642c1099e0 2024-05-03T14:46:01.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:01.064+0000 7f6432600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f642c119ce0 con 0x7f642c10a3a0 2024-05-03T14:46:01.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:01.064+0000 7f6432600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f642c119fe0 con 0x7f642c10e8b0 2024-05-03T14:46:01.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:01.065+0000 7f6431200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f642c1099e0 0x7f642c1a0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:46:01.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:01.065+0000 7f6431200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f642c1099e0 0x7f642c1a0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50662/0 (socket says 172.21.15.60:50662) 2024-05-03T14:46:01.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:01.065+0000 7f6431200700 1 -- 172.21.15.60:0/318981839 learned_addr learned my addr 172.21.15.60:0/318981839 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:46:01.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:01.065+0000 7f6423e00700 1 --2- 172.21.15.60:0/318981839 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f642c10a3a0 0x7f642c1a07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:46:01.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:01.065+0000 7f6431c00700 1 --2- 172.21.15.60:0/318981839 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f642c10e8b0 0x7f642c1a4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:46:01.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:01.065+0000 7f6423e00700 1 -- 172.21.15.60:0/318981839 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f642c10e8b0 msgr2=0x7f642c1a4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:01.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:01.065+0000 7f6423e00700 1 --2- 172.21.15.60:0/318981839 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f642c10e8b0 0x7f642c1a4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:01.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:01.065+0000 7f6423e00700 1 -- 172.21.15.60:0/318981839 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f642c1099e0 msgr2=0x7f642c1a0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:01.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:01.065+0000 7f6423e00700 1 --2- 172.21.15.60:0/318981839 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f642c1099e0 0x7f642c1a0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:01.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:01.065+0000 7f6423e00700 1 -- 172.21.15.60:0/318981839 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f642c1a5280 con 0x7f642c10a3a0 2024-05-03T14:46:01.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:01.065+0000 7f6423e00700 1 --2- 172.21.15.60:0/318981839 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f642c10a3a0 0x7f642c1a07b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f641400e9f0 tx=0x7f641400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:46:01.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:01.066+0000 7f642a000700 1 -- 172.21.15.60:0/318981839 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f641400cb70 con 0x7f642c10a3a0 2024-05-03T14:46:01.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:01.066+0000 7f6432600700 1 -- 172.21.15.60:0/318981839 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f642c1a5570 con 0x7f642c10a3a0 2024-05-03T14:46:01.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:01.066+0000 7f642a000700 1 -- 172.21.15.60:0/318981839 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f641400ee90 con 0x7f642c10a3a0 2024-05-03T14:46:01.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:01.066+0000 7f642a000700 1 -- 172.21.15.60:0/318981839 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6414010430 con 0x7f642c10a3a0 2024-05-03T14:46:01.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:01.066+0000 7f6432600700 1 -- 172.21.15.60:0/318981839 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f642c1acdf0 con 0x7f642c10a3a0 2024-05-03T14:46:01.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:01.067+0000 7f6432600700 1 -- 172.21.15.60:0/318981839 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f642c06a190 con 0x7f642c10a3a0 2024-05-03T14:46:01.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:01.069+0000 7f642a000700 1 -- 172.21.15.60:0/318981839 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f6414003ec0 con 0x7f642c10a3a0 2024-05-03T14:46:01.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:01.071+0000 7f642a000700 1 --2- 172.21.15.60:0/318981839 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f64100b2880 0x7f64100b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:46:01.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:01.071+0000 7f642a000700 1 -- 172.21.15.60:0/318981839 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f64140e2790 con 0x7f642c10a3a0 2024-05-03T14:46:01.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:01.071+0000 7f6431200700 1 --2- 172.21.15.60:0/318981839 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f64100b2880 0x7f64100b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:46:01.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:01.072+0000 7f6431200700 1 --2- 172.21.15.60:0/318981839 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f64100b2880 0x7f64100b4d40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f641c006010 tx=0x7f641c00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:46:01.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:01.075+0000 7f642a000700 1 -- 172.21.15.60:0/318981839 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f64140aa0f0 con 0x7f642c10a3a0 2024-05-03T14:46:01.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:01.365+0000 7f6432600700 1 -- 172.21.15.60:0/318981839 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f642c02e510 con 0x7f642c10a3a0 2024-05-03T14:46:01.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:01.555+0000 7f642a000700 1 -- 172.21.15.60:0/318981839 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v29) ==== 115+0+0 (secure 0 0 0) 0x7f64140a99f0 con 0x7f642c10a3a0 2024-05-03T14:46:01.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:01.559+0000 7f6432600700 1 -- 172.21.15.60:0/318981839 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f64100b2880 msgr2=0x7f64100b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:01.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:01.559+0000 7f6432600700 1 --2- 172.21.15.60:0/318981839 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f64100b2880 0x7f64100b4d40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f641c006010 tx=0x7f641c00b260 comp rx=0 tx=0).stop 2024-05-03T14:46:01.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:01.559+0000 7f6432600700 1 -- 172.21.15.60:0/318981839 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f642c10a3a0 msgr2=0x7f642c1a07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:01.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:01.559+0000 7f6432600700 1 --2- 172.21.15.60:0/318981839 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f642c10a3a0 0x7f642c1a07b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f641400e9f0 tx=0x7f641400ed00 comp rx=0 tx=0).stop 2024-05-03T14:46:01.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:01.560+0000 7f6432600700 1 -- 172.21.15.60:0/318981839 shutdown_connections 2024-05-03T14:46:01.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:01.560+0000 7f6432600700 1 --2- 172.21.15.60:0/318981839 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f64100b2880 0x7f64100b4d40 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:01.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:01.560+0000 7f6432600700 1 --2- 172.21.15.60:0/318981839 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f642c10e8b0 0x7f642c1a4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:01.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:01.560+0000 7f6432600700 1 --2- 172.21.15.60:0/318981839 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f642c1099e0 0x7f642c1a0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:01.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:01.560+0000 7f6432600700 1 --2- 172.21.15.60:0/318981839 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f642c10a3a0 0x7f642c1a07b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:01.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:01.560+0000 7f6432600700 1 -- 172.21.15.60:0/318981839 >> 172.21.15.60:0/318981839 conn(0x7f642c079230 msgr2=0x7f642c079770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:46:01.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:01.561+0000 7f6432600700 1 -- 172.21.15.60:0/318981839 shutdown_connections 2024-05-03T14:46:01.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:01.561+0000 7f6432600700 1 -- 172.21.15.60:0/318981839 wait complete. 2024-05-03T14:46:01.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:01 smithi060 ceph-mon[28873]: pgmap v375: 129 pgs: 129 active+clean; 585 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 3.7 KiB/s wr, 15 op/s 2024-05-03T14:46:01.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:01 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/318981839' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-03T14:46:01.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:01 smithi060 ceph-mon[28873]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-03T14:46:01.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:01 smithi090 ceph-mon[33568]: pgmap v375: 129 pgs: 129 active+clean; 585 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 3.7 KiB/s wr, 15 op/s 2024-05-03T14:46:01.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:01 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/318981839' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-03T14:46:01.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:01 smithi090 ceph-mon[33568]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-03T14:46:01.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:01 smithi149 ceph-mon[33332]: pgmap v375: 129 pgs: 129 active+clean; 585 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 3.7 KiB/s wr, 15 op/s 2024-05-03T14:46:01.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:01 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/318981839' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-03T14:46:01.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:01 smithi149 ceph-mon[33332]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-03T14:46:02.419 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-05-03T14:46:02.420 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-05-03T14:46:02.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:02 smithi060 ceph-mon[28873]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-03T14:46:02.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:02 smithi060 ceph-mon[28873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-03T14:46:02.862 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:46:02.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:02 smithi090 ceph-mon[33568]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-03T14:46:02.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:02 smithi090 ceph-mon[33568]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-03T14:46:02.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:02 smithi149 ceph-mon[33332]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-03T14:46:02.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:02 smithi149 ceph-mon[33332]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-03T14:46:03.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:03 smithi060 ceph-mon[28873]: pgmap v376: 129 pgs: 129 active+clean; 587 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 4.0 KiB/s wr, 17 op/s 2024-05-03T14:46:03.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:03 smithi090 ceph-mon[33568]: pgmap v376: 129 pgs: 129 active+clean; 587 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 4.0 KiB/s wr, 17 op/s 2024-05-03T14:46:03.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:03 smithi149 ceph-mon[33332]: pgmap v376: 129 pgs: 129 active+clean; 587 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 4.0 KiB/s wr, 17 op/s 2024-05-03T14:46:04.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:04 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]: dispatch 2024-05-03T14:46:04.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:04 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]: dispatch 2024-05-03T14:46:04.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:04 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]: dispatch 2024-05-03T14:46:05.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:05.157+0000 7f99fac00700 1 -- 172.21.15.60:0/938867124 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99f4109510 msgr2=0x7f99f41098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:05.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:05.157+0000 7f99fac00700 1 --2- 172.21.15.60:0/938867124 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99f4109510 0x7f99f41098f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f99e40098f0 tx=0x7f99e4009c00 comp rx=0 tx=0).stop 2024-05-03T14:46:05.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:05.157+0000 7f99fac00700 1 -- 172.21.15.60:0/938867124 shutdown_connections 2024-05-03T14:46:05.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:05.157+0000 7f99fac00700 1 --2- 172.21.15.60:0/938867124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99f406bb40 0x7f99f4110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:05.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:05.157+0000 7f99fac00700 1 --2- 172.21.15.60:0/938867124 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99f4109510 0x7f99f41098f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:05.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:05.157+0000 7f99fac00700 1 --2- 172.21.15.60:0/938867124 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99f406b180 0x7f99f406b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:05.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:05.157+0000 7f99fac00700 1 -- 172.21.15.60:0/938867124 >> 172.21.15.60:0/938867124 conn(0x7f99f4079350 msgr2=0x7f99f4079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:46:05.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:05.158+0000 7f99fac00700 1 -- 172.21.15.60:0/938867124 shutdown_connections 2024-05-03T14:46:05.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:05.158+0000 7f99fac00700 1 -- 172.21.15.60:0/938867124 wait complete. 2024-05-03T14:46:05.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:05.159+0000 7f99fac00700 1 Processor -- start 2024-05-03T14:46:05.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:05.159+0000 7f99fac00700 1 -- start start 2024-05-03T14:46:05.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:05.160+0000 7f99fac00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99f406b180 0x7f99f419bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:46:05.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:05.160+0000 7f99fac00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99f406bb40 0x7f99f419c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:46:05.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:05.160+0000 7f99fac00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99f4109510 0x7f99f41a0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:46:05.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:05.160+0000 7f99fac00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f99f4113900 con 0x7f99f406bb40 2024-05-03T14:46:05.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:05.160+0000 7f99fac00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f99f4113780 con 0x7f99f4109510 2024-05-03T14:46:05.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:05.160+0000 7f99fac00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f99f4113a80 con 0x7f99f406b180 2024-05-03T14:46:05.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:05.160+0000 7f99f3e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99f406b180 0x7f99f419bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:46:05.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:05.160+0000 7f99eb400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99f406bb40 0x7f99f419c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:46:05.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:05.160+0000 7f99f3e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99f406b180 0x7f99f419bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.60:35860/0 (socket says 172.21.15.60:35860) 2024-05-03T14:46:05.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:05.160+0000 7f99f3e00700 1 -- 172.21.15.60:0/2517475739 learned_addr learned my addr 172.21.15.60:0/2517475739 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:46:05.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:05.160+0000 7f99f8c00700 1 --2- 172.21.15.60:0/2517475739 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99f4109510 0x7f99f41a0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:46:05.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:05.161+0000 7f99eb400700 1 -- 172.21.15.60:0/2517475739 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99f406b180 msgr2=0x7f99f419bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:05.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:05.161+0000 7f99eb400700 1 --2- 172.21.15.60:0/2517475739 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99f406b180 0x7f99f419bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:05.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:05.161+0000 7f99eb400700 1 -- 172.21.15.60:0/2517475739 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99f4109510 msgr2=0x7f99f41a0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:05.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:05.161+0000 7f99eb400700 1 --2- 172.21.15.60:0/2517475739 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99f4109510 0x7f99f41a0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:05.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:05.161+0000 7f99eb400700 1 -- 172.21.15.60:0/2517475739 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99f41a0fa0 con 0x7f99f406bb40 2024-05-03T14:46:05.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:05.161+0000 7f99f3e00700 1 --2- 172.21.15.60:0/2517475739 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99f406b180 0x7f99f419bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:46:05.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:05.161+0000 7f99eb400700 1 --2- 172.21.15.60:0/2517475739 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99f406bb40 0x7f99f419c4d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f99dc00e9f0 tx=0x7f99dc00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:46:05.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:05.161+0000 7f99f1600700 1 -- 172.21.15.60:0/2517475739 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99dc00c9c0 con 0x7f99f406bb40 2024-05-03T14:46:05.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:05.161+0000 7f99fac00700 1 -- 172.21.15.60:0/2517475739 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99f41a1290 con 0x7f99f406bb40 2024-05-03T14:46:05.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:05.161+0000 7f99f1600700 1 -- 172.21.15.60:0/2517475739 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f99dc00cb60 con 0x7f99f406bb40 2024-05-03T14:46:05.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:05.161+0000 7f99f1600700 1 -- 172.21.15.60:0/2517475739 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99dc018660 con 0x7f99f406bb40 2024-05-03T14:46:05.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:05.161+0000 7f99fac00700 1 -- 172.21.15.60:0/2517475739 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99f41a8b10 con 0x7f99f406bb40 2024-05-03T14:46:05.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:05.163+0000 7f99fac00700 1 -- 172.21.15.60:0/2517475739 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99f406a190 con 0x7f99f406bb40 2024-05-03T14:46:05.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:05.165+0000 7f99f1600700 1 -- 172.21.15.60:0/2517475739 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f99dc00cd10 con 0x7f99f406bb40 2024-05-03T14:46:05.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:05.167+0000 7f99f1600700 1 --2- 172.21.15.60:0/2517475739 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f99d80b2880 0x7f99d80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:46:05.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:05.168+0000 7f99f1600700 1 -- 172.21.15.60:0/2517475739 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9121+0+0 (secure 0 0 0) 0x7f99dc0e2720 con 0x7f99f406bb40 2024-05-03T14:46:05.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:05.168+0000 7f99f3e00700 1 --2- 172.21.15.60:0/2517475739 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f99d80b2880 0x7f99d80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:46:05.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:05.169+0000 7f99f3e00700 1 --2- 172.21.15.60:0/2517475739 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f99d80b2880 0x7f99d80b4d40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f99e4009810 tx=0x7f99e4019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:46:05.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:05.172+0000 7f99f1600700 1 -- 172.21.15.60:0/2517475739 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f99dc0aa060 con 0x7f99f406bb40 2024-05-03T14:46:05.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:05.449+0000 7f99fac00700 1 -- 172.21.15.60:0/2517475739 --> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f99f410ed90 con 0x7f99d80b2880 2024-05-03T14:46:05.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:05.483+0000 7f99f1600700 1 -- 172.21.15.60:0/2517475739 <== mgr.14152 v2:172.21.15.60:6800/3066142956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f99f410ed90 con 0x7f99d80b2880 2024-05-03T14:46:05.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:05.486+0000 7f99fac00700 1 -- 172.21.15.60:0/2517475739 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f99d80b2880 msgr2=0x7f99d80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:05.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:05.486+0000 7f99fac00700 1 --2- 172.21.15.60:0/2517475739 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f99d80b2880 0x7f99d80b4d40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f99e4009810 tx=0x7f99e4019040 comp rx=0 tx=0).stop 2024-05-03T14:46:05.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:05.486+0000 7f99fac00700 1 -- 172.21.15.60:0/2517475739 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99f406bb40 msgr2=0x7f99f419c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:05.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:05.486+0000 7f99fac00700 1 --2- 172.21.15.60:0/2517475739 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99f406bb40 0x7f99f419c4d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f99dc00e9f0 tx=0x7f99dc00ed00 comp rx=0 tx=0).stop 2024-05-03T14:46:05.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:05.487+0000 7f99fac00700 1 -- 172.21.15.60:0/2517475739 shutdown_connections 2024-05-03T14:46:05.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:05.487+0000 7f99fac00700 1 --2- 172.21.15.60:0/2517475739 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f99d80b2880 0x7f99d80b4d40 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:05.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:05.487+0000 7f99fac00700 1 --2- 172.21.15.60:0/2517475739 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99f4109510 0x7f99f41a0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:05.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:05.487+0000 7f99fac00700 1 --2- 172.21.15.60:0/2517475739 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99f406b180 0x7f99f419bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:05.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:05.487+0000 7f99fac00700 1 --2- 172.21.15.60:0/2517475739 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99f406bb40 0x7f99f419c4d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:05.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:05.487+0000 7f99fac00700 1 -- 172.21.15.60:0/2517475739 >> 172.21.15.60:0/2517475739 conn(0x7f99f4079350 msgr2=0x7f99f410e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:46:05.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:05.487+0000 7f99fac00700 1 -- 172.21.15.60:0/2517475739 shutdown_connections 2024-05-03T14:46:05.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:05.487+0000 7f99fac00700 1 -- 172.21.15.60:0/2517475739 wait complete. 2024-05-03T14:46:05.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:05 smithi060 ceph-mon[28873]: pgmap v377: 129 pgs: 129 active+clean; 587 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-05-03T14:46:05.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:05 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]': finished 2024-05-03T14:46:05.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:05 smithi060 ceph-mon[28873]: osdmap e88: 12 total, 12 up, 12 in 2024-05-03T14:46:05.838 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:46:05 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:46:05.459+0000 7fd414800700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-03T14:46:05.838 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:46:05 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:46:05.459+0000 7fd414800700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-03T14:46:05.838 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:46:05 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:46:05.459+0000 7fd414800700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-03T14:46:05.838 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:46:05 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:46:05.459+0000 7fd414800700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-03T14:46:05.838 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 14:46:05 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T14:46:05.459+0000 7fd414800700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-03T14:46:05.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:05 smithi090 ceph-mon[33568]: pgmap v377: 129 pgs: 129 active+clean; 587 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-05-03T14:46:05.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:05 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]': finished 2024-05-03T14:46:05.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:05 smithi090 ceph-mon[33568]: osdmap e88: 12 total, 12 up, 12 in 2024-05-03T14:46:05.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:05 smithi149 ceph-mon[33332]: pgmap v377: 129 pgs: 129 active+clean; 587 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-05-03T14:46:05.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:05 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]': finished 2024-05-03T14:46:05.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:05 smithi149 ceph-mon[33332]: osdmap e88: 12 total, 12 up, 12 in 2024-05-03T14:46:06.260 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-05-03T14:46:06.710 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:46:06.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:06 smithi090 ceph-mon[33568]: from='client.14577 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:46:06.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:06 smithi090 ceph-mon[33568]: osdmap e89: 12 total, 12 up, 12 in 2024-05-03T14:46:06.896 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:06 smithi060 ceph-mon[28873]: from='client.14577 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:46:06.896 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:06 smithi060 ceph-mon[28873]: osdmap e89: 12 total, 12 up, 12 in 2024-05-03T14:46:06.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:06 smithi149 ceph-mon[33332]: from='client.14577 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:46:06.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:06 smithi149 ceph-mon[33332]: osdmap e89: 12 total, 12 up, 12 in 2024-05-03T14:46:07.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:07 smithi090 ceph-mon[33568]: pgmap v380: 129 pgs: 129 active+clean; 587 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 3.2 KiB/s wr, 21 op/s 2024-05-03T14:46:07.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:07 smithi090 ceph-mon[33568]: mgrmap e17: x(active, since 10m), standbys: z, y 2024-05-03T14:46:07.906 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:07 smithi060 ceph-mon[28873]: pgmap v380: 129 pgs: 129 active+clean; 587 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 3.2 KiB/s wr, 21 op/s 2024-05-03T14:46:07.906 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:07 smithi060 ceph-mon[28873]: mgrmap e17: x(active, since 10m), standbys: z, y 2024-05-03T14:46:07.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:07 smithi149 ceph-mon[33332]: pgmap v380: 129 pgs: 129 active+clean; 587 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 3.2 KiB/s wr, 21 op/s 2024-05-03T14:46:07.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:07 smithi149 ceph-mon[33332]: mgrmap e17: x(active, since 10m), standbys: z, y 2024-05-03T14:46:08.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:08.894+0000 7fdf6a200700 1 -- 172.21.15.60:0/3449891697 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf6410a490 msgr2=0x7fdf641169a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:08.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:08.894+0000 7fdf6a200700 1 --2- 172.21.15.60:0/3449891697 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf6410a490 0x7fdf641169a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c00b0a0 tx=0x7fdf5c00b3b0 comp rx=0 tx=0).stop 2024-05-03T14:46:08.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:08.895+0000 7fdf6a200700 1 -- 172.21.15.60:0/3449891697 shutdown_connections 2024-05-03T14:46:08.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:08.895+0000 7fdf6a200700 1 --2- 172.21.15.60:0/3449891697 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf6410a490 0x7fdf641169a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:08.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:08.895+0000 7fdf6a200700 1 --2- 172.21.15.60:0/3449891697 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf6410fb60 0x7fdf6410ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:08.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:08.895+0000 7fdf6a200700 1 --2- 172.21.15.60:0/3449891697 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf64109ad0 0x7fdf64109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:08.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:08.895+0000 7fdf6a200700 1 -- 172.21.15.60:0/3449891697 >> 172.21.15.60:0/3449891697 conn(0x7fdf64076c70 msgr2=0x7fdf641082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:46:08.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:08.895+0000 7fdf6a200700 1 -- 172.21.15.60:0/3449891697 shutdown_connections 2024-05-03T14:46:08.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:08.895+0000 7fdf6a200700 1 -- 172.21.15.60:0/3449891697 wait complete. 2024-05-03T14:46:08.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:08.896+0000 7fdf6a200700 1 Processor -- start 2024-05-03T14:46:08.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:08.896+0000 7fdf6a200700 1 -- start start 2024-05-03T14:46:08.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:08.896+0000 7fdf6a200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf64109ad0 0x7fdf641a0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:46:08.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:08.896+0000 7fdf6a200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf6410a490 0x7fdf641a0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:46:08.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:08.897+0000 7fdf6a200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf6410fb60 0x7fdf641a4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:46:08.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:08.897+0000 7fdf6a200700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fdf64119f70 con 0x7fdf64109ad0 2024-05-03T14:46:08.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:08.897+0000 7fdf6a200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdf64119df0 con 0x7fdf6410a490 2024-05-03T14:46:08.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:08.897+0000 7fdf6a200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6411a0f0 con 0x7fdf6410fb60 2024-05-03T14:46:08.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:08.897+0000 7fdf63400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf64109ad0 0x7fdf641a0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:46:08.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:08.897+0000 7fdf63e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf6410fb60 0x7fdf641a4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:46:08.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:08.897+0000 7fdf63400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf64109ad0 0x7fdf641a0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44930/0 (socket says 172.21.15.60:44930) 2024-05-03T14:46:08.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:08.897+0000 7fdf63e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf6410fb60 0x7fdf641a4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.60:35894/0 (socket says 172.21.15.60:35894) 2024-05-03T14:46:08.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:08.897+0000 7fdf63400700 1 -- 172.21.15.60:0/2897849832 learned_addr learned my addr 172.21.15.60:0/2897849832 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:46:08.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:08.897+0000 7fdf62a00700 1 --2- 172.21.15.60:0/2897849832 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf6410a490 0x7fdf641a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:46:08.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:08.897+0000 7fdf63e00700 1 -- 172.21.15.60:0/2897849832 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf6410a490 msgr2=0x7fdf641a0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:08.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:08.897+0000 7fdf63e00700 1 --2- 172.21.15.60:0/2897849832 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf6410a490 0x7fdf641a0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:08.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:08.897+0000 7fdf63e00700 1 -- 172.21.15.60:0/2897849832 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf64109ad0 msgr2=0x7fdf641a0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:08.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:08.897+0000 7fdf63e00700 1 --2- 172.21.15.60:0/2897849832 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf64109ad0 0x7fdf641a0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:08.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:08.897+0000 7fdf63e00700 1 -- 172.21.15.60:0/2897849832 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf641a5310 con 0x7fdf6410fb60 2024-05-03T14:46:08.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:08.898+0000 7fdf62a00700 1 --2- 172.21.15.60:0/2897849832 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf6410a490 0x7fdf641a0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:46:08.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:08.898+0000 7fdf63e00700 1 --2- 172.21.15.60:0/2897849832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf6410fb60 0x7fdf641a4bd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c00b070 tx=0x7fdf5c004c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:46:08.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:08.898+0000 7fdf5be00700 1 -- 172.21.15.60:0/2897849832 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c021070 con 0x7fdf6410fb60 2024-05-03T14:46:08.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:08.898+0000 7fdf6a200700 1 -- 172.21.15.60:0/2897849832 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf641a55a0 con 0x7fdf6410fb60 2024-05-03T14:46:08.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:08.898+0000 7fdf5be00700 1 -- 172.21.15.60:0/2897849832 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7fdf5c007b80 con 0x7fdf6410fb60 2024-05-03T14:46:08.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:08.898+0000 7fdf5be00700 1 -- 172.21.15.60:0/2897849832 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c0052a0 con 0x7fdf6410fb60 2024-05-03T14:46:08.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:08.899+0000 7fdf6a200700 1 -- 172.21.15.60:0/2897849832 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf641ace80 con 0x7fdf6410fb60 2024-05-03T14:46:08.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:08.900+0000 7fdf6a200700 1 -- 172.21.15.60:0/2897849832 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf6406a190 con 0x7fdf6410fb60 2024-05-03T14:46:08.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:08.903+0000 7fdf5be00700 1 -- 172.21.15.60:0/2897849832 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fdf5c024070 con 0x7fdf6410fb60 2024-05-03T14:46:08.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:08.904+0000 7fdf5be00700 1 --2- 172.21.15.60:0/2897849832 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fdf480b2910 0x7fdf480b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:46:08.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:08.904+0000 7fdf5be00700 1 -- 172.21.15.60:0/2897849832 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7fdf5c00f020 con 0x7fdf6410fb60 2024-05-03T14:46:08.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:08.905+0000 7fdf63400700 1 --2- 172.21.15.60:0/2897849832 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fdf480b2910 0x7fdf480b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:46:08.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:08.905+0000 7fdf63400700 1 --2- 172.21.15.60:0/2897849832 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fdf480b2910 0x7fdf480b4dd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdf54009680 tx=0x7fdf54009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:46:08.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:08.906+0000 7fdf5be00700 1 -- 172.21.15.60:0/2897849832 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fdf5c02b020 con 0x7fdf6410fb60 2024-05-03T14:46:09.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:09.180+0000 7fdf6a200700 1 -- 172.21.15.60:0/2897849832 --> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fdf64077800 con 0x7fdf480b2910 2024-05-03T14:46:09.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:09.230+0000 7fdf5be00700 1 -- 172.21.15.60:0/2897849832 <== mgr.14152 v2:172.21.15.60:6800/3066142956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fdf64077800 con 0x7fdf480b2910 2024-05-03T14:46:09.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:09.234+0000 7fdf6a200700 1 -- 172.21.15.60:0/2897849832 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fdf480b2910 msgr2=0x7fdf480b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:09.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:09.234+0000 7fdf6a200700 1 --2- 172.21.15.60:0/2897849832 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fdf480b2910 0x7fdf480b4dd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdf54009680 tx=0x7fdf54009260 comp rx=0 tx=0).stop 2024-05-03T14:46:09.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:09.234+0000 7fdf6a200700 1 -- 172.21.15.60:0/2897849832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf6410fb60 msgr2=0x7fdf641a4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:09.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:09.234+0000 7fdf6a200700 1 --2- 172.21.15.60:0/2897849832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf6410fb60 0x7fdf641a4bd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c00b070 tx=0x7fdf5c004c80 comp rx=0 tx=0).stop 2024-05-03T14:46:09.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:09.234+0000 7fdf6a200700 1 -- 172.21.15.60:0/2897849832 shutdown_connections 2024-05-03T14:46:09.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:09.234+0000 7fdf6a200700 1 --2- 172.21.15.60:0/2897849832 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fdf480b2910 0x7fdf480b4dd0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:09.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:09.234+0000 7fdf6a200700 1 --2- 172.21.15.60:0/2897849832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf6410fb60 0x7fdf641a4bd0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:09.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:09.234+0000 7fdf6a200700 1 --2- 172.21.15.60:0/2897849832 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf64109ad0 0x7fdf641a0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:09.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:09.234+0000 7fdf6a200700 1 --2- 172.21.15.60:0/2897849832 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf6410a490 0x7fdf641a0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:09.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:09.234+0000 7fdf6a200700 1 -- 172.21.15.60:0/2897849832 >> 172.21.15.60:0/2897849832 conn(0x7fdf64076c70 msgr2=0x7fdf640770f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:46:09.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:09.234+0000 7fdf6a200700 1 -- 172.21.15.60:0/2897849832 shutdown_connections 2024-05-03T14:46:09.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:09.234+0000 7fdf6a200700 1 -- 172.21.15.60:0/2897849832 wait complete. 2024-05-03T14:46:09.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:09 smithi090 ceph-mon[33568]: pgmap v381: 129 pgs: 129 active+clean; 587 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 10 op/s; 83 B/s, 0 objects/s recovering 2024-05-03T14:46:09.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:09 smithi149 ceph-mon[33332]: pgmap v381: 129 pgs: 129 active+clean; 587 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 10 op/s; 83 B/s, 0 objects/s recovering 2024-05-03T14:46:09.960 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:09 smithi060 ceph-mon[28873]: pgmap v381: 129 pgs: 129 active+clean; 587 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 10 op/s; 83 B/s, 0 objects/s recovering 2024-05-03T14:46:10.036 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-05-03T14:46:10.515 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:46:10.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:10 smithi090 ceph-mon[33568]: from='client.24697 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:46:10.888 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:10 smithi060 ceph-mon[28873]: from='client.24697 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:46:10.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:10 smithi149 ceph-mon[33332]: from='client.24697 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:46:11.892 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:11 smithi060 ceph-mon[28873]: pgmap v382: 129 pgs: 129 active+clean; 607 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.1 KiB/s wr, 13 op/s; 83 B/s, 0 objects/s recovering 2024-05-03T14:46:11.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:11 smithi149 ceph-mon[33332]: pgmap v382: 129 pgs: 129 active+clean; 607 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.1 KiB/s wr, 13 op/s; 83 B/s, 0 objects/s recovering 2024-05-03T14:46:12.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:11 smithi090 ceph-mon[33568]: pgmap v382: 129 pgs: 129 active+clean; 607 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.1 KiB/s wr, 13 op/s; 83 B/s, 0 objects/s recovering 2024-05-03T14:46:12.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:12.852+0000 7f8e0f200700 1 -- 172.21.15.60:0/1705298542 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8e080779d0 msgr2=0x7f8e08077db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:12.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:12.852+0000 7f8e0f200700 1 --2- 172.21.15.60:0/1705298542 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8e080779d0 0x7f8e08077db0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8df800b0a0 tx=0x7f8df800b3b0 comp rx=0 tx=0).stop 2024-05-03T14:46:12.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:12.852+0000 7f8e0f200700 1 -- 172.21.15.60:0/1705298542 shutdown_connections 2024-05-03T14:46:12.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:12.852+0000 7f8e0f200700 1 --2- 172.21.15.60:0/1705298542 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e08114330 0x7f8e08116720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:12.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:12.852+0000 7f8e0f200700 1 --2- 172.21.15.60:0/1705298542 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8e080779d0 0x7f8e08077db0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:12.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:12.852+0000 7f8e0f200700 1 --2- 172.21.15.60:0/1705298542 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8e080782f0 0x7f8e08113df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:12.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:12.852+0000 7f8e0f200700 1 -- 172.21.15.60:0/1705298542 >> 172.21.15.60:0/1705298542 conn(0x7f8e08101180 msgr2=0x7f8e081035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:46:12.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:12.852+0000 7f8e0f200700 1 -- 172.21.15.60:0/1705298542 shutdown_connections 2024-05-03T14:46:12.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:12.853+0000 7f8e0f200700 1 -- 172.21.15.60:0/1705298542 wait complete. 2024-05-03T14:46:12.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:12.854+0000 7f8e0f200700 1 Processor -- start 2024-05-03T14:46:12.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:12.854+0000 7f8e0f200700 1 -- start start 2024-05-03T14:46:12.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:12.854+0000 7f8e0f200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e080779d0 0x7f8e081a03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:46:12.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:12.854+0000 7f8e0f200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8e080782f0 0x7f8e081a0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:46:12.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:12.854+0000 7f8e0f200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8e08114330 0x7f8e081a4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:46:12.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:12.854+0000 7f8e0f200700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f8e08119f90 con 0x7f8e080779d0 2024-05-03T14:46:12.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:12.854+0000 7f8e0f200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8e08119e10 con 0x7f8e080782f0 2024-05-03T14:46:12.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:12.854+0000 7f8e0f200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0811a110 con 0x7f8e08114330 2024-05-03T14:46:12.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:12.855+0000 7f8e07e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e080779d0 0x7f8e081a03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:46:12.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:12.855+0000 7f8dff400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8e080782f0 0x7f8e081a0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:46:12.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:12.855+0000 7f8dff400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8e080782f0 0x7f8e081a0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.60:34192/0 (socket says 172.21.15.60:34192) 2024-05-03T14:46:12.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:12.855+0000 7f8dff400700 1 -- 172.21.15.60:0/3769735234 learned_addr learned my addr 172.21.15.60:0/3769735234 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:46:12.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:12.855+0000 7f8e0d200700 1 --2- 172.21.15.60:0/3769735234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8e08114330 0x7f8e081a4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:46:12.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:12.855+0000 7f8dff400700 1 -- 172.21.15.60:0/3769735234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8e08114330 msgr2=0x7f8e081a4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:12.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:12.855+0000 7f8dff400700 1 --2- 172.21.15.60:0/3769735234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8e08114330 0x7f8e081a4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:12.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:12.855+0000 7f8dff400700 1 -- 172.21.15.60:0/3769735234 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e080779d0 msgr2=0x7f8e081a03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:12.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:12.855+0000 7f8dff400700 1 --2- 172.21.15.60:0/3769735234 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e080779d0 0x7f8e081a03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:12.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:12.855+0000 7f8dff400700 1 -- 172.21.15.60:0/3769735234 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e081a53d0 con 0x7f8e080782f0 2024-05-03T14:46:12.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:12.856+0000 7f8dff400700 1 --2- 172.21.15.60:0/3769735234 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8e080782f0 0x7f8e081a0900 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8df000e9a0 tx=0x7f8df000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:46:12.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:12.856+0000 7f8e05600700 1 -- 172.21.15.60:0/3769735234 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df000ca10 con 0x7f8e080782f0 2024-05-03T14:46:12.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:12.856+0000 7f8e05600700 1 -- 172.21.15.60:0/3769735234 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f8df000cbb0 con 0x7f8e080782f0 2024-05-03T14:46:12.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:12.856+0000 7f8e05600700 1 -- 172.21.15.60:0/3769735234 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df00187a0 con 0x7f8e080782f0 2024-05-03T14:46:12.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:12.856+0000 7f8e0f200700 1 -- 172.21.15.60:0/3769735234 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e081a56c0 con 0x7f8e080782f0 2024-05-03T14:46:12.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:12.856+0000 7f8e0f200700 1 -- 172.21.15.60:0/3769735234 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e081acf40 con 0x7f8e080782f0 2024-05-03T14:46:12.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:12.858+0000 7f8e0f200700 1 -- 172.21.15.60:0/3769735234 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e0806a190 con 0x7f8e080782f0 2024-05-03T14:46:12.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:12.860+0000 7f8e05600700 1 -- 172.21.15.60:0/3769735234 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f8df0010430 con 0x7f8e080782f0 2024-05-03T14:46:12.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:12.861+0000 7f8e05600700 1 --2- 172.21.15.60:0/3769735234 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f8dec0b2990 0x7f8dec0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:46:12.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:12.861+0000 7f8e05600700 1 -- 172.21.15.60:0/3769735234 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f8df00e28b0 con 0x7f8e080782f0 2024-05-03T14:46:12.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:12.861+0000 7f8e07e00700 1 --2- 172.21.15.60:0/3769735234 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f8dec0b2990 0x7f8dec0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:46:12.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:12.862+0000 7f8e07e00700 1 --2- 172.21.15.60:0/3769735234 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f8dec0b2990 0x7f8dec0b4e50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8df800b780 tx=0x7f8df8006210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:46:12.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:12.865+0000 7f8e05600700 1 -- 172.21.15.60:0/3769735234 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f8df00aa090 con 0x7f8e080782f0 2024-05-03T14:46:13.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:13.146+0000 7f8e0f200700 1 -- 172.21.15.60:0/3769735234 --> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f8e080638e0 con 0x7f8dec0b2990 2024-05-03T14:46:13.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:13.159+0000 7f8e05600700 1 -- 172.21.15.60:0/3769735234 <== mgr.14152 v2:172.21.15.60:6800/3066142956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f8e080638e0 con 0x7f8dec0b2990 2024-05-03T14:46:13.162 INFO:teuthology.orchestra.run.smithi060.stdout:/volumes/qa/sv_0/56c4748f-1237-4f00-a455-4f8dbf232a34 2024-05-03T14:46:13.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:13.161+0000 7f8e0f200700 1 -- 172.21.15.60:0/3769735234 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f8dec0b2990 msgr2=0x7f8dec0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:13.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:13.161+0000 7f8e0f200700 1 --2- 172.21.15.60:0/3769735234 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f8dec0b2990 0x7f8dec0b4e50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8df800b780 tx=0x7f8df8006210 comp rx=0 tx=0).stop 2024-05-03T14:46:13.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:13.161+0000 7f8e0f200700 1 -- 172.21.15.60:0/3769735234 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8e080782f0 msgr2=0x7f8e081a0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:13.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:13.161+0000 7f8e0f200700 1 --2- 172.21.15.60:0/3769735234 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8e080782f0 0x7f8e081a0900 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8df000e9a0 tx=0x7f8df000ecb0 comp rx=0 tx=0).stop 2024-05-03T14:46:13.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:13.162+0000 7f8e0f200700 1 -- 172.21.15.60:0/3769735234 shutdown_connections 2024-05-03T14:46:13.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:13.162+0000 7f8e0f200700 1 --2- 172.21.15.60:0/3769735234 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f8dec0b2990 0x7f8dec0b4e50 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:13.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:13.162+0000 7f8e0f200700 1 --2- 172.21.15.60:0/3769735234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8e08114330 0x7f8e081a4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:13.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:13.162+0000 7f8e0f200700 1 --2- 172.21.15.60:0/3769735234 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e080779d0 0x7f8e081a03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:13.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:13.162+0000 7f8e0f200700 1 --2- 172.21.15.60:0/3769735234 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8e080782f0 0x7f8e081a0900 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:13.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:13.162+0000 7f8e0f200700 1 -- 172.21.15.60:0/3769735234 >> 172.21.15.60:0/3769735234 conn(0x7f8e08101180 msgr2=0x7f8e08103570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:46:13.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:13.162+0000 7f8e0f200700 1 -- 172.21.15.60:0/3769735234 shutdown_connections 2024-05-03T14:46:13.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:13.162+0000 7f8e0f200700 1 -- 172.21.15.60:0/3769735234 wait complete. 2024-05-03T14:46:13.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:13 smithi149 ceph-mon[33332]: pgmap v383: 129 pgs: 129 active+clean; 607 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 3.2 KiB/s wr, 11 op/s; 83 B/s, 0 objects/s recovering 2024-05-03T14:46:14.016 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:13 smithi060 ceph-mon[28873]: pgmap v383: 129 pgs: 129 active+clean; 607 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 3.2 KiB/s wr, 11 op/s; 83 B/s, 0 objects/s recovering 2024-05-03T14:46:14.095 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-05-03T14:46:14.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:13 smithi090 ceph-mon[33568]: pgmap v383: 129 pgs: 129 active+clean; 607 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 3.2 KiB/s wr, 11 op/s; 83 B/s, 0 objects/s recovering 2024-05-03T14:46:14.559 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:46:14.894 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:14 smithi060 ceph-mon[28873]: from='client.24692 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:46:14.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:14 smithi149 ceph-mon[33332]: from='client.24692 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:46:15.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:14 smithi090 ceph-mon[33568]: from='client.24692 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:46:15.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:15 smithi060 ceph-mon[28873]: pgmap v384: 129 pgs: 129 active+clean; 607 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.8 KiB/s wr, 11 op/s; 68 B/s, 0 objects/s recovering 2024-05-03T14:46:15.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:15 smithi149 ceph-mon[33332]: pgmap v384: 129 pgs: 129 active+clean; 607 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.8 KiB/s wr, 11 op/s; 68 B/s, 0 objects/s recovering 2024-05-03T14:46:16.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:15 smithi090 ceph-mon[33568]: pgmap v384: 129 pgs: 129 active+clean; 607 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.8 KiB/s wr, 11 op/s; 68 B/s, 0 objects/s recovering 2024-05-03T14:46:16.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:16.798+0000 7f9e90800700 1 -- 172.21.15.60:0/2097766124 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e88108410 msgr2=0x7f9e881087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:16.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:16.798+0000 7f9e90800700 1 --2- 172.21.15.60:0/2097766124 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e88108410 0x7f9e881087f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c009990 tx=0x7f9e7c009ca0 comp rx=0 tx=0).stop 2024-05-03T14:46:16.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:16.799+0000 7f9e90800700 1 -- 172.21.15.60:0/2097766124 shutdown_connections 2024-05-03T14:46:16.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:16.799+0000 7f9e90800700 1 --2- 172.21.15.60:0/2097766124 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9e88102d60 0x7f9e8810f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:16.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:16.799+0000 7f9e90800700 1 --2- 172.21.15.60:0/2097766124 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e88108410 0x7f9e881087f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:16.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:16.799+0000 7f9e90800700 1 --2- 172.21.15.60:0/2097766124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e881023a0 0x7f9e88102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:16.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:16.799+0000 7f9e90800700 1 -- 172.21.15.60:0/2097766124 >> 172.21.15.60:0/2097766124 conn(0x7f9e88079350 msgr2=0x7f9e88079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:46:16.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:16.799+0000 7f9e90800700 1 -- 172.21.15.60:0/2097766124 shutdown_connections 2024-05-03T14:46:16.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:16.799+0000 7f9e90800700 1 -- 172.21.15.60:0/2097766124 wait complete. 2024-05-03T14:46:16.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:16.800+0000 7f9e90800700 1 Processor -- start 2024-05-03T14:46:16.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:16.800+0000 7f9e90800700 1 -- start start 2024-05-03T14:46:16.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:16.801+0000 7f9e90800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e881023a0 0x7f9e8819bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:46:16.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:16.801+0000 7f9e90800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9e88102d60 0x7f9e8819c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:46:16.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:16.801+0000 7f9e90800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e88108410 0x7f9e881a0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:46:16.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:16.801+0000 7f9e90800700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f9e881127c0 con 0x7f9e88108410 2024-05-03T14:46:16.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:16.801+0000 7f9e90800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9e88112640 con 0x7f9e881023a0 2024-05-03T14:46:16.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:16.801+0000 7f9e90800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9e88112940 con 0x7f9e88102d60 2024-05-03T14:46:16.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:16.801+0000 7f9e8de00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e881023a0 0x7f9e8819bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:46:16.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:16.801+0000 7f9e8de00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e881023a0 0x7f9e8819bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.60:34218/0 (socket says 172.21.15.60:34218) 2024-05-03T14:46:16.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:16.801+0000 7f9e8de00700 1 -- 172.21.15.60:0/1558232969 learned_addr learned my addr 172.21.15.60:0/1558232969 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:46:16.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:16.802+0000 7f9e8d400700 1 --2- 172.21.15.60:0/1558232969 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9e88102d60 0x7f9e8819c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:46:16.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:16.802+0000 7f9e8e800700 1 --2- 172.21.15.60:0/1558232969 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e88108410 0x7f9e881a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:46:16.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:16.802+0000 7f9e8d400700 1 -- 172.21.15.60:0/1558232969 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e881023a0 msgr2=0x7f9e8819bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:16.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:16.802+0000 7f9e8d400700 1 --2- 172.21.15.60:0/1558232969 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e881023a0 0x7f9e8819bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:16.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:16.802+0000 7f9e8d400700 1 -- 172.21.15.60:0/1558232969 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e88108410 msgr2=0x7f9e881a0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:16.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:16.802+0000 7f9e8d400700 1 --2- 172.21.15.60:0/1558232969 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e88108410 0x7f9e881a0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:16.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:16.802+0000 7f9e8d400700 1 -- 172.21.15.60:0/1558232969 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e881a0f70 con 0x7f9e88102d60 2024-05-03T14:46:16.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:16.802+0000 7f9e8de00700 1 --2- 172.21.15.60:0/1558232969 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e881023a0 0x7f9e8819bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:46:16.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:16.802+0000 7f9e8d400700 1 --2- 172.21.15.60:0/1558232969 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9e88102d60 0x7f9e8819c4a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9e7800e9f0 tx=0x7f9e7800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:46:16.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:16.803+0000 7f9e82a00700 1 -- 172.21.15.60:0/1558232969 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7800cbd0 con 0x7f9e88102d60 2024-05-03T14:46:16.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:16.803+0000 7f9e82a00700 1 -- 172.21.15.60:0/1558232969 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f9e7800cd70 con 0x7f9e88102d60 2024-05-03T14:46:16.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:16.803+0000 7f9e82a00700 1 -- 172.21.15.60:0/1558232969 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e780187a0 con 0x7f9e88102d60 2024-05-03T14:46:16.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:16.803+0000 7f9e90800700 1 -- 172.21.15.60:0/1558232969 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e881a1260 con 0x7f9e88102d60 2024-05-03T14:46:16.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:16.803+0000 7f9e90800700 1 -- 172.21.15.60:0/1558232969 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e881a8ae0 con 0x7f9e88102d60 2024-05-03T14:46:16.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:16.804+0000 7f9e90800700 1 -- 172.21.15.60:0/1558232969 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e8806a190 con 0x7f9e88102d60 2024-05-03T14:46:16.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:16.806+0000 7f9e82a00700 1 -- 172.21.15.60:0/1558232969 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f9e78010430 con 0x7f9e88102d60 2024-05-03T14:46:16.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:16.808+0000 7f9e82a00700 1 --2- 172.21.15.60:0/1558232969 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f9e700b2910 0x7f9e700b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:46:16.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:16.808+0000 7f9e82a00700 1 -- 172.21.15.60:0/1558232969 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f9e780e2950 con 0x7f9e88102d60 2024-05-03T14:46:16.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:16.808+0000 7f9e8de00700 1 --2- 172.21.15.60:0/1558232969 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f9e700b2910 0x7f9e700b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:46:16.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:16.809+0000 7f9e8de00700 1 --2- 172.21.15.60:0/1558232969 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f9e700b2910 0x7f9e700b4dd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c00b2e0 tx=0x7f9e7c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:46:16.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:16.812+0000 7f9e82a00700 1 -- 172.21.15.60:0/1558232969 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f9e780aa130 con 0x7f9e88102d60 2024-05-03T14:46:17.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:17.082+0000 7f9e90800700 1 -- 172.21.15.60:0/1558232969 --> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f9e8806b990 con 0x7f9e700b2910 2024-05-03T14:46:17.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:17.135+0000 7f9e82a00700 1 -- 172.21.15.60:0/1558232969 <== mgr.14152 v2:172.21.15.60:6800/3066142956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9e8806b990 con 0x7f9e700b2910 2024-05-03T14:46:17.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:17.139+0000 7f9e90800700 1 -- 172.21.15.60:0/1558232969 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f9e700b2910 msgr2=0x7f9e700b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:17.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:17.139+0000 7f9e90800700 1 --2- 172.21.15.60:0/1558232969 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f9e700b2910 0x7f9e700b4dd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c00b2e0 tx=0x7f9e7c01a040 comp rx=0 tx=0).stop 2024-05-03T14:46:17.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:17.139+0000 7f9e90800700 1 -- 172.21.15.60:0/1558232969 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9e88102d60 msgr2=0x7f9e8819c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:17.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:17.139+0000 7f9e90800700 1 --2- 172.21.15.60:0/1558232969 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9e88102d60 0x7f9e8819c4a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9e7800e9f0 tx=0x7f9e7800ed00 comp rx=0 tx=0).stop 2024-05-03T14:46:17.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:17.139+0000 7f9e90800700 1 -- 172.21.15.60:0/1558232969 shutdown_connections 2024-05-03T14:46:17.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:17.139+0000 7f9e90800700 1 --2- 172.21.15.60:0/1558232969 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f9e700b2910 0x7f9e700b4dd0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:17.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:17.139+0000 7f9e90800700 1 --2- 172.21.15.60:0/1558232969 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e88108410 0x7f9e881a0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:17.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:17.139+0000 7f9e90800700 1 --2- 172.21.15.60:0/1558232969 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e881023a0 0x7f9e8819bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:17.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:17.139+0000 7f9e90800700 1 --2- 172.21.15.60:0/1558232969 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9e88102d60 0x7f9e8819c4a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:17.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:17.139+0000 7f9e90800700 1 -- 172.21.15.60:0/1558232969 >> 172.21.15.60:0/1558232969 conn(0x7f9e88079350 msgr2=0x7f9e8806b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:46:17.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:17.140+0000 7f9e90800700 1 -- 172.21.15.60:0/1558232969 shutdown_connections 2024-05-03T14:46:17.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:17.140+0000 7f9e90800700 1 -- 172.21.15.60:0/1558232969 wait complete. 2024-05-03T14:46:17.915 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:17 smithi060 ceph-mon[28873]: pgmap v385: 129 pgs: 129 active+clean; 613 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.2 KiB/s wr, 10 op/s; 62 B/s, 0 objects/s recovering 2024-05-03T14:46:17.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:17 smithi149 ceph-mon[33332]: pgmap v385: 129 pgs: 129 active+clean; 613 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.2 KiB/s wr, 10 op/s; 62 B/s, 0 objects/s recovering 2024-05-03T14:46:17.998 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-05-03T14:46:18.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:17 smithi090 ceph-mon[33568]: pgmap v385: 129 pgs: 129 active+clean; 613 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.2 KiB/s wr, 10 op/s; 62 B/s, 0 objects/s recovering 2024-05-03T14:46:18.481 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:46:18.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:18 smithi149 ceph-mon[33332]: from='client.24709 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:46:19.019 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:18 smithi060 ceph-mon[28873]: from='client.24709 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:46:19.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:18 smithi090 ceph-mon[33568]: from='client.24709 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:46:19.773 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:19 smithi060 ceph-mon[28873]: pgmap v386: 129 pgs: 129 active+clean; 613 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.9 KiB/s wr, 10 op/s; 55 B/s, 0 objects/s recovering 2024-05-03T14:46:19.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:19 smithi149 ceph-mon[33332]: pgmap v386: 129 pgs: 129 active+clean; 613 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.9 KiB/s wr, 10 op/s; 55 B/s, 0 objects/s recovering 2024-05-03T14:46:20.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:19 smithi090 ceph-mon[33568]: pgmap v386: 129 pgs: 129 active+clean; 613 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.9 KiB/s wr, 10 op/s; 55 B/s, 0 objects/s recovering 2024-05-03T14:46:20.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:20.665+0000 7f5bd3e00700 1 -- 172.21.15.60:0/364091607 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5bcc1056e0 msgr2=0x7f5bcc105b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:20.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:20.665+0000 7f5bd3e00700 1 --2- 172.21.15.60:0/364091607 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5bcc1056e0 0x7f5bcc105b60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8009990 tx=0x7f5bb8009ca0 comp rx=0 tx=0).stop 2024-05-03T14:46:20.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:20.666+0000 7f5bd3e00700 1 -- 172.21.15.60:0/364091607 shutdown_connections 2024-05-03T14:46:20.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:20.666+0000 7f5bd3e00700 1 --2- 172.21.15.60:0/364091607 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5bcc1060a0 0x7f5bcc1125b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:20.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:20.666+0000 7f5bd3e00700 1 --2- 172.21.15.60:0/364091607 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5bcc10b6e0 0x7f5bcc10bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:20.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:20.666+0000 7f5bd3e00700 1 --2- 172.21.15.60:0/364091607 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5bcc1056e0 0x7f5bcc105b60 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:20.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:20.666+0000 7f5bd3e00700 1 -- 172.21.15.60:0/364091607 >> 172.21.15.60:0/364091607 conn(0x7f5bcc1011d0 msgr2=0x7f5bcc1035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:46:20.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:20.666+0000 7f5bd3e00700 1 -- 172.21.15.60:0/364091607 shutdown_connections 2024-05-03T14:46:20.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:20.666+0000 7f5bd3e00700 1 -- 172.21.15.60:0/364091607 wait complete. 2024-05-03T14:46:20.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:20.667+0000 7f5bd3e00700 1 Processor -- start 2024-05-03T14:46:20.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:20.667+0000 7f5bd3e00700 1 -- start start 2024-05-03T14:46:20.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:20.667+0000 7f5bd3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5bcc1056e0 0x7f5bcc19bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:46:20.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:20.668+0000 7f5bd3e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5bcc1060a0 0x7f5bcc19c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:46:20.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:20.668+0000 7f5bd3e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5bcc10b6e0 0x7f5bcc1a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:46:20.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:20.668+0000 7f5bd3e00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5bcc115b80 con 0x7f5bcc1060a0 2024-05-03T14:46:20.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:20.668+0000 7f5bd3e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5bcc115a00 con 0x7f5bcc1056e0 2024-05-03T14:46:20.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:20.668+0000 7f5bd3e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5bcc115d00 con 0x7f5bcc10b6e0 2024-05-03T14:46:20.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:20.668+0000 7f5bd1400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5bcc1056e0 0x7f5bcc19bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:46:20.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:20.668+0000 7f5bd0a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5bcc1060a0 0x7f5bcc19c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:46:20.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:20.668+0000 7f5bd1400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5bcc1056e0 0x7f5bcc19bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.60:34230/0 (socket says 172.21.15.60:34230) 2024-05-03T14:46:20.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:20.668+0000 7f5bd1400700 1 -- 172.21.15.60:0/2706123814 learned_addr learned my addr 172.21.15.60:0/2706123814 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:46:20.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:20.668+0000 7f5bd1e00700 1 --2- 172.21.15.60:0/2706123814 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5bcc10b6e0 0x7f5bcc1a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:46:20.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:20.668+0000 7f5bd1400700 1 -- 172.21.15.60:0/2706123814 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5bcc10b6e0 msgr2=0x7f5bcc1a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:20.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:20.668+0000 7f5bd1400700 1 --2- 172.21.15.60:0/2706123814 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5bcc10b6e0 0x7f5bcc1a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:20.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:20.668+0000 7f5bd1400700 1 -- 172.21.15.60:0/2706123814 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5bcc1060a0 msgr2=0x7f5bcc19c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:20.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:20.668+0000 7f5bd1400700 1 --2- 172.21.15.60:0/2706123814 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5bcc1060a0 0x7f5bcc19c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:20.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:20.668+0000 7f5bd1400700 1 -- 172.21.15.60:0/2706123814 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bcc1a0f60 con 0x7f5bcc1056e0 2024-05-03T14:46:20.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:20.669+0000 7f5bd0a00700 1 --2- 172.21.15.60:0/2706123814 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5bcc1060a0 0x7f5bcc19c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:46:20.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:20.669+0000 7f5bd1400700 1 --2- 172.21.15.60:0/2706123814 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5bcc1056e0 0x7f5bcc19bf50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5bc000d980 tx=0x7f5bc000dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:46:20.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:20.669+0000 7f5bca000700 1 -- 172.21.15.60:0/2706123814 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc0016070 con 0x7f5bcc1056e0 2024-05-03T14:46:20.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:20.669+0000 7f5bca000700 1 -- 172.21.15.60:0/2706123814 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f5bc0004d30 con 0x7f5bcc1056e0 2024-05-03T14:46:20.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:20.669+0000 7f5bd3e00700 1 -- 172.21.15.60:0/2706123814 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bcc1a1250 con 0x7f5bcc1056e0 2024-05-03T14:46:20.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:20.669+0000 7f5bca000700 1 -- 172.21.15.60:0/2706123814 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc00056c0 con 0x7f5bcc1056e0 2024-05-03T14:46:20.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:20.669+0000 7f5bd3e00700 1 -- 172.21.15.60:0/2706123814 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bcc1a8ad0 con 0x7f5bcc1056e0 2024-05-03T14:46:20.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:20.670+0000 7f5bd3e00700 1 -- 172.21.15.60:0/2706123814 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bcc06a190 con 0x7f5bcc1056e0 2024-05-03T14:46:20.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:20.674+0000 7f5bca000700 1 -- 172.21.15.60:0/2706123814 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f5bc0004750 con 0x7f5bcc1056e0 2024-05-03T14:46:20.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:20.675+0000 7f5bca000700 1 --2- 172.21.15.60:0/2706123814 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f5bb40b2990 0x7f5bb40b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:46:20.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:20.676+0000 7f5bca000700 1 -- 172.21.15.60:0/2706123814 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f5bc00e2380 con 0x7f5bcc1056e0 2024-05-03T14:46:20.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:20.676+0000 7f5bd0a00700 1 --2- 172.21.15.60:0/2706123814 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f5bb40b2990 0x7f5bb40b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:46:20.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:20.677+0000 7f5bd0a00700 1 --2- 172.21.15.60:0/2706123814 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f5bb40b2990 0x7f5bb40b4e50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc10abc0 tx=0x7f5bb801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:46:20.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:20.680+0000 7f5bca000700 1 -- 172.21.15.60:0/2706123814 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f5bc001f020 con 0x7f5bcc1056e0 2024-05-03T14:46:20.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:20.956+0000 7f5bd3e00700 1 -- 172.21.15.60:0/2706123814 --> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f5bcc101dc0 con 0x7f5bb40b2990 2024-05-03T14:46:20.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:20.967+0000 7f5bca000700 1 -- 172.21.15.60:0/2706123814 <== mgr.14152 v2:172.21.15.60:6800/3066142956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f5bcc101dc0 con 0x7f5bb40b2990 2024-05-03T14:46:20.970 INFO:teuthology.orchestra.run.smithi060.stdout:/volumes/qa/sv_1/65eceeb5-e868-44a6-8a80-e9d8bf9f5757 2024-05-03T14:46:20.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:20.970+0000 7f5bd3e00700 1 -- 172.21.15.60:0/2706123814 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f5bb40b2990 msgr2=0x7f5bb40b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:20.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:20.970+0000 7f5bd3e00700 1 --2- 172.21.15.60:0/2706123814 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f5bb40b2990 0x7f5bb40b4e50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc10abc0 tx=0x7f5bb801a040 comp rx=0 tx=0).stop 2024-05-03T14:46:20.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:20.970+0000 7f5bd3e00700 1 -- 172.21.15.60:0/2706123814 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5bcc1056e0 msgr2=0x7f5bcc19bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:20.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:20.970+0000 7f5bd3e00700 1 --2- 172.21.15.60:0/2706123814 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5bcc1056e0 0x7f5bcc19bf50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5bc000d980 tx=0x7f5bc000dc90 comp rx=0 tx=0).stop 2024-05-03T14:46:20.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:20.970+0000 7f5bd3e00700 1 -- 172.21.15.60:0/2706123814 shutdown_connections 2024-05-03T14:46:20.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:20.970+0000 7f5bd3e00700 1 --2- 172.21.15.60:0/2706123814 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f5bb40b2990 0x7f5bb40b4e50 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:20.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:20.970+0000 7f5bd3e00700 1 --2- 172.21.15.60:0/2706123814 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5bcc10b6e0 0x7f5bcc1a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:20.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:20.970+0000 7f5bd3e00700 1 --2- 172.21.15.60:0/2706123814 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5bcc1056e0 0x7f5bcc19bf50 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:20.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:20.970+0000 7f5bd3e00700 1 --2- 172.21.15.60:0/2706123814 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5bcc1060a0 0x7f5bcc19c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:20.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:20.970+0000 7f5bd3e00700 1 -- 172.21.15.60:0/2706123814 >> 172.21.15.60:0/2706123814 conn(0x7f5bcc1011d0 msgr2=0x7f5bcc1016b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:46:20.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:20.971+0000 7f5bd3e00700 1 -- 172.21.15.60:0/2706123814 shutdown_connections 2024-05-03T14:46:20.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:20.971+0000 7f5bd3e00700 1 -- 172.21.15.60:0/2706123814 wait complete. 2024-05-03T14:46:21.796 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T14:46:21.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:21 smithi149 ceph-mon[33332]: pgmap v387: 129 pgs: 129 active+clean; 630 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 4.5 KiB/s wr, 14 op/s 2024-05-03T14:46:22.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:21 smithi060 ceph-mon[28873]: pgmap v387: 129 pgs: 129 active+clean; 630 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 4.5 KiB/s wr, 14 op/s 2024-05-03T14:46:22.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:21 smithi090 ceph-mon[33568]: pgmap v387: 129 pgs: 129 active+clean; 630 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 4.5 KiB/s wr, 14 op/s 2024-05-03T14:46:22.291 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:46:22.792 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:22 smithi060 ceph-mon[28873]: from='client.24704 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:46:23.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:22 smithi090 ceph-mon[33568]: from='client.24704 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:46:23.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:22 smithi149 ceph-mon[33332]: from='client.24704 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:46:23.941 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:23 smithi060 ceph-mon[28873]: pgmap v388: 129 pgs: 129 active+clean; 630 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-05-03T14:46:24.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:23 smithi090 ceph-mon[33568]: pgmap v388: 129 pgs: 129 active+clean; 630 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-05-03T14:46:24.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:23 smithi149 ceph-mon[33332]: pgmap v388: 129 pgs: 129 active+clean; 630 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-05-03T14:46:24.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:24.502+0000 7fe0ce800700 1 -- 172.21.15.60:0/3395954032 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe0c81056c0 msgr2=0x7fe0c8105b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:24.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:24.502+0000 7fe0ce800700 1 --2- 172.21.15.60:0/3395954032 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe0c81056c0 0x7fe0c8105b40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4009990 tx=0x7fe0b4009ca0 comp rx=0 tx=0).stop 2024-05-03T14:46:24.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:24.502+0000 7fe0ce800700 1 -- 172.21.15.60:0/3395954032 shutdown_connections 2024-05-03T14:46:24.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:24.502+0000 7fe0ce800700 1 --2- 172.21.15.60:0/3395954032 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0c8106080 0x7fe0c8112590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:24.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:24.502+0000 7fe0ce800700 1 --2- 172.21.15.60:0/3395954032 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0c810b6c0 0x7fe0c810baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:24.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:24.502+0000 7fe0ce800700 1 --2- 172.21.15.60:0/3395954032 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe0c81056c0 0x7fe0c8105b40 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:24.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:24.502+0000 7fe0ce800700 1 -- 172.21.15.60:0/3395954032 >> 172.21.15.60:0/3395954032 conn(0x7fe0c81011b0 msgr2=0x7fe0c81035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:46:24.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:24.503+0000 7fe0ce800700 1 -- 172.21.15.60:0/3395954032 shutdown_connections 2024-05-03T14:46:24.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:24.503+0000 7fe0ce800700 1 -- 172.21.15.60:0/3395954032 wait complete. 2024-05-03T14:46:24.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:24.504+0000 7fe0ce800700 1 Processor -- start 2024-05-03T14:46:24.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:24.504+0000 7fe0ce800700 1 -- start start 2024-05-03T14:46:24.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:24.504+0000 7fe0ce800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0c81056c0 0x7fe0c819bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:46:24.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:24.504+0000 7fe0ce800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0c8106080 0x7fe0c819c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:46:24.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:24.504+0000 7fe0ce800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe0c810b6c0 0x7fe0c81a0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:46:24.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:24.504+0000 7fe0ce800700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c8115bc0 con 0x7fe0c8106080 2024-05-03T14:46:24.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:24.504+0000 7fe0ce800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c8115a40 con 0x7fe0c81056c0 2024-05-03T14:46:24.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:24.504+0000 7fe0ce800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c8115d40 con 0x7fe0c810b6c0 2024-05-03T14:46:24.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:24.505+0000 7fe0c6a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0c8106080 0x7fe0c819c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:46:24.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:24.505+0000 7fe0c7400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0c81056c0 0x7fe0c819bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:46:24.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:24.505+0000 7fe0c7400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0c81056c0 0x7fe0c819bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.60:54710/0 (socket says 172.21.15.60:54710) 2024-05-03T14:46:24.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:24.505+0000 7fe0c7400700 1 -- 172.21.15.60:0/2619167542 learned_addr learned my addr 172.21.15.60:0/2619167542 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:46:24.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:24.505+0000 7fe0c7e00700 1 --2- 172.21.15.60:0/2619167542 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe0c810b6c0 0x7fe0c81a0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:46:24.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:24.505+0000 7fe0c7400700 1 -- 172.21.15.60:0/2619167542 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe0c810b6c0 msgr2=0x7fe0c81a0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:24.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:24.505+0000 7fe0c7400700 1 --2- 172.21.15.60:0/2619167542 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe0c810b6c0 0x7fe0c81a0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:24.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:24.505+0000 7fe0c7400700 1 -- 172.21.15.60:0/2619167542 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0c8106080 msgr2=0x7fe0c819c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:24.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:24.505+0000 7fe0c7400700 1 --2- 172.21.15.60:0/2619167542 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0c8106080 0x7fe0c819c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:24.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:24.505+0000 7fe0c7400700 1 -- 172.21.15.60:0/2619167542 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0c81a0f90 con 0x7fe0c81056c0 2024-05-03T14:46:24.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:24.506+0000 7fe0c7400700 1 --2- 172.21.15.60:0/2619167542 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0c81056c0 0x7fe0c819bfd0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe0b800e9a0 tx=0x7fe0b800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:46:24.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:24.506+0000 7fe0bfe00700 1 -- 172.21.15.60:0/2619167542 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b800ca10 con 0x7fe0c81056c0 2024-05-03T14:46:24.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:24.506+0000 7fe0bfe00700 1 -- 172.21.15.60:0/2619167542 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7fe0b800cbb0 con 0x7fe0c81056c0 2024-05-03T14:46:24.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:24.506+0000 7fe0ce800700 1 -- 172.21.15.60:0/2619167542 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0c81a1280 con 0x7fe0c81056c0 2024-05-03T14:46:24.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:24.506+0000 7fe0bfe00700 1 -- 172.21.15.60:0/2619167542 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b80187a0 con 0x7fe0c81056c0 2024-05-03T14:46:24.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:24.506+0000 7fe0ce800700 1 -- 172.21.15.60:0/2619167542 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0c81a8b00 con 0x7fe0c81056c0 2024-05-03T14:46:24.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:24.507+0000 7fe0ce800700 1 -- 172.21.15.60:0/2619167542 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0c806a190 con 0x7fe0c81056c0 2024-05-03T14:46:24.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:24.510+0000 7fe0bfe00700 1 -- 172.21.15.60:0/2619167542 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fe0b804aae0 con 0x7fe0c81056c0 2024-05-03T14:46:24.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:24.511+0000 7fe0bfe00700 1 --2- 172.21.15.60:0/2619167542 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fe0ac0b2990 0x7fe0ac0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:46:24.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:24.511+0000 7fe0bfe00700 1 -- 172.21.15.60:0/2619167542 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7fe0b80e2880 con 0x7fe0c81056c0 2024-05-03T14:46:24.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:24.512+0000 7fe0c6a00700 1 --2- 172.21.15.60:0/2619167542 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fe0ac0b2990 0x7fe0ac0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:46:24.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:24.512+0000 7fe0c6a00700 1 --2- 172.21.15.60:0/2619167542 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fe0ac0b2990 0x7fe0ac0b4e50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe0c810ab80 tx=0x7fe0b400b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:46:24.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:24.514+0000 7fe0bfe00700 1 -- 172.21.15.60:0/2619167542 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fe0b80aa060 con 0x7fe0c81056c0 2024-05-03T14:46:24.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:24.803+0000 7fe0ce800700 1 -- 172.21.15.60:0/2619167542 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe0c802e510 con 0x7fe0c81056c0 2024-05-03T14:46:24.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:24.804+0000 7fe0bfe00700 1 -- 172.21.15.60:0/2619167542 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11929 (secure 0 0 0) 0x7fe0b80b20e0 con 0x7fe0c81056c0 2024-05-03T14:46:24.808 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:46:24.808 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1849151522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1849151522},{"type":"v1","addr":"172.21.15.60:6835","nonce":1849151522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/523710346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":523710346},{"type":"v1","addr":"172.21.15.60:6837","nonce":523710346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T14:44:34.226052+0000","modified":"2024-05-03T14:46:01.367913+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24511,"mds_1":24509,"mds_2":24487,"mds_3":24475,"mds_4":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14397":{"gid":14397,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.60:6839/3846205424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3846205424},{"type":"v1","addr":"172.21.15.60:6839","nonce":3846205424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6833/3033424631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":3033424631},{"type":"v1","addr":"172.21.15.90:6833","nonce":3033424631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6833/3454021291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3454021291},{"type":"v1","addr":"172.21.15.149:6833","nonce":3454021291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6835/1057049585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1057049585},{"type":"v1","addr":"172.21.15.90:6835","nonce":1057049585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/882239924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":882239924},{"type":"v1","addr":"172.21.15.149:6835","nonce":882239924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":31,"addr":"172.21.15.90:6837/2243702844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":2243702844},{"type":"v1","addr":"172.21.15.90:6837","nonce":2243702844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":32,"addr":"172.21.15.149:6837/2611278690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2611278690},{"type":"v1","addr":"172.21.15.149:6837","nonce":2611278690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.149:6839/2527622212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2527622212},{"type":"v1","addr":"172.21.15.149:6839","nonce":2527622212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.90:6839/3512381172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3512381172},{"type":"v1","addr":"172.21.15.90:6839","nonce":3512381172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.60:6841/924163182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":924163182},{"type":"v1","addr":"172.21.15.60:6841","nonce":924163182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24511,"qdb_cluster":[24470,24475,24487,24509,24511]},"id":1}]} 2024-05-03T14:46:24.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:24.807+0000 7fe0ce800700 1 -- 172.21.15.60:0/2619167542 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fe0ac0b2990 msgr2=0x7fe0ac0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:24.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:24.807+0000 7fe0ce800700 1 --2- 172.21.15.60:0/2619167542 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fe0ac0b2990 0x7fe0ac0b4e50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe0c810ab80 tx=0x7fe0b400b260 comp rx=0 tx=0).stop 2024-05-03T14:46:24.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:24.807+0000 7fe0ce800700 1 -- 172.21.15.60:0/2619167542 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0c81056c0 msgr2=0x7fe0c819bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:24.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:24.807+0000 7fe0ce800700 1 --2- 172.21.15.60:0/2619167542 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0c81056c0 0x7fe0c819bfd0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe0b800e9a0 tx=0x7fe0b800ecb0 comp rx=0 tx=0).stop 2024-05-03T14:46:24.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:24.808+0000 7fe0ce800700 1 -- 172.21.15.60:0/2619167542 shutdown_connections 2024-05-03T14:46:24.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:24.808+0000 7fe0ce800700 1 --2- 172.21.15.60:0/2619167542 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fe0ac0b2990 0x7fe0ac0b4e50 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:24.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:24.808+0000 7fe0ce800700 1 --2- 172.21.15.60:0/2619167542 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe0c810b6c0 0x7fe0c81a0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:24.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:24.808+0000 7fe0ce800700 1 --2- 172.21.15.60:0/2619167542 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0c81056c0 0x7fe0c819bfd0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:24.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:24.808+0000 7fe0ce800700 1 --2- 172.21.15.60:0/2619167542 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0c8106080 0x7fe0c819c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:24.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:24.808+0000 7fe0ce800700 1 -- 172.21.15.60:0/2619167542 >> 172.21.15.60:0/2619167542 conn(0x7fe0c81011b0 msgr2=0x7fe0c81016d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:46:24.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:24.808+0000 7fe0ce800700 1 -- 172.21.15.60:0/2619167542 shutdown_connections 2024-05-03T14:46:24.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:24.808+0000 7fe0ce800700 1 -- 172.21.15.60:0/2619167542 wait complete. 2024-05-03T14:46:24.812 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 29 2024-05-03T14:46:25.625 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-03T14:46:25.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:25 smithi060 ceph-mon[28873]: pgmap v389: 129 pgs: 129 active+clean; 630 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.5 KiB/s wr, 11 op/s 2024-05-03T14:46:25.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:25 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/2619167542' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T14:46:26.103 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:46:26.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:25 smithi090 ceph-mon[33568]: pgmap v389: 129 pgs: 129 active+clean; 630 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.5 KiB/s wr, 11 op/s 2024-05-03T14:46:26.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:25 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/2619167542' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T14:46:26.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:25 smithi149 ceph-mon[33332]: pgmap v389: 129 pgs: 129 active+clean; 630 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.5 KiB/s wr, 11 op/s 2024-05-03T14:46:26.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:25 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/2619167542' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T14:46:27.957 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:27 smithi060 ceph-mon[28873]: pgmap v390: 129 pgs: 129 active+clean; 634 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.9 KiB/s wr, 12 op/s 2024-05-03T14:46:28.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:27 smithi090 ceph-mon[33568]: pgmap v390: 129 pgs: 129 active+clean; 634 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.9 KiB/s wr, 12 op/s 2024-05-03T14:46:28.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:27 smithi149 ceph-mon[33332]: pgmap v390: 129 pgs: 129 active+clean; 634 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.9 KiB/s wr, 12 op/s 2024-05-03T14:46:28.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:28.276+0000 7f8526800700 1 -- 172.21.15.60:0/3088941363 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8520109ad0 msgr2=0x7f8520109f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:28.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:28.276+0000 7f8526800700 1 --2- 172.21.15.60:0/3088941363 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8520109ad0 0x7f8520109f50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8508009990 tx=0x7f8508009ca0 comp rx=0 tx=0).stop 2024-05-03T14:46:28.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:28.276+0000 7f8526800700 1 -- 172.21.15.60:0/3088941363 shutdown_connections 2024-05-03T14:46:28.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:28.276+0000 7f8526800700 1 --2- 172.21.15.60:0/3088941363 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f852010a490 0x7f85201169a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:28.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:28.276+0000 7f8526800700 1 --2- 172.21.15.60:0/3088941363 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f852010fb60 0x7f852010ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:28.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:28.276+0000 7f8526800700 1 --2- 172.21.15.60:0/3088941363 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8520109ad0 0x7f8520109f50 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:28.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:28.276+0000 7f8526800700 1 -- 172.21.15.60:0/3088941363 >> 172.21.15.60:0/3088941363 conn(0x7f8520076c70 msgr2=0x7f85201082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:46:28.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:28.276+0000 7f8526800700 1 -- 172.21.15.60:0/3088941363 shutdown_connections 2024-05-03T14:46:28.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:28.276+0000 7f8526800700 1 -- 172.21.15.60:0/3088941363 wait complete. 2024-05-03T14:46:28.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:28.277+0000 7f8526800700 1 Processor -- start 2024-05-03T14:46:28.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:28.277+0000 7f8526800700 1 -- start start 2024-05-03T14:46:28.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:28.278+0000 7f8526800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8520109ad0 0x7f85201a0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:46:28.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:28.278+0000 7f8526800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f852010a490 0x7f85201a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:46:28.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:28.278+0000 7f8526800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f852010fb60 0x7f85201a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:46:28.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:28.278+0000 7f8526800700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f8520119f70 con 0x7f852010fb60 2024-05-03T14:46:28.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:28.278+0000 7f8526800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8520119df0 con 0x7f8520109ad0 2024-05-03T14:46:28.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:28.278+0000 7f8526800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f852011a0f0 con 0x7f852010a490 2024-05-03T14:46:28.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:28.278+0000 7f8516a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f852010a490 0x7f85201a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:46:28.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:28.278+0000 7f851f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8520109ad0 0x7f85201a0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:46:28.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:28.278+0000 7f8516a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f852010a490 0x7f85201a08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.60:35100/0 (socket says 172.21.15.60:35100) 2024-05-03T14:46:28.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:28.278+0000 7f8516a00700 1 -- 172.21.15.60:0/3359446385 learned_addr learned my addr 172.21.15.60:0/3359446385 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:46:28.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:28.278+0000 7f851fe00700 1 --2- 172.21.15.60:0/3359446385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f852010fb60 0x7f85201a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:46:28.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:28.278+0000 7f8516a00700 1 -- 172.21.15.60:0/3359446385 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8520109ad0 msgr2=0x7f85201a0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:28.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:28.278+0000 7f8516a00700 1 --2- 172.21.15.60:0/3359446385 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8520109ad0 0x7f85201a0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:28.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:28.278+0000 7f8516a00700 1 -- 172.21.15.60:0/3359446385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f852010fb60 msgr2=0x7f85201a4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:28.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:28.278+0000 7f8516a00700 1 --2- 172.21.15.60:0/3359446385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f852010fb60 0x7f85201a4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:28.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:28.279+0000 7f8516a00700 1 -- 172.21.15.60:0/3359446385 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85201a5340 con 0x7f852010a490 2024-05-03T14:46:28.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:28.279+0000 7f8516a00700 1 --2- 172.21.15.60:0/3359446385 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f852010a490 0x7f85201a08c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8508005910 tx=0x7f85080047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:46:28.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:28.279+0000 7f851cc00700 1 -- 172.21.15.60:0/3359446385 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8508027070 con 0x7f852010a490 2024-05-03T14:46:28.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:28.279+0000 7f8526800700 1 -- 172.21.15.60:0/3359446385 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85201a55d0 con 0x7f852010a490 2024-05-03T14:46:28.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:28.279+0000 7f851cc00700 1 -- 172.21.15.60:0/3359446385 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f850800be40 con 0x7f852010a490 2024-05-03T14:46:28.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:28.279+0000 7f851cc00700 1 -- 172.21.15.60:0/3359446385 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8508017650 con 0x7f852010a490 2024-05-03T14:46:28.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:28.279+0000 7f8526800700 1 -- 172.21.15.60:0/3359446385 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85201aceb0 con 0x7f852010a490 2024-05-03T14:46:28.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:28.281+0000 7f8526800700 1 -- 172.21.15.60:0/3359446385 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f852006a190 con 0x7f852010a490 2024-05-03T14:46:28.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:28.283+0000 7f851cc00700 1 -- 172.21.15.60:0/3359446385 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f850802a070 con 0x7f852010a490 2024-05-03T14:46:28.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:28.285+0000 7f851cc00700 1 --2- 172.21.15.60:0/3359446385 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f85040b2990 0x7f85040b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:46:28.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:28.285+0000 7f851cc00700 1 -- 172.21.15.60:0/3359446385 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f85080e3680 con 0x7f852010a490 2024-05-03T14:46:28.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:28.285+0000 7f851f400700 1 --2- 172.21.15.60:0/3359446385 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f85040b2990 0x7f85040b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:46:28.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:28.286+0000 7f851f400700 1 --2- 172.21.15.60:0/3359446385 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f85040b2990 0x7f85040b4e50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8510005950 tx=0x7f851000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:46:28.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:28.289+0000 7f851cc00700 1 -- 172.21.15.60:0/3359446385 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f8508013070 con 0x7f852010a490 2024-05-03T14:46:28.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:28.528+0000 7f8526800700 1 -- 172.21.15.60:0/3359446385 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f852002e510 con 0x7f852010a490 2024-05-03T14:46:28.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:28.530+0000 7f851cc00700 1 -- 172.21.15.60:0/3359446385 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24674 (secure 0 0 0) 0x7f85080aaef0 con 0x7f852010a490 2024-05-03T14:46:28.534 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:46:28.534 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":89,"fsid":"285d64b2-095a-11ef-bc97-c7b262605968","created":"2024-05-03T14:34:53.560460+0000","modified":"2024-05-03T14:46:05.582395+0000","last_up_change":"2024-05-03T14:43:14.877043+0000","last_in_change":"2024-05-03T14:42:55.085959+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-03T14:38:54.333714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-03T14:44:25.608599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-03T14:44:30.154009+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-03T14:44:42.549502+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f0797730-c4c1-47ea-9dce-9270d5ebb4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2952930162},{"type":"v1","addr":"172.21.15.60:6803","nonce":2952930162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2952930162},{"type":"v1","addr":"172.21.15.60:6805","nonce":2952930162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":2952930162},{"type":"v1","addr":"172.21.15.60:6809","nonce":2952930162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2952930162},{"type":"v1","addr":"172.21.15.60:6807","nonce":2952930162}]},"public_addr":"172.21.15.60:6803/2952930162","cluster_addr":"172.21.15.60:6805/2952930162","heartbeat_back_addr":"172.21.15.60:6809/2952930162","heartbeat_front_addr":"172.21.15.60:6807/2952930162","state":["exists","up"]},{"osd":1,"uuid":"96d0ad08-cda4-4824-bad4-e64973508c0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":720063242},{"type":"v1","addr":"172.21.15.90:6801","nonce":720063242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":720063242},{"type":"v1","addr":"172.21.15.90:6803","nonce":720063242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":720063242},{"type":"v1","addr":"172.21.15.90:6807","nonce":720063242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":720063242},{"type":"v1","addr":"172.21.15.90:6805","nonce":720063242}]},"public_addr":"172.21.15.90:6801/720063242","cluster_addr":"172.21.15.90:6803/720063242","heartbeat_back_addr":"172.21.15.90:6807/720063242","heartbeat_front_addr":"172.21.15.90:6805/720063242","state":["exists","up"]},{"osd":2,"uuid":"ce95fb7c-1611-4dd1-aee7-97c420d6a466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1359662077},{"type":"v1","addr":"172.21.15.149:6801","nonce":1359662077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1359662077},{"type":"v1","addr":"172.21.15.149:6803","nonce":1359662077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1359662077},{"type":"v1","addr":"172.21.15.149:6807","nonce":1359662077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1359662077},{"type":"v1","addr":"172.21.15.149:6805","nonce":1359662077}]},"public_addr":"172.21.15.149:6801/1359662077","cluster_addr":"172.21.15.149:6803/1359662077","heartbeat_back_addr":"172.21.15.149:6807/1359662077","heartbeat_front_addr":"172.21.15.149:6805/1359662077","state":["exists","up"]},{"osd":3,"uuid":"ef31136a-9b36-4b87-8389-f53d311bc5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3226133334},{"type":"v1","addr":"172.21.15.60:6811","nonce":3226133334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3226133334},{"type":"v1","addr":"172.21.15.60:6813","nonce":3226133334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3226133334},{"type":"v1","addr":"172.21.15.60:6817","nonce":3226133334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3226133334},{"type":"v1","addr":"172.21.15.60:6815","nonce":3226133334}]},"public_addr":"172.21.15.60:6811/3226133334","cluster_addr":"172.21.15.60:6813/3226133334","heartbeat_back_addr":"172.21.15.60:6817/3226133334","heartbeat_front_addr":"172.21.15.60:6815/3226133334","state":["exists","up"]},{"osd":4,"uuid":"73db7408-a2ff-47e9-b13d-d31688a93610","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2201391883},{"type":"v1","addr":"172.21.15.90:6809","nonce":2201391883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":2201391883},{"type":"v1","addr":"172.21.15.90:6811","nonce":2201391883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":2201391883},{"type":"v1","addr":"172.21.15.90:6815","nonce":2201391883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":2201391883},{"type":"v1","addr":"172.21.15.90:6813","nonce":2201391883}]},"public_addr":"172.21.15.90:6809/2201391883","cluster_addr":"172.21.15.90:6811/2201391883","heartbeat_back_addr":"172.21.15.90:6815/2201391883","heartbeat_front_addr":"172.21.15.90:6813/2201391883","state":["exists","up"]},{"osd":5,"uuid":"aefb78a2-522a-4b35-92b8-e5c04fa47b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2139732836},{"type":"v1","addr":"172.21.15.149:6809","nonce":2139732836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":2139732836},{"type":"v1","addr":"172.21.15.149:6811","nonce":2139732836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":2139732836},{"type":"v1","addr":"172.21.15.149:6815","nonce":2139732836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":2139732836},{"type":"v1","addr":"172.21.15.149:6813","nonce":2139732836}]},"public_addr":"172.21.15.149:6809/2139732836","cluster_addr":"172.21.15.149:6811/2139732836","heartbeat_back_addr":"172.21.15.149:6815/2139732836","heartbeat_front_addr":"172.21.15.149:6813/2139732836","state":["exists","up"]},{"osd":6,"uuid":"7c887819-6182-4636-8cfd-d845c30bd13c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3738521002},{"type":"v1","addr":"172.21.15.60:6819","nonce":3738521002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3738521002},{"type":"v1","addr":"172.21.15.60:6821","nonce":3738521002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3738521002},{"type":"v1","addr":"172.21.15.60:6825","nonce":3738521002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3738521002},{"type":"v1","addr":"172.21.15.60:6823","nonce":3738521002}]},"public_addr":"172.21.15.60:6819/3738521002","cluster_addr":"172.21.15.60:6821/3738521002","heartbeat_back_addr":"172.21.15.60:6825/3738521002","heartbeat_front_addr":"172.21.15.60:6823/3738521002","state":["exists","up"]},{"osd":7,"uuid":"ebce6acf-8873-4b35-bd5f-80de2163a307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":1686298661},{"type":"v1","addr":"172.21.15.90:6817","nonce":1686298661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":1686298661},{"type":"v1","addr":"172.21.15.90:6819","nonce":1686298661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":1686298661},{"type":"v1","addr":"172.21.15.90:6823","nonce":1686298661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1686298661},{"type":"v1","addr":"172.21.15.90:6821","nonce":1686298661}]},"public_addr":"172.21.15.90:6817/1686298661","cluster_addr":"172.21.15.90:6819/1686298661","heartbeat_back_addr":"172.21.15.90:6823/1686298661","heartbeat_front_addr":"172.21.15.90:6821/1686298661","state":["exists","up"]},{"osd":8,"uuid":"4ae04b98-e9b2-4956-83d9-17e5ac3c963f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":2608348248},{"type":"v1","addr":"172.21.15.149:6817","nonce":2608348248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2608348248},{"type":"v1","addr":"172.21.15.149:6819","nonce":2608348248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2608348248},{"type":"v1","addr":"172.21.15.149:6823","nonce":2608348248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2608348248},{"type":"v1","addr":"172.21.15.149:6821","nonce":2608348248}]},"public_addr":"172.21.15.149:6817/2608348248","cluster_addr":"172.21.15.149:6819/2608348248","heartbeat_back_addr":"172.21.15.149:6823/2608348248","heartbeat_front_addr":"172.21.15.149:6821/2608348248","state":["exists","up"]},{"osd":9,"uuid":"d0c0a70d-aa76-47e4-91e8-c343c8c41197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":2156130999},{"type":"v1","addr":"172.21.15.60:6827","nonce":2156130999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":2156130999},{"type":"v1","addr":"172.21.15.60:6829","nonce":2156130999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":2156130999},{"type":"v1","addr":"172.21.15.60:6833","nonce":2156130999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":2156130999},{"type":"v1","addr":"172.21.15.60:6831","nonce":2156130999}]},"public_addr":"172.21.15.60:6827/2156130999","cluster_addr":"172.21.15.60:6829/2156130999","heartbeat_back_addr":"172.21.15.60:6833/2156130999","heartbeat_front_addr":"172.21.15.60:6831/2156130999","state":["exists","up"]},{"osd":10,"uuid":"cdc3d7b8-33c7-4cc4-be4e-6887d15a3d8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":2492593590},{"type":"v1","addr":"172.21.15.90:6825","nonce":2492593590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":2492593590},{"type":"v1","addr":"172.21.15.90:6827","nonce":2492593590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":2492593590},{"type":"v1","addr":"172.21.15.90:6831","nonce":2492593590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":2492593590},{"type":"v1","addr":"172.21.15.90:6829","nonce":2492593590}]},"public_addr":"172.21.15.90:6825/2492593590","cluster_addr":"172.21.15.90:6827/2492593590","heartbeat_back_addr":"172.21.15.90:6831/2492593590","heartbeat_front_addr":"172.21.15.90:6829/2492593590","state":["exists","up"]},{"osd":11,"uuid":"8d17ec1a-bdd5-41f1-9b59-c2b6a527bfbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":1063245126},{"type":"v1","addr":"172.21.15.149:6825","nonce":1063245126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":1063245126},{"type":"v1","addr":"172.21.15.149:6827","nonce":1063245126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":1063245126},{"type":"v1","addr":"172.21.15.149:6831","nonce":1063245126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":1063245126},{"type":"v1","addr":"172.21.15.149:6829","nonce":1063245126}]},"public_addr":"172.21.15.149:6825/1063245126","cluster_addr":"172.21.15.149:6827/1063245126","heartbeat_back_addr":"172.21.15.149:6831/1063245126","heartbeat_front_addr":"172.21.15.149:6829/1063245126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:38:02.056563+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:38:27.140677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:38:51.287070+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:39:25.484410+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:39:51.483133+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:40:17.200423+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:40:51.979804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:41:18.742542+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:41:44.922541+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:42:19.778451+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:42:47.438440+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:43:13.429419+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6801/424391417":"2024-05-04T14:36:04.154333+0000","172.21.15.60:6800/424391417":"2024-05-04T14:36:04.154333+0000","172.21.15.60:0/1426532561":"2024-05-04T14:36:04.154333+0000","172.21.15.60:0/3230079926":"2024-05-04T14:35:28.066961+0000","172.21.15.60:0/2334895610":"2024-05-04T14:35:28.066961+0000","172.21.15.60:6800/888447185":"2024-05-04T14:35:28.066961+0000","172.21.15.60:0/3996899937":"2024-05-04T14:35:28.066961+0000","172.21.15.60:0/2779644018":"2024-05-04T14:36:04.154333+0000","172.21.15.60:0/71440838":"2024-05-04T14:36:04.154333+0000","172.21.15.60:6801/888447185":"2024-05-04T14:35:28.066961+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-03T14:46:28.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:28.534+0000 7f8526800700 1 -- 172.21.15.60:0/3359446385 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f85040b2990 msgr2=0x7f85040b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:28.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:28.534+0000 7f8526800700 1 --2- 172.21.15.60:0/3359446385 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f85040b2990 0x7f85040b4e50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8510005950 tx=0x7f851000b410 comp rx=0 tx=0).stop 2024-05-03T14:46:28.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:28.534+0000 7f8526800700 1 -- 172.21.15.60:0/3359446385 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f852010a490 msgr2=0x7f85201a08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:28.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:28.534+0000 7f8526800700 1 --2- 172.21.15.60:0/3359446385 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f852010a490 0x7f85201a08c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8508005910 tx=0x7f85080047b0 comp rx=0 tx=0).stop 2024-05-03T14:46:28.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:28.534+0000 7f8526800700 1 -- 172.21.15.60:0/3359446385 shutdown_connections 2024-05-03T14:46:28.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:28.534+0000 7f8526800700 1 --2- 172.21.15.60:0/3359446385 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f85040b2990 0x7f85040b4e50 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:28.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:28.534+0000 7f8526800700 1 --2- 172.21.15.60:0/3359446385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f852010fb60 0x7f85201a4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:28.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:28.534+0000 7f8526800700 1 --2- 172.21.15.60:0/3359446385 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8520109ad0 0x7f85201a0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:28.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:28.534+0000 7f8526800700 1 --2- 172.21.15.60:0/3359446385 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f852010a490 0x7f85201a08c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:28.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:28.534+0000 7f8526800700 1 -- 172.21.15.60:0/3359446385 >> 172.21.15.60:0/3359446385 conn(0x7f8520076c70 msgr2=0x7f8520077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:46:28.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:28.534+0000 7f8526800700 1 -- 172.21.15.60:0/3359446385 shutdown_connections 2024-05-03T14:46:28.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:28.534+0000 7f8526800700 1 -- 172.21.15.60:0/3359446385 wait complete. 2024-05-03T14:46:28.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:28 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/3359446385' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T14:46:29.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:28 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/3359446385' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T14:46:29.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:28 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/3359446385' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T14:46:29.457 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-03T14:46:29.457 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-03T14:46:29.950 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:46:30.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:29 smithi060 ceph-mon[28873]: pgmap v391: 129 pgs: 129 active+clean; 634 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.4 KiB/s wr, 10 op/s 2024-05-03T14:46:30.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:29 smithi090 ceph-mon[33568]: pgmap v391: 129 pgs: 129 active+clean; 634 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.4 KiB/s wr, 10 op/s 2024-05-03T14:46:30.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:29 smithi149 ceph-mon[33332]: pgmap v391: 129 pgs: 129 active+clean; 634 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.4 KiB/s wr, 10 op/s 2024-05-03T14:46:31.983 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:31 smithi060 ceph-mon[28873]: pgmap v392: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 3.0 KiB/s wr, 13 op/s 2024-05-03T14:46:31.983 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:31 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:46:32.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:31 smithi090 ceph-mon[33568]: pgmap v392: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 3.0 KiB/s wr, 13 op/s 2024-05-03T14:46:32.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:31 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:46:32.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:32.142+0000 7fc13b200700 1 -- 172.21.15.60:0/4027244622 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc1340756d0 msgr2=0x7fc134075ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:32.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:32.142+0000 7fc13b200700 1 --2- 172.21.15.60:0/4027244622 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc1340756d0 0x7fc134075ab0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc124009990 tx=0x7fc124009ca0 comp rx=0 tx=0).stop 2024-05-03T14:46:32.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:32.142+0000 7fc13b200700 1 -- 172.21.15.60:0/4027244622 shutdown_connections 2024-05-03T14:46:32.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:32.142+0000 7fc13b200700 1 --2- 172.21.15.60:0/4027244622 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc13410ff00 0x7fc1341122f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:32.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:32.142+0000 7fc13b200700 1 --2- 172.21.15.60:0/4027244622 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc1340756d0 0x7fc134075ab0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:32.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:32.142+0000 7fc13b200700 1 --2- 172.21.15.60:0/4027244622 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc134075ff0 0x7fc13410f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:32.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:32.143+0000 7fc13b200700 1 -- 172.21.15.60:0/4027244622 >> 172.21.15.60:0/4027244622 conn(0x7fc13407a600 msgr2=0x7fc13407aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:46:32.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:32.143+0000 7fc13b200700 1 -- 172.21.15.60:0/4027244622 shutdown_connections 2024-05-03T14:46:32.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:32.143+0000 7fc13b200700 1 -- 172.21.15.60:0/4027244622 wait complete. 2024-05-03T14:46:32.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:32.144+0000 7fc13b200700 1 Processor -- start 2024-05-03T14:46:32.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:32.144+0000 7fc13b200700 1 -- start start 2024-05-03T14:46:32.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:32.144+0000 7fc13b200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc1340756d0 0x7fc13419bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:46:32.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:32.144+0000 7fc13b200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc134075ff0 0x7fc13419c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:46:32.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:32.144+0000 7fc13b200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc13410ff00 0x7fc1341a07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:46:32.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:32.144+0000 7fc13b200700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc134115bb0 con 0x7fc134075ff0 2024-05-03T14:46:32.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:32.144+0000 7fc13b200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc134115a30 con 0x7fc13410ff00 2024-05-03T14:46:32.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:32.144+0000 7fc13b200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc134115d30 con 0x7fc1340756d0 2024-05-03T14:46:32.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:32.145+0000 7fc133e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc1340756d0 0x7fc13419bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:46:32.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:32.145+0000 7fc133e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc1340756d0 0x7fc13419bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.60:37936/0 (socket says 172.21.15.60:37936) 2024-05-03T14:46:32.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:32.145+0000 7fc133e00700 1 -- 172.21.15.60:0/1879665689 learned_addr learned my addr 172.21.15.60:0/1879665689 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:46:32.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:32.145+0000 7fc139200700 1 --2- 172.21.15.60:0/1879665689 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc13410ff00 0x7fc1341a07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:46:32.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:32.145+0000 7fc133400700 1 --2- 172.21.15.60:0/1879665689 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc134075ff0 0x7fc13419c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:46:32.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:32.145+0000 7fc133e00700 1 -- 172.21.15.60:0/1879665689 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc13410ff00 msgr2=0x7fc1341a07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:32.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:32.145+0000 7fc133e00700 1 --2- 172.21.15.60:0/1879665689 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc13410ff00 0x7fc1341a07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:32.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:32.145+0000 7fc133e00700 1 -- 172.21.15.60:0/1879665689 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc134075ff0 msgr2=0x7fc13419c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:32.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:32.145+0000 7fc133e00700 1 --2- 172.21.15.60:0/1879665689 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc134075ff0 0x7fc13419c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:32.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:32.145+0000 7fc133e00700 1 -- 172.21.15.60:0/1879665689 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1341a0ef0 con 0x7fc1340756d0 2024-05-03T14:46:32.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:32.145+0000 7fc133e00700 1 --2- 172.21.15.60:0/1879665689 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc1340756d0 0x7fc13419bee0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc124000c00 tx=0x7fc124005590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:46:32.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:32.146+0000 7fc130c00700 1 -- 172.21.15.60:0/1879665689 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc12400f8c0 con 0x7fc1340756d0 2024-05-03T14:46:32.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:32.146+0000 7fc130c00700 1 -- 172.21.15.60:0/1879665689 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7fc124022470 con 0x7fc1340756d0 2024-05-03T14:46:32.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:32.146+0000 7fc13b200700 1 -- 172.21.15.60:0/1879665689 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1341a1180 con 0x7fc1340756d0 2024-05-03T14:46:32.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:32.146+0000 7fc130c00700 1 -- 172.21.15.60:0/1879665689 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc124017630 con 0x7fc1340756d0 2024-05-03T14:46:32.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:32.146+0000 7fc13b200700 1 -- 172.21.15.60:0/1879665689 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1341a8a60 con 0x7fc1340756d0 2024-05-03T14:46:32.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:32.147+0000 7fc13b200700 1 -- 172.21.15.60:0/1879665689 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc13406a190 con 0x7fc1340756d0 2024-05-03T14:46:32.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:32.149+0000 7fc130c00700 1 -- 172.21.15.60:0/1879665689 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fc124009d10 con 0x7fc1340756d0 2024-05-03T14:46:32.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:32.150+0000 7fc130c00700 1 --2- 172.21.15.60:0/1879665689 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fc1180b2910 0x7fc1180b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:46:32.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:32.151+0000 7fc130c00700 1 -- 172.21.15.60:0/1879665689 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7fc1240e3640 con 0x7fc1340756d0 2024-05-03T14:46:32.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:32.151+0000 7fc133400700 1 --2- 172.21.15.60:0/1879665689 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fc1180b2910 0x7fc1180b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:46:32.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:32.152+0000 7fc133400700 1 --2- 172.21.15.60:0/1879665689 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fc1180b2910 0x7fc1180b4dd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc134111900 tx=0x7fc120009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:46:32.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:32.155+0000 7fc130c00700 1 -- 172.21.15.60:0/1879665689 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fc124013070 con 0x7fc1340756d0 2024-05-03T14:46:32.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:31 smithi149 ceph-mon[33332]: pgmap v392: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 3.0 KiB/s wr, 13 op/s 2024-05-03T14:46:32.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:31 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:46:32.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:32.394+0000 7fc13b200700 1 -- 172.21.15.60:0/1879665689 --> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc134100ca0 con 0x7fc1180b2910 2024-05-03T14:46:32.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:32.411+0000 7fc130c00700 1 -- 172.21.15.60:0/1879665689 <== mgr.14152 v2:172.21.15.60:6800/3066142956 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340675 (secure 0 0 0) 0x7fc134100ca0 con 0x7fc1180b2910 2024-05-03T14:46:32.416 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:46:32.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:32.420+0000 7fc13b200700 1 -- 172.21.15.60:0/1879665689 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fc1180b2910 msgr2=0x7fc1180b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:32.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:32.420+0000 7fc13b200700 1 --2- 172.21.15.60:0/1879665689 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fc1180b2910 0x7fc1180b4dd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc134111900 tx=0x7fc120009260 comp rx=0 tx=0).stop 2024-05-03T14:46:32.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:32.420+0000 7fc13b200700 1 -- 172.21.15.60:0/1879665689 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc1340756d0 msgr2=0x7fc13419bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:32.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:32.420+0000 7fc13b200700 1 --2- 172.21.15.60:0/1879665689 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc1340756d0 0x7fc13419bee0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc124000c00 tx=0x7fc124005590 comp rx=0 tx=0).stop 2024-05-03T14:46:32.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:32.420+0000 7fc13b200700 1 -- 172.21.15.60:0/1879665689 shutdown_connections 2024-05-03T14:46:32.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:32.420+0000 7fc13b200700 1 --2- 172.21.15.60:0/1879665689 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fc1180b2910 0x7fc1180b4dd0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:32.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:32.420+0000 7fc13b200700 1 --2- 172.21.15.60:0/1879665689 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc13410ff00 0x7fc1341a07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:32.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:32.420+0000 7fc13b200700 1 --2- 172.21.15.60:0/1879665689 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc1340756d0 0x7fc13419bee0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:32.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:32.420+0000 7fc13b200700 1 --2- 172.21.15.60:0/1879665689 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc134075ff0 0x7fc13419c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:32.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:32.420+0000 7fc13b200700 1 -- 172.21.15.60:0/1879665689 >> 172.21.15.60:0/1879665689 conn(0x7fc13407a600 msgr2=0x7fc134100590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:46:32.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:32.420+0000 7fc13b200700 1 -- 172.21.15.60:0/1879665689 shutdown_connections 2024-05-03T14:46:32.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:32.420+0000 7fc13b200700 1 -- 172.21.15.60:0/1879665689 wait complete. 2024-05-03T14:46:32.426 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-05-03T14:46:33.278 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":393,"stamp":"2024-05-03T14:46:32.364328+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655940,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":437,"num_read_kb":316,"num_write":369,"num_write_kb":2288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":2952512,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":305,"ondisk_log_size":305,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361264,"kb_used_data":12104,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580560,"statfs":{"total":1151940427776,"available":1151570493440,"internally_reserved":0,"allocated":12394496,"data_stored":8122874,"data_compressed":496272,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19140,"internal_metadata":357414204},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":11051,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":107,"num_read_kb":62,"num_write":11,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005613"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110356+0000","last_change":"2024-05-03T14:44:31.452444+0000","last_active":"2024-05-03T14:46:06.110356+0000","last_peered":"2024-05-03T14:46:06.110356+0000","last_clean":"2024-05-03T14:46:06.110356+0000","last_became_active":"2024-05-03T14:44:31.451574+0000","last_became_peered":"2024-05-03T14:44:31.451574+0000","last_unstale":"2024-05-03T14:46:06.110356+0000","last_undegraded":"2024-05-03T14:46:06.110356+0000","last_fullsized":"2024-05-03T14:46:06.110356+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:14:31.009833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.590338+0000","last_change":"2024-05-03T14:44:31.461736+0000","last_active":"2024-05-03T14:46:05.590338+0000","last_peered":"2024-05-03T14:46:05.590338+0000","last_clean":"2024-05-03T14:46:05.590338+0000","last_became_active":"2024-05-03T14:44:31.457963+0000","last_became_peered":"2024-05-03T14:44:31.457963+0000","last_unstale":"2024-05-03T14:46:05.590338+0000","last_undegraded":"2024-05-03T14:46:05.590338+0000","last_fullsized":"2024-05-03T14:46:05.590338+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:59:16.029675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.589959+0000","last_change":"2024-05-03T14:44:31.457163+0000","last_active":"2024-05-03T14:46:05.589959+0000","last_peered":"2024-05-03T14:46:05.589959+0000","last_clean":"2024-05-03T14:46:05.589959+0000","last_became_active":"2024-05-03T14:44:31.455564+0000","last_became_peered":"2024-05-03T14:44:31.455564+0000","last_unstale":"2024-05-03T14:46:05.589959+0000","last_undegraded":"2024-05-03T14:46:05.589959+0000","last_fullsized":"2024-05-03T14:46:05.589959+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T16:14:00.761333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109829+0000","last_change":"2024-05-03T14:44:31.450623+0000","last_active":"2024-05-03T14:46:06.109829+0000","last_peered":"2024-05-03T14:46:06.109829+0000","last_clean":"2024-05-03T14:46:06.109829+0000","last_became_active":"2024-05-03T14:44:31.449944+0000","last_became_peered":"2024-05-03T14:44:31.449944+0000","last_unstale":"2024-05-03T14:46:06.109829+0000","last_undegraded":"2024-05-03T14:46:06.109829+0000","last_fullsized":"2024-05-03T14:46:06.109829+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:40:24.155342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.593903+0000","last_change":"2024-05-03T14:44:31.473624+0000","last_active":"2024-05-03T14:46:05.593903+0000","last_peered":"2024-05-03T14:46:05.593903+0000","last_clean":"2024-05-03T14:46:05.593903+0000","last_became_active":"2024-05-03T14:44:31.473339+0000","last_became_peered":"2024-05-03T14:44:31.473339+0000","last_unstale":"2024-05-03T14:46:05.593903+0000","last_undegraded":"2024-05-03T14:46:05.593903+0000","last_fullsized":"2024-05-03T14:46:05.593903+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:21:42.683546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.608065+0000","last_change":"2024-05-03T14:44:31.466037+0000","last_active":"2024-05-03T14:46:05.608065+0000","last_peered":"2024-05-03T14:46:05.608065+0000","last_clean":"2024-05-03T14:46:05.608065+0000","last_became_active":"2024-05-03T14:44:31.465202+0000","last_became_peered":"2024-05-03T14:44:31.465202+0000","last_unstale":"2024-05-03T14:46:05.608065+0000","last_undegraded":"2024-05-03T14:46:05.608065+0000","last_fullsized":"2024-05-03T14:46:05.608065+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:46:04.189133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.594232+0000","last_change":"2024-05-03T14:44:31.458708+0000","last_active":"2024-05-03T14:46:05.594232+0000","last_peered":"2024-05-03T14:46:05.594232+0000","last_clean":"2024-05-03T14:46:05.594232+0000","last_became_active":"2024-05-03T14:44:31.458292+0000","last_became_peered":"2024-05-03T14:44:31.458292+0000","last_unstale":"2024-05-03T14:46:05.594232+0000","last_undegraded":"2024-05-03T14:46:05.594232+0000","last_fullsized":"2024-05-03T14:46:05.594232+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:15:28.850661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.113279+0000","last_change":"2024-05-03T14:44:31.466051+0000","last_active":"2024-05-03T14:46:06.113279+0000","last_peered":"2024-05-03T14:46:06.113279+0000","last_clean":"2024-05-03T14:46:06.113279+0000","last_became_active":"2024-05-03T14:44:31.465620+0000","last_became_peered":"2024-05-03T14:44:31.465620+0000","last_unstale":"2024-05-03T14:46:06.113279+0000","last_undegraded":"2024-05-03T14:46:06.113279+0000","last_fullsized":"2024-05-03T14:46:06.113279+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:27:02.749377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109419+0000","last_change":"2024-05-03T14:44:31.451507+0000","last_active":"2024-05-03T14:46:06.109419+0000","last_peered":"2024-05-03T14:46:06.109419+0000","last_clean":"2024-05-03T14:46:06.109419+0000","last_became_active":"2024-05-03T14:44:31.451163+0000","last_became_peered":"2024-05-03T14:44:31.451163+0000","last_unstale":"2024-05-03T14:46:06.109419+0000","last_undegraded":"2024-05-03T14:46:06.109419+0000","last_fullsized":"2024-05-03T14:46:06.109419+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:56:50.415619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.608433+0000","last_change":"2024-05-03T14:45:05.421817+0000","last_active":"2024-05-03T14:46:05.608433+0000","last_peered":"2024-05-03T14:46:05.608433+0000","last_clean":"2024-05-03T14:46:05.608433+0000","last_became_active":"2024-05-03T14:45:05.421446+0000","last_became_peered":"2024-05-03T14:45:05.421446+0000","last_unstale":"2024-05-03T14:46:05.608433+0000","last_undegraded":"2024-05-03T14:46:05.608433+0000","last_fullsized":"2024-05-03T14:46:05.608433+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:31:36.867429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.594745+0000","last_change":"2024-05-03T14:44:31.454474+0000","last_active":"2024-05-03T14:46:05.594745+0000","last_peered":"2024-05-03T14:46:05.594745+0000","last_clean":"2024-05-03T14:46:05.594745+0000","last_became_active":"2024-05-03T14:44:31.454089+0000","last_became_peered":"2024-05-03T14:44:31.454089+0000","last_unstale":"2024-05-03T14:46:05.594745+0000","last_undegraded":"2024-05-03T14:46:05.594745+0000","last_fullsized":"2024-05-03T14:46:05.594745+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:54:59.739325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.592878+0000","last_change":"2024-05-03T14:45:05.617724+0000","last_active":"2024-05-03T14:46:05.592878+0000","last_peered":"2024-05-03T14:46:05.592878+0000","last_clean":"2024-05-03T14:46:05.592878+0000","last_became_active":"2024-05-03T14:45:05.616554+0000","last_became_peered":"2024-05-03T14:45:05.616554+0000","last_unstale":"2024-05-03T14:46:05.592878+0000","last_undegraded":"2024-05-03T14:46:05.592878+0000","last_fullsized":"2024-05-03T14:46:05.592878+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:24:20.890688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110600+0000","last_change":"2024-05-03T14:44:31.462676+0000","last_active":"2024-05-03T14:46:06.110600+0000","last_peered":"2024-05-03T14:46:06.110600+0000","last_clean":"2024-05-03T14:46:06.110600+0000","last_became_active":"2024-05-03T14:44:31.459326+0000","last_became_peered":"2024-05-03T14:44:31.459326+0000","last_unstale":"2024-05-03T14:46:06.110600+0000","last_undegraded":"2024-05-03T14:46:06.110600+0000","last_fullsized":"2024-05-03T14:46:06.110600+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:36:22.975112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.589702+0000","last_change":"2024-05-03T14:44:31.447665+0000","last_active":"2024-05-03T14:46:05.589702+0000","last_peered":"2024-05-03T14:46:05.589702+0000","last_clean":"2024-05-03T14:46:05.589702+0000","last_became_active":"2024-05-03T14:44:31.447204+0000","last_became_peered":"2024-05-03T14:44:31.447204+0000","last_unstale":"2024-05-03T14:46:05.589702+0000","last_undegraded":"2024-05-03T14:46:05.589702+0000","last_fullsized":"2024-05-03T14:46:05.589702+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:23:41.313986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.590107+0000","last_change":"2024-05-03T14:44:31.458869+0000","last_active":"2024-05-03T14:46:05.590107+0000","last_peered":"2024-05-03T14:46:05.590107+0000","last_clean":"2024-05-03T14:46:05.590107+0000","last_became_active":"2024-05-03T14:44:31.453157+0000","last_became_peered":"2024-05-03T14:44:31.453157+0000","last_unstale":"2024-05-03T14:46:05.590107+0000","last_undegraded":"2024-05-03T14:46:05.590107+0000","last_fullsized":"2024-05-03T14:46:05.590107+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:26:15.764522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.108515+0000","last_change":"2024-05-03T14:44:31.461320+0000","last_active":"2024-05-03T14:46:06.108515+0000","last_peered":"2024-05-03T14:46:06.108515+0000","last_clean":"2024-05-03T14:46:06.108515+0000","last_became_active":"2024-05-03T14:44:31.460935+0000","last_became_peered":"2024-05-03T14:44:31.460935+0000","last_unstale":"2024-05-03T14:46:06.108515+0000","last_undegraded":"2024-05-03T14:46:06.108515+0000","last_fullsized":"2024-05-03T14:46:06.108515+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:32:22.368935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110099+0000","last_change":"2024-05-03T14:44:31.452694+0000","last_active":"2024-05-03T14:46:06.110099+0000","last_peered":"2024-05-03T14:46:06.110099+0000","last_clean":"2024-05-03T14:46:06.110099+0000","last_became_active":"2024-05-03T14:44:31.452012+0000","last_became_peered":"2024-05-03T14:44:31.452012+0000","last_unstale":"2024-05-03T14:46:06.110099+0000","last_undegraded":"2024-05-03T14:46:06.110099+0000","last_fullsized":"2024-05-03T14:46:06.110099+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:59:34.721990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.591501+0000","last_change":"2024-05-03T14:44:31.461029+0000","last_active":"2024-05-03T14:46:05.591501+0000","last_peered":"2024-05-03T14:46:05.591501+0000","last_clean":"2024-05-03T14:46:05.591501+0000","last_became_active":"2024-05-03T14:44:31.460506+0000","last_became_peered":"2024-05-03T14:44:31.460506+0000","last_unstale":"2024-05-03T14:46:05.591501+0000","last_undegraded":"2024-05-03T14:46:05.591501+0000","last_fullsized":"2024-05-03T14:46:05.591501+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:09:23.903606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.591580+0000","last_change":"2024-05-03T14:44:31.467556+0000","last_active":"2024-05-03T14:46:05.591580+0000","last_peered":"2024-05-03T14:46:05.591580+0000","last_clean":"2024-05-03T14:46:05.591580+0000","last_became_active":"2024-05-03T14:44:31.467191+0000","last_became_peered":"2024-05-03T14:44:31.467191+0000","last_unstale":"2024-05-03T14:46:05.591580+0000","last_undegraded":"2024-05-03T14:46:05.591580+0000","last_fullsized":"2024-05-03T14:46:05.591580+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:33:42.980620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.591049+0000","last_change":"2024-05-03T14:45:05.618172+0000","last_active":"2024-05-03T14:46:05.591049+0000","last_peered":"2024-05-03T14:46:05.591049+0000","last_clean":"2024-05-03T14:46:05.591049+0000","last_became_active":"2024-05-03T14:45:05.617555+0000","last_became_peered":"2024-05-03T14:45:05.617555+0000","last_unstale":"2024-05-03T14:46:05.591049+0000","last_undegraded":"2024-05-03T14:46:05.591049+0000","last_fullsized":"2024-05-03T14:46:05.591049+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T16:44:40.610265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'13","reported_seq":108,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:26.160366+0000","last_change":"2024-05-03T14:46:06.129243+0000","last_active":"2024-05-03T14:46:26.160366+0000","last_peered":"2024-05-03T14:46:26.160366+0000","last_clean":"2024-05-03T14:46:26.160366+0000","last_became_active":"2024-05-03T14:46:06.112952+0000","last_became_peered":"2024-05-03T14:46:06.112952+0000","last_unstale":"2024-05-03T14:46:26.160366+0000","last_undegraded":"2024-05-03T14:46:26.160366+0000","last_fullsized":"2024-05-03T14:46:26.160366+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:12:22.401171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.113554+0000","last_change":"2024-05-03T14:44:31.467383+0000","last_active":"2024-05-03T14:46:06.113554+0000","last_peered":"2024-05-03T14:46:06.113554+0000","last_clean":"2024-05-03T14:46:06.113554+0000","last_became_active":"2024-05-03T14:44:31.466993+0000","last_became_peered":"2024-05-03T14:44:31.466993+0000","last_unstale":"2024-05-03T14:46:06.113554+0000","last_undegraded":"2024-05-03T14:46:06.113554+0000","last_fullsized":"2024-05-03T14:46:06.113554+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T16:43:50.237083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.108877+0000","last_change":"2024-05-03T14:44:44.583870+0000","last_active":"2024-05-03T14:46:06.108877+0000","last_peered":"2024-05-03T14:46:06.108877+0000","last_clean":"2024-05-03T14:46:06.108877+0000","last_became_active":"2024-05-03T14:44:44.583296+0000","last_became_peered":"2024-05-03T14:44:44.583296+0000","last_unstale":"2024-05-03T14:46:06.108877+0000","last_undegraded":"2024-05-03T14:46:06.108877+0000","last_fullsized":"2024-05-03T14:46:06.108877+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:55:54.345707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.115090+0000","last_change":"2024-05-03T14:44:27.415797+0000","last_active":"2024-05-03T14:46:06.115090+0000","last_peered":"2024-05-03T14:46:06.115090+0000","last_clean":"2024-05-03T14:46:06.115090+0000","last_became_active":"2024-05-03T14:44:27.414960+0000","last_became_peered":"2024-05-03T14:44:27.414960+0000","last_unstale":"2024-05-03T14:46:06.115090+0000","last_undegraded":"2024-05-03T14:46:06.115090+0000","last_fullsized":"2024-05-03T14:46:06.115090+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:15:54.374392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.590908+0000","last_change":"2024-05-03T14:44:31.470241+0000","last_active":"2024-05-03T14:46:05.590908+0000","last_peered":"2024-05-03T14:46:05.590908+0000","last_clean":"2024-05-03T14:46:05.590908+0000","last_became_active":"2024-05-03T14:44:31.469882+0000","last_became_peered":"2024-05-03T14:44:31.469882+0000","last_unstale":"2024-05-03T14:46:05.590908+0000","last_undegraded":"2024-05-03T14:46:05.590908+0000","last_fullsized":"2024-05-03T14:46:05.590908+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:40:43.970897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109037+0000","last_change":"2024-05-03T14:44:44.583776+0000","last_active":"2024-05-03T14:46:06.109037+0000","last_peered":"2024-05-03T14:46:06.109037+0000","last_clean":"2024-05-03T14:46:06.109037+0000","last_became_active":"2024-05-03T14:44:44.583117+0000","last_became_peered":"2024-05-03T14:44:44.583117+0000","last_unstale":"2024-05-03T14:46:06.109037+0000","last_undegraded":"2024-05-03T14:46:06.109037+0000","last_fullsized":"2024-05-03T14:46:06.109037+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:58:49.428720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":57,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.594170+0000","last_change":"2024-05-03T14:44:27.404806+0000","last_active":"2024-05-03T14:46:05.594170+0000","last_peered":"2024-05-03T14:46:05.594170+0000","last_clean":"2024-05-03T14:46:05.594170+0000","last_became_active":"2024-05-03T14:44:27.404415+0000","last_became_peered":"2024-05-03T14:44:27.404415+0000","last_unstale":"2024-05-03T14:46:05.594170+0000","last_undegraded":"2024-05-03T14:46:05.594170+0000","last_fullsized":"2024-05-03T14:46:05.594170+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:04:40.569591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.108822+0000","last_change":"2024-05-03T14:44:31.465714+0000","last_active":"2024-05-03T14:46:06.108822+0000","last_peered":"2024-05-03T14:46:06.108822+0000","last_clean":"2024-05-03T14:46:06.108822+0000","last_became_active":"2024-05-03T14:44:31.465264+0000","last_became_peered":"2024-05-03T14:44:31.465264+0000","last_unstale":"2024-05-03T14:46:06.108822+0000","last_undegraded":"2024-05-03T14:46:06.108822+0000","last_fullsized":"2024-05-03T14:46:06.108822+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:19:38.206564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109120+0000","last_change":"2024-05-03T14:44:44.583611+0000","last_active":"2024-05-03T14:46:06.109120+0000","last_peered":"2024-05-03T14:46:06.109120+0000","last_clean":"2024-05-03T14:46:06.109120+0000","last_became_active":"2024-05-03T14:44:44.583149+0000","last_became_peered":"2024-05-03T14:44:44.583149+0000","last_unstale":"2024-05-03T14:46:06.109120+0000","last_undegraded":"2024-05-03T14:46:06.109120+0000","last_fullsized":"2024-05-03T14:46:06.109120+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:25:23.550168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.593940+0000","last_change":"2024-05-03T14:44:27.405942+0000","last_active":"2024-05-03T14:46:05.593940+0000","last_peered":"2024-05-03T14:46:05.593940+0000","last_clean":"2024-05-03T14:46:05.593940+0000","last_became_active":"2024-05-03T14:44:27.405556+0000","last_became_peered":"2024-05-03T14:44:27.405556+0000","last_unstale":"2024-05-03T14:46:05.593940+0000","last_undegraded":"2024-05-03T14:46:05.593940+0000","last_fullsized":"2024-05-03T14:46:05.593940+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:16:07.798364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110278+0000","last_change":"2024-05-03T14:44:31.451421+0000","last_active":"2024-05-03T14:46:06.110278+0000","last_peered":"2024-05-03T14:46:06.110278+0000","last_clean":"2024-05-03T14:46:06.110278+0000","last_became_active":"2024-05-03T14:44:31.451072+0000","last_became_peered":"2024-05-03T14:44:31.451072+0000","last_unstale":"2024-05-03T14:46:06.110278+0000","last_undegraded":"2024-05-03T14:46:06.110278+0000","last_fullsized":"2024-05-03T14:46:06.110278+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:12:31.040518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.591370+0000","last_change":"2024-05-03T14:44:44.567926+0000","last_active":"2024-05-03T14:46:05.591370+0000","last_peered":"2024-05-03T14:46:05.591370+0000","last_clean":"2024-05-03T14:46:05.591370+0000","last_became_active":"2024-05-03T14:44:44.567494+0000","last_became_peered":"2024-05-03T14:44:44.567494+0000","last_unstale":"2024-05-03T14:46:05.591370+0000","last_undegraded":"2024-05-03T14:46:05.591370+0000","last_fullsized":"2024-05-03T14:46:05.591370+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:51:11.144925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'5","reported_seq":89,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:27.590344+0000","last_change":"2024-05-03T14:45:05.614704+0000","last_active":"2024-05-03T14:46:27.590344+0000","last_peered":"2024-05-03T14:46:27.590344+0000","last_clean":"2024-05-03T14:46:27.590344+0000","last_became_active":"2024-05-03T14:45:05.614173+0000","last_became_peered":"2024-05-03T14:45:05.614173+0000","last_unstale":"2024-05-03T14:46:27.590344+0000","last_undegraded":"2024-05-03T14:46:27.590344+0000","last_fullsized":"2024-05-03T14:46:27.590344+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:50:42.956239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.593674+0000","last_change":"2024-05-03T14:44:31.464666+0000","last_active":"2024-05-03T14:46:05.593674+0000","last_peered":"2024-05-03T14:46:05.593674+0000","last_clean":"2024-05-03T14:46:05.593674+0000","last_became_active":"2024-05-03T14:44:31.464298+0000","last_became_peered":"2024-05-03T14:44:31.464298+0000","last_unstale":"2024-05-03T14:46:05.593674+0000","last_undegraded":"2024-05-03T14:46:05.593674+0000","last_fullsized":"2024-05-03T14:46:05.593674+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:26:35.425908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.591268+0000","last_change":"2024-05-03T14:44:44.580268+0000","last_active":"2024-05-03T14:46:05.591268+0000","last_peered":"2024-05-03T14:46:05.591268+0000","last_clean":"2024-05-03T14:46:05.591268+0000","last_became_active":"2024-05-03T14:44:44.579617+0000","last_became_peered":"2024-05-03T14:44:44.579617+0000","last_unstale":"2024-05-03T14:46:05.591268+0000","last_undegraded":"2024-05-03T14:46:05.591268+0000","last_fullsized":"2024-05-03T14:46:05.591268+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:16:25.580698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.608061+0000","last_change":"2024-05-03T14:44:27.418578+0000","last_active":"2024-05-03T14:46:05.608061+0000","last_peered":"2024-05-03T14:46:05.608061+0000","last_clean":"2024-05-03T14:46:05.608061+0000","last_became_active":"2024-05-03T14:44:27.418206+0000","last_became_peered":"2024-05-03T14:44:27.418206+0000","last_unstale":"2024-05-03T14:46:05.608061+0000","last_undegraded":"2024-05-03T14:46:05.608061+0000","last_fullsized":"2024-05-03T14:46:05.608061+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:48:47.085451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110188+0000","last_change":"2024-05-03T14:44:31.467353+0000","last_active":"2024-05-03T14:46:06.110188+0000","last_peered":"2024-05-03T14:46:06.110188+0000","last_clean":"2024-05-03T14:46:06.110188+0000","last_became_active":"2024-05-03T14:44:31.466606+0000","last_became_peered":"2024-05-03T14:44:31.466606+0000","last_unstale":"2024-05-03T14:46:06.110188+0000","last_undegraded":"2024-05-03T14:46:06.110188+0000","last_fullsized":"2024-05-03T14:46:06.110188+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T14:50:22.624015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109972+0000","last_change":"2024-05-03T14:44:44.567367+0000","last_active":"2024-05-03T14:46:06.109972+0000","last_peered":"2024-05-03T14:46:06.109972+0000","last_clean":"2024-05-03T14:46:06.109972+0000","last_became_active":"2024-05-03T14:44:44.566916+0000","last_became_peered":"2024-05-03T14:44:44.566916+0000","last_unstale":"2024-05-03T14:46:06.109972+0000","last_undegraded":"2024-05-03T14:46:06.109972+0000","last_fullsized":"2024-05-03T14:46:06.109972+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:07:39.966644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110169+0000","last_change":"2024-05-03T14:44:27.424704+0000","last_active":"2024-05-03T14:46:06.110169+0000","last_peered":"2024-05-03T14:46:06.110169+0000","last_clean":"2024-05-03T14:46:06.110169+0000","last_became_active":"2024-05-03T14:44:27.424231+0000","last_became_peered":"2024-05-03T14:44:27.424231+0000","last_unstale":"2024-05-03T14:46:06.110169+0000","last_undegraded":"2024-05-03T14:46:06.110169+0000","last_fullsized":"2024-05-03T14:46:06.110169+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:00:22.076458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.113124+0000","last_change":"2024-05-03T14:44:31.474599+0000","last_active":"2024-05-03T14:46:06.113124+0000","last_peered":"2024-05-03T14:46:06.113124+0000","last_clean":"2024-05-03T14:46:06.113124+0000","last_became_active":"2024-05-03T14:44:31.473970+0000","last_became_peered":"2024-05-03T14:44:31.473970+0000","last_unstale":"2024-05-03T14:46:06.113124+0000","last_undegraded":"2024-05-03T14:46:06.113124+0000","last_fullsized":"2024-05-03T14:46:06.113124+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:20:14.259534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.590230+0000","last_change":"2024-05-03T14:44:44.580126+0000","last_active":"2024-05-03T14:46:05.590230+0000","last_peered":"2024-05-03T14:46:05.590230+0000","last_clean":"2024-05-03T14:46:05.590230+0000","last_became_active":"2024-05-03T14:44:44.578708+0000","last_became_peered":"2024-05-03T14:44:44.578708+0000","last_unstale":"2024-05-03T14:46:05.590230+0000","last_undegraded":"2024-05-03T14:46:05.590230+0000","last_fullsized":"2024-05-03T14:46:05.590230+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:42:43.279786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109173+0000","last_change":"2024-05-03T14:44:27.405332+0000","last_active":"2024-05-03T14:46:06.109173+0000","last_peered":"2024-05-03T14:46:06.109173+0000","last_clean":"2024-05-03T14:46:06.109173+0000","last_became_active":"2024-05-03T14:44:27.404994+0000","last_became_peered":"2024-05-03T14:44:27.404994+0000","last_unstale":"2024-05-03T14:46:06.109173+0000","last_undegraded":"2024-05-03T14:46:06.109173+0000","last_fullsized":"2024-05-03T14:46:06.109173+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:18:23.716014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.590783+0000","last_change":"2024-05-03T14:44:31.470396+0000","last_active":"2024-05-03T14:46:05.590783+0000","last_peered":"2024-05-03T14:46:05.590783+0000","last_clean":"2024-05-03T14:46:05.590783+0000","last_became_active":"2024-05-03T14:44:31.467973+0000","last_became_peered":"2024-05-03T14:44:31.467973+0000","last_unstale":"2024-05-03T14:46:05.590783+0000","last_undegraded":"2024-05-03T14:46:05.590783+0000","last_fullsized":"2024-05-03T14:46:05.590783+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:44:21.266589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.113100+0000","last_change":"2024-05-03T14:44:44.582678+0000","last_active":"2024-05-03T14:46:06.113100+0000","last_peered":"2024-05-03T14:46:06.113100+0000","last_clean":"2024-05-03T14:46:06.113100+0000","last_became_active":"2024-05-03T14:44:44.582155+0000","last_became_peered":"2024-05-03T14:44:44.582155+0000","last_unstale":"2024-05-03T14:46:06.113100+0000","last_undegraded":"2024-05-03T14:46:06.113100+0000","last_fullsized":"2024-05-03T14:46:06.113100+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T15:21:05.986627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":114,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:29.592330+0000","last_change":"2024-05-03T14:44:27.415880+0000","last_active":"2024-05-03T14:46:29.592330+0000","last_peered":"2024-05-03T14:46:29.592330+0000","last_clean":"2024-05-03T14:46:29.592330+0000","last_became_active":"2024-05-03T14:44:27.415354+0000","last_became_peered":"2024-05-03T14:44:27.415354+0000","last_unstale":"2024-05-03T14:46:29.592330+0000","last_undegraded":"2024-05-03T14:46:29.592330+0000","last_fullsized":"2024-05-03T14:46:29.592330+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T16:55:41.808314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":66,"num_read_kb":66,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109516+0000","last_change":"2024-05-03T14:44:31.462685+0000","last_active":"2024-05-03T14:46:06.109516+0000","last_peered":"2024-05-03T14:46:06.109516+0000","last_clean":"2024-05-03T14:46:06.109516+0000","last_became_active":"2024-05-03T14:44:31.462325+0000","last_became_peered":"2024-05-03T14:44:31.462325+0000","last_unstale":"2024-05-03T14:46:06.109516+0000","last_undegraded":"2024-05-03T14:46:06.109516+0000","last_fullsized":"2024-05-03T14:46:06.109516+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:59:54.715648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.590180+0000","last_change":"2024-05-03T14:44:44.573855+0000","last_active":"2024-05-03T14:46:05.590180+0000","last_peered":"2024-05-03T14:46:05.590180+0000","last_clean":"2024-05-03T14:46:05.590180+0000","last_became_active":"2024-05-03T14:44:44.573314+0000","last_became_peered":"2024-05-03T14:44:44.573314+0000","last_unstale":"2024-05-03T14:46:05.590180+0000","last_undegraded":"2024-05-03T14:46:05.590180+0000","last_fullsized":"2024-05-03T14:46:05.590180+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:04:24.282503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.594812+0000","last_change":"2024-05-03T14:45:05.613037+0000","last_active":"2024-05-03T14:46:05.594812+0000","last_peered":"2024-05-03T14:46:05.594812+0000","last_clean":"2024-05-03T14:46:05.594812+0000","last_became_active":"2024-05-03T14:45:05.612581+0000","last_became_peered":"2024-05-03T14:45:05.612581+0000","last_unstale":"2024-05-03T14:46:05.594812+0000","last_undegraded":"2024-05-03T14:46:05.594812+0000","last_fullsized":"2024-05-03T14:46:05.594812+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:34:07.365447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.590800+0000","last_change":"2024-05-03T14:44:31.451912+0000","last_active":"2024-05-03T14:46:05.590800+0000","last_peered":"2024-05-03T14:46:05.590800+0000","last_clean":"2024-05-03T14:46:05.590800+0000","last_became_active":"2024-05-03T14:44:31.449029+0000","last_became_peered":"2024-05-03T14:44:31.449029+0000","last_unstale":"2024-05-03T14:46:05.590800+0000","last_undegraded":"2024-05-03T14:46:05.590800+0000","last_fullsized":"2024-05-03T14:46:05.590800+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:27:13.018719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109400+0000","last_change":"2024-05-03T14:44:44.577001+0000","last_active":"2024-05-03T14:46:06.109400+0000","last_peered":"2024-05-03T14:46:06.109400+0000","last_clean":"2024-05-03T14:46:06.109400+0000","last_became_active":"2024-05-03T14:44:44.576224+0000","last_became_peered":"2024-05-03T14:44:44.576224+0000","last_unstale":"2024-05-03T14:46:06.109400+0000","last_undegraded":"2024-05-03T14:46:06.109400+0000","last_fullsized":"2024-05-03T14:46:06.109400+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T15:15:30.497801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110238+0000","last_change":"2024-05-03T14:44:27.402496+0000","last_active":"2024-05-03T14:46:06.110238+0000","last_peered":"2024-05-03T14:46:06.110238+0000","last_clean":"2024-05-03T14:46:06.110238+0000","last_became_active":"2024-05-03T14:44:27.402062+0000","last_became_peered":"2024-05-03T14:44:27.402062+0000","last_unstale":"2024-05-03T14:46:06.110238+0000","last_undegraded":"2024-05-03T14:46:06.110238+0000","last_fullsized":"2024-05-03T14:46:06.110238+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:47:10.090902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.108858+0000","last_change":"2024-05-03T14:44:31.452654+0000","last_active":"2024-05-03T14:46:06.108858+0000","last_peered":"2024-05-03T14:46:06.108858+0000","last_clean":"2024-05-03T14:46:06.108858+0000","last_became_active":"2024-05-03T14:44:31.452305+0000","last_became_peered":"2024-05-03T14:44:31.452305+0000","last_unstale":"2024-05-03T14:46:06.108858+0000","last_undegraded":"2024-05-03T14:46:06.108858+0000","last_fullsized":"2024-05-03T14:46:06.108858+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:20:45.816113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.591172+0000","last_change":"2024-05-03T14:44:44.576004+0000","last_active":"2024-05-03T14:46:05.591172+0000","last_peered":"2024-05-03T14:46:05.591172+0000","last_clean":"2024-05-03T14:46:05.591172+0000","last_became_active":"2024-05-03T14:44:44.572486+0000","last_became_peered":"2024-05-03T14:44:44.572486+0000","last_unstale":"2024-05-03T14:46:05.591172+0000","last_undegraded":"2024-05-03T14:46:05.591172+0000","last_fullsized":"2024-05-03T14:46:05.591172+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T15:11:51.125404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'36","reported_seq":155,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:29.592766+0000","last_change":"2024-05-03T14:44:27.405937+0000","last_active":"2024-05-03T14:46:29.592766+0000","last_peered":"2024-05-03T14:46:29.592766+0000","last_clean":"2024-05-03T14:46:29.592766+0000","last_became_active":"2024-05-03T14:44:27.405511+0000","last_became_peered":"2024-05-03T14:44:27.405511+0000","last_unstale":"2024-05-03T14:46:29.592766+0000","last_undegraded":"2024-05-03T14:46:29.592766+0000","last_fullsized":"2024-05-03T14:46:29.592766+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:00:56.824015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58916,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":75,"num_read_kb":95,"num_write":38,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.594060+0000","last_change":"2024-05-03T14:44:31.448712+0000","last_active":"2024-05-03T14:46:05.594060+0000","last_peered":"2024-05-03T14:46:05.594060+0000","last_clean":"2024-05-03T14:46:05.594060+0000","last_became_active":"2024-05-03T14:44:31.448346+0000","last_became_peered":"2024-05-03T14:44:31.448346+0000","last_unstale":"2024-05-03T14:46:05.594060+0000","last_undegraded":"2024-05-03T14:46:05.594060+0000","last_fullsized":"2024-05-03T14:46:05.594060+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T16:32:40.770487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110612+0000","last_change":"2024-05-03T14:44:44.585600+0000","last_active":"2024-05-03T14:46:06.110612+0000","last_peered":"2024-05-03T14:46:06.110612+0000","last_clean":"2024-05-03T14:46:06.110612+0000","last_became_active":"2024-05-03T14:44:44.585221+0000","last_became_peered":"2024-05-03T14:44:44.585221+0000","last_unstale":"2024-05-03T14:46:06.110612+0000","last_undegraded":"2024-05-03T14:46:06.110612+0000","last_fullsized":"2024-05-03T14:46:06.110612+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:28:45.804470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.591282+0000","last_change":"2024-05-03T14:44:27.416524+0000","last_active":"2024-05-03T14:46:05.591282+0000","last_peered":"2024-05-03T14:46:05.591282+0000","last_clean":"2024-05-03T14:46:05.591282+0000","last_became_active":"2024-05-03T14:44:27.416211+0000","last_became_peered":"2024-05-03T14:44:27.416211+0000","last_unstale":"2024-05-03T14:46:05.591282+0000","last_undegraded":"2024-05-03T14:46:05.591282+0000","last_fullsized":"2024-05-03T14:46:05.591282+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:00:26.361267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.111040+0000","last_change":"2024-05-03T14:44:31.468973+0000","last_active":"2024-05-03T14:46:06.111040+0000","last_peered":"2024-05-03T14:46:06.111040+0000","last_clean":"2024-05-03T14:46:06.111040+0000","last_became_active":"2024-05-03T14:44:31.468547+0000","last_became_peered":"2024-05-03T14:44:31.468547+0000","last_unstale":"2024-05-03T14:46:06.111040+0000","last_undegraded":"2024-05-03T14:46:06.111040+0000","last_fullsized":"2024-05-03T14:46:06.111040+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:16:16.223859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109286+0000","last_change":"2024-05-03T14:44:44.579157+0000","last_active":"2024-05-03T14:46:06.109286+0000","last_peered":"2024-05-03T14:46:06.109286+0000","last_clean":"2024-05-03T14:46:06.109286+0000","last_became_active":"2024-05-03T14:44:44.578718+0000","last_became_peered":"2024-05-03T14:44:44.578718+0000","last_unstale":"2024-05-03T14:46:06.109286+0000","last_undegraded":"2024-05-03T14:46:06.109286+0000","last_fullsized":"2024-05-03T14:46:06.109286+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:03:31.574958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110029+0000","last_change":"2024-05-03T14:44:27.415290+0000","last_active":"2024-05-03T14:46:06.110029+0000","last_peered":"2024-05-03T14:46:06.110029+0000","last_clean":"2024-05-03T14:46:06.110029+0000","last_became_active":"2024-05-03T14:44:27.414930+0000","last_became_peered":"2024-05-03T14:44:27.414930+0000","last_unstale":"2024-05-03T14:46:06.110029+0000","last_undegraded":"2024-05-03T14:46:06.110029+0000","last_fullsized":"2024-05-03T14:46:06.110029+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:06:40.675902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.594341+0000","last_change":"2024-05-03T14:44:31.464631+0000","last_active":"2024-05-03T14:46:05.594341+0000","last_peered":"2024-05-03T14:46:05.594341+0000","last_clean":"2024-05-03T14:46:05.594341+0000","last_became_active":"2024-05-03T14:44:31.464325+0000","last_became_peered":"2024-05-03T14:44:31.464325+0000","last_unstale":"2024-05-03T14:46:05.594341+0000","last_undegraded":"2024-05-03T14:46:05.594341+0000","last_fullsized":"2024-05-03T14:46:05.594341+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:09:39.140391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.591327+0000","last_change":"2024-05-03T14:44:44.583495+0000","last_active":"2024-05-03T14:46:05.591327+0000","last_peered":"2024-05-03T14:46:05.591327+0000","last_clean":"2024-05-03T14:46:05.591327+0000","last_became_active":"2024-05-03T14:44:44.582971+0000","last_became_peered":"2024-05-03T14:44:44.582971+0000","last_unstale":"2024-05-03T14:46:05.591327+0000","last_undegraded":"2024-05-03T14:46:05.591327+0000","last_fullsized":"2024-05-03T14:46:05.591327+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:23:12.086610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"87'5","reported_seq":83,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:27.610381+0000","last_change":"2024-05-03T14:44:27.403319+0000","last_active":"2024-05-03T14:46:27.610381+0000","last_peered":"2024-05-03T14:46:27.610381+0000","last_clean":"2024-05-03T14:46:27.610381+0000","last_became_active":"2024-05-03T14:44:27.402936+0000","last_became_peered":"2024-05-03T14:44:27.402936+0000","last_unstale":"2024-05-03T14:46:27.610381+0000","last_undegraded":"2024-05-03T14:46:27.610381+0000","last_fullsized":"2024-05-03T14:46:27.610381+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:06:24.851356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":35,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.594223+0000","last_change":"2024-05-03T14:44:31.453514+0000","last_active":"2024-05-03T14:46:05.594223+0000","last_peered":"2024-05-03T14:46:05.594223+0000","last_clean":"2024-05-03T14:46:05.594223+0000","last_became_active":"2024-05-03T14:44:31.452755+0000","last_became_peered":"2024-05-03T14:44:31.452755+0000","last_unstale":"2024-05-03T14:46:05.594223+0000","last_undegraded":"2024-05-03T14:46:05.594223+0000","last_fullsized":"2024-05-03T14:46:05.594223+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:29:45.095487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.113797+0000","last_change":"2024-05-03T14:44:44.587105+0000","last_active":"2024-05-03T14:46:06.113797+0000","last_peered":"2024-05-03T14:46:06.113797+0000","last_clean":"2024-05-03T14:46:06.113797+0000","last_became_active":"2024-05-03T14:44:44.586669+0000","last_became_peered":"2024-05-03T14:44:44.586669+0000","last_unstale":"2024-05-03T14:46:06.113797+0000","last_undegraded":"2024-05-03T14:46:06.113797+0000","last_fullsized":"2024-05-03T14:46:06.113797+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:18:16.364544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":62,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.593578+0000","last_change":"2024-05-03T14:44:27.412481+0000","last_active":"2024-05-03T14:46:05.593578+0000","last_peered":"2024-05-03T14:46:05.593578+0000","last_clean":"2024-05-03T14:46:05.593578+0000","last_became_active":"2024-05-03T14:44:27.412146+0000","last_became_peered":"2024-05-03T14:44:27.412146+0000","last_unstale":"2024-05-03T14:46:05.593578+0000","last_undegraded":"2024-05-03T14:46:05.593578+0000","last_fullsized":"2024-05-03T14:46:05.593578+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T16:03:40.205892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109727+0000","last_change":"2024-05-03T14:44:31.453208+0000","last_active":"2024-05-03T14:46:06.109727+0000","last_peered":"2024-05-03T14:46:06.109727+0000","last_clean":"2024-05-03T14:46:06.109727+0000","last_became_active":"2024-05-03T14:44:31.452788+0000","last_became_peered":"2024-05-03T14:44:31.452788+0000","last_unstale":"2024-05-03T14:46:06.109727+0000","last_undegraded":"2024-05-03T14:46:06.109727+0000","last_fullsized":"2024-05-03T14:46:06.109727+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:43:31.554510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109677+0000","last_change":"2024-05-03T14:44:44.582993+0000","last_active":"2024-05-03T14:46:06.109677+0000","last_peered":"2024-05-03T14:46:06.109677+0000","last_clean":"2024-05-03T14:46:06.109677+0000","last_became_active":"2024-05-03T14:44:44.582511+0000","last_became_peered":"2024-05-03T14:44:44.582511+0000","last_unstale":"2024-05-03T14:46:06.109677+0000","last_undegraded":"2024-05-03T14:46:06.109677+0000","last_fullsized":"2024-05-03T14:46:06.109677+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:22:26.380344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.594298+0000","last_change":"2024-05-03T14:44:27.401742+0000","last_active":"2024-05-03T14:46:05.594298+0000","last_peered":"2024-05-03T14:46:05.594298+0000","last_clean":"2024-05-03T14:46:05.594298+0000","last_became_active":"2024-05-03T14:44:27.401347+0000","last_became_peered":"2024-05-03T14:44:27.401347+0000","last_unstale":"2024-05-03T14:46:05.594298+0000","last_undegraded":"2024-05-03T14:46:05.594298+0000","last_fullsized":"2024-05-03T14:46:05.594298+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T15:39:41.144999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109323+0000","last_change":"2024-05-03T14:44:31.453480+0000","last_active":"2024-05-03T14:46:06.109323+0000","last_peered":"2024-05-03T14:46:06.109323+0000","last_clean":"2024-05-03T14:46:06.109323+0000","last_became_active":"2024-05-03T14:44:31.453171+0000","last_became_peered":"2024-05-03T14:44:31.453171+0000","last_unstale":"2024-05-03T14:46:06.109323+0000","last_undegraded":"2024-05-03T14:46:06.109323+0000","last_fullsized":"2024-05-03T14:46:06.109323+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:30:16.567938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.593599+0000","last_change":"2024-05-03T14:44:45.208587+0000","last_active":"2024-05-03T14:46:05.593599+0000","last_peered":"2024-05-03T14:46:05.593599+0000","last_clean":"2024-05-03T14:46:05.593599+0000","last_became_active":"2024-05-03T14:44:45.207987+0000","last_became_peered":"2024-05-03T14:44:45.207987+0000","last_unstale":"2024-05-03T14:46:05.593599+0000","last_undegraded":"2024-05-03T14:46:05.593599+0000","last_fullsized":"2024-05-03T14:46:05.593599+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:22:14.854390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.592981+0000","last_change":"2024-05-03T14:44:27.421333+0000","last_active":"2024-05-03T14:46:05.592981+0000","last_peered":"2024-05-03T14:46:05.592981+0000","last_clean":"2024-05-03T14:46:05.592981+0000","last_became_active":"2024-05-03T14:44:27.419745+0000","last_became_peered":"2024-05-03T14:44:27.419745+0000","last_unstale":"2024-05-03T14:46:05.592981+0000","last_undegraded":"2024-05-03T14:46:05.592981+0000","last_fullsized":"2024-05-03T14:46:05.592981+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:40:57.258260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.608091+0000","last_change":"2024-05-03T14:44:31.467871+0000","last_active":"2024-05-03T14:46:05.608091+0000","last_peered":"2024-05-03T14:46:05.608091+0000","last_clean":"2024-05-03T14:46:05.608091+0000","last_became_active":"2024-05-03T14:44:31.467408+0000","last_became_peered":"2024-05-03T14:44:31.467408+0000","last_unstale":"2024-05-03T14:46:05.608091+0000","last_undegraded":"2024-05-03T14:46:05.608091+0000","last_fullsized":"2024-05-03T14:46:05.608091+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:19:24.843591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110417+0000","last_change":"2024-05-03T14:44:44.583116+0000","last_active":"2024-05-03T14:46:06.110417+0000","last_peered":"2024-05-03T14:46:06.110417+0000","last_clean":"2024-05-03T14:46:06.110417+0000","last_became_active":"2024-05-03T14:44:44.581838+0000","last_became_peered":"2024-05-03T14:44:44.581838+0000","last_unstale":"2024-05-03T14:46:06.110417+0000","last_undegraded":"2024-05-03T14:46:06.110417+0000","last_fullsized":"2024-05-03T14:46:06.110417+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:14:24.608777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'5","reported_seq":83,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:29.169125+0000","last_change":"2024-05-03T14:44:27.416068+0000","last_active":"2024-05-03T14:46:29.169125+0000","last_peered":"2024-05-03T14:46:29.169125+0000","last_clean":"2024-05-03T14:46:29.169125+0000","last_became_active":"2024-05-03T14:44:27.415574+0000","last_became_peered":"2024-05-03T14:44:27.415574+0000","last_unstale":"2024-05-03T14:46:29.169125+0000","last_undegraded":"2024-05-03T14:46:29.169125+0000","last_fullsized":"2024-05-03T14:46:29.169125+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T15:31:40.186754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.608007+0000","last_change":"2024-05-03T14:44:31.453935+0000","last_active":"2024-05-03T14:46:05.608007+0000","last_peered":"2024-05-03T14:46:05.608007+0000","last_clean":"2024-05-03T14:46:05.608007+0000","last_became_active":"2024-05-03T14:44:31.453565+0000","last_became_peered":"2024-05-03T14:44:31.453565+0000","last_unstale":"2024-05-03T14:46:05.608007+0000","last_undegraded":"2024-05-03T14:46:05.608007+0000","last_fullsized":"2024-05-03T14:46:05.608007+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:34:04.855830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.111343+0000","last_change":"2024-05-03T14:44:45.210274+0000","last_active":"2024-05-03T14:46:06.111343+0000","last_peered":"2024-05-03T14:46:06.111343+0000","last_clean":"2024-05-03T14:46:06.111343+0000","last_became_active":"2024-05-03T14:44:45.208951+0000","last_became_peered":"2024-05-03T14:44:45.208951+0000","last_unstale":"2024-05-03T14:46:06.111343+0000","last_undegraded":"2024-05-03T14:46:06.111343+0000","last_fullsized":"2024-05-03T14:46:06.111343+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:56:30.953019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109755+0000","last_change":"2024-05-03T14:44:31.449470+0000","last_active":"2024-05-03T14:46:06.109755+0000","last_peered":"2024-05-03T14:46:06.109755+0000","last_clean":"2024-05-03T14:46:06.109755+0000","last_became_active":"2024-05-03T14:44:31.449031+0000","last_became_peered":"2024-05-03T14:44:31.449031+0000","last_unstale":"2024-05-03T14:46:06.109755+0000","last_undegraded":"2024-05-03T14:46:06.109755+0000","last_fullsized":"2024-05-03T14:46:06.109755+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T14:51:54.576685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":83,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:28.611584+0000","last_change":"2024-05-03T14:44:27.422378+0000","last_active":"2024-05-03T14:46:28.611584+0000","last_peered":"2024-05-03T14:46:28.611584+0000","last_clean":"2024-05-03T14:46:28.611584+0000","last_became_active":"2024-05-03T14:44:27.421988+0000","last_became_peered":"2024-05-03T14:44:27.421988+0000","last_unstale":"2024-05-03T14:46:28.611584+0000","last_undegraded":"2024-05-03T14:46:28.611584+0000","last_fullsized":"2024-05-03T14:46:28.611584+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:20:10.371418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":34,"num_read_kb":34,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.594688+0000","last_change":"2024-05-03T14:44:44.576084+0000","last_active":"2024-05-03T14:46:05.594688+0000","last_peered":"2024-05-03T14:46:05.594688+0000","last_clean":"2024-05-03T14:46:05.594688+0000","last_became_active":"2024-05-03T14:44:44.575633+0000","last_became_peered":"2024-05-03T14:44:44.575633+0000","last_unstale":"2024-05-03T14:46:05.594688+0000","last_undegraded":"2024-05-03T14:46:05.594688+0000","last_fullsized":"2024-05-03T14:46:05.594688+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:35:38.570413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.113711+0000","last_change":"2024-05-03T14:44:31.467024+0000","last_active":"2024-05-03T14:46:06.113711+0000","last_peered":"2024-05-03T14:46:06.113711+0000","last_clean":"2024-05-03T14:46:06.113711+0000","last_became_active":"2024-05-03T14:44:31.466545+0000","last_became_peered":"2024-05-03T14:44:31.466545+0000","last_unstale":"2024-05-03T14:46:06.113711+0000","last_undegraded":"2024-05-03T14:46:06.113711+0000","last_fullsized":"2024-05-03T14:46:06.113711+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:34:41.289638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109266+0000","last_change":"2024-05-03T14:44:27.410463+0000","last_active":"2024-05-03T14:46:06.109266+0000","last_peered":"2024-05-03T14:46:06.109266+0000","last_clean":"2024-05-03T14:46:06.109266+0000","last_became_active":"2024-05-03T14:44:27.410017+0000","last_became_peered":"2024-05-03T14:44:27.410017+0000","last_unstale":"2024-05-03T14:46:06.109266+0000","last_undegraded":"2024-05-03T14:46:06.109266+0000","last_fullsized":"2024-05-03T14:46:06.109266+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T16:44:40.376214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":119,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.593994+0000","last_change":"2024-05-03T14:42:51.347913+0000","last_active":"2024-05-03T14:46:05.593994+0000","last_peered":"2024-05-03T14:46:05.593994+0000","last_clean":"2024-05-03T14:46:05.593994+0000","last_became_active":"2024-05-03T14:42:51.341302+0000","last_became_peered":"2024-05-03T14:42:51.341302+0000","last_unstale":"2024-05-03T14:46:05.593994+0000","last_undegraded":"2024-05-03T14:46:05.593994+0000","last_fullsized":"2024-05-03T14:46:05.593994+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:38:55.096374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:38:55.096374+0000","last_clean_scrub_stamp":"2024-05-03T14:38:55.096374+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:35:46.708686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110529+0000","last_change":"2024-05-03T14:44:45.209369+0000","last_active":"2024-05-03T14:46:06.110529+0000","last_peered":"2024-05-03T14:46:06.110529+0000","last_clean":"2024-05-03T14:46:06.110529+0000","last_became_active":"2024-05-03T14:44:45.208835+0000","last_became_peered":"2024-05-03T14:44:45.208835+0000","last_unstale":"2024-05-03T14:46:06.110529+0000","last_undegraded":"2024-05-03T14:46:06.110529+0000","last_fullsized":"2024-05-03T14:46:06.110529+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:35:35.343030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109853+0000","last_change":"2024-05-03T14:44:31.462199+0000","last_active":"2024-05-03T14:46:06.109853+0000","last_peered":"2024-05-03T14:46:06.109853+0000","last_clean":"2024-05-03T14:46:06.109853+0000","last_became_active":"2024-05-03T14:44:31.460972+0000","last_became_peered":"2024-05-03T14:44:31.460972+0000","last_unstale":"2024-05-03T14:46:06.109853+0000","last_undegraded":"2024-05-03T14:46:06.109853+0000","last_fullsized":"2024-05-03T14:46:06.109853+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:21:49.619023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.594437+0000","last_change":"2024-05-03T14:44:27.403303+0000","last_active":"2024-05-03T14:46:05.594437+0000","last_peered":"2024-05-03T14:46:05.594437+0000","last_clean":"2024-05-03T14:46:05.594437+0000","last_became_active":"2024-05-03T14:44:27.402870+0000","last_became_peered":"2024-05-03T14:44:27.402870+0000","last_unstale":"2024-05-03T14:46:05.594437+0000","last_undegraded":"2024-05-03T14:46:05.594437+0000","last_fullsized":"2024-05-03T14:46:05.594437+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:57:23.209298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109237+0000","last_change":"2024-05-03T14:44:44.579532+0000","last_active":"2024-05-03T14:46:06.109237+0000","last_peered":"2024-05-03T14:46:06.109237+0000","last_clean":"2024-05-03T14:46:06.109237+0000","last_became_active":"2024-05-03T14:44:44.579052+0000","last_became_peered":"2024-05-03T14:44:44.579052+0000","last_unstale":"2024-05-03T14:46:06.109237+0000","last_undegraded":"2024-05-03T14:46:06.109237+0000","last_fullsized":"2024-05-03T14:46:06.109237+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:07:43.567880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109477+0000","last_change":"2024-05-03T14:45:05.422044+0000","last_active":"2024-05-03T14:46:06.109477+0000","last_peered":"2024-05-03T14:46:06.109477+0000","last_clean":"2024-05-03T14:46:06.109477+0000","last_became_active":"2024-05-03T14:45:05.421557+0000","last_became_peered":"2024-05-03T14:45:05.421557+0000","last_unstale":"2024-05-03T14:46:06.109477+0000","last_undegraded":"2024-05-03T14:46:06.109477+0000","last_fullsized":"2024-05-03T14:46:06.109477+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:44:03.074990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110826+0000","last_change":"2024-05-03T14:44:31.460150+0000","last_active":"2024-05-03T14:46:06.110826+0000","last_peered":"2024-05-03T14:46:06.110826+0000","last_clean":"2024-05-03T14:46:06.110826+0000","last_became_active":"2024-05-03T14:44:31.459550+0000","last_became_peered":"2024-05-03T14:44:31.459550+0000","last_unstale":"2024-05-03T14:46:06.110826+0000","last_undegraded":"2024-05-03T14:46:06.110826+0000","last_fullsized":"2024-05-03T14:46:06.110826+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:28:17.556068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.112955+0000","last_change":"2024-05-03T14:44:45.215035+0000","last_active":"2024-05-03T14:46:06.112955+0000","last_peered":"2024-05-03T14:46:06.112955+0000","last_clean":"2024-05-03T14:46:06.112955+0000","last_became_active":"2024-05-03T14:44:45.208667+0000","last_became_peered":"2024-05-03T14:44:45.208667+0000","last_unstale":"2024-05-03T14:46:06.112955+0000","last_undegraded":"2024-05-03T14:46:06.112955+0000","last_fullsized":"2024-05-03T14:46:06.112955+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:11:47.512884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'13","reported_seq":57,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:17.491776+0000","last_change":"2024-05-03T14:44:27.417377+0000","last_active":"2024-05-03T14:46:17.491776+0000","last_peered":"2024-05-03T14:46:17.491776+0000","last_clean":"2024-05-03T14:46:17.491776+0000","last_became_active":"2024-05-03T14:44:27.416983+0000","last_became_peered":"2024-05-03T14:44:27.416983+0000","last_unstale":"2024-05-03T14:46:17.491776+0000","last_undegraded":"2024-05-03T14:46:17.491776+0000","last_fullsized":"2024-05-03T14:46:17.491776+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T15:37:31.610517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.115229+0000","last_change":"2024-05-03T14:44:31.467316+0000","last_active":"2024-05-03T14:46:06.115229+0000","last_peered":"2024-05-03T14:46:06.115229+0000","last_clean":"2024-05-03T14:46:06.115229+0000","last_became_active":"2024-05-03T14:44:31.466025+0000","last_became_peered":"2024-05-03T14:44:31.466025+0000","last_unstale":"2024-05-03T14:46:06.115229+0000","last_undegraded":"2024-05-03T14:46:06.115229+0000","last_fullsized":"2024-05-03T14:46:06.115229+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:13:00.130689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.590957+0000","last_change":"2024-05-03T14:44:44.567031+0000","last_active":"2024-05-03T14:46:05.590957+0000","last_peered":"2024-05-03T14:46:05.590957+0000","last_clean":"2024-05-03T14:46:05.590957+0000","last_became_active":"2024-05-03T14:44:44.566562+0000","last_became_peered":"2024-05-03T14:44:44.566562+0000","last_unstale":"2024-05-03T14:46:05.590957+0000","last_undegraded":"2024-05-03T14:46:05.590957+0000","last_fullsized":"2024-05-03T14:46:05.590957+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T15:47:12.335283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110029+0000","last_change":"2024-05-03T14:44:28.279115+0000","last_active":"2024-05-03T14:46:06.110029+0000","last_peered":"2024-05-03T14:46:06.110029+0000","last_clean":"2024-05-03T14:46:06.110029+0000","last_became_active":"2024-05-03T14:44:28.278781+0000","last_became_peered":"2024-05-03T14:44:28.278781+0000","last_unstale":"2024-05-03T14:46:06.110029+0000","last_undegraded":"2024-05-03T14:46:06.110029+0000","last_fullsized":"2024-05-03T14:46:06.110029+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:40:41.273763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.590447+0000","last_change":"2024-05-03T14:44:31.475575+0000","last_active":"2024-05-03T14:46:05.590447+0000","last_peered":"2024-05-03T14:46:05.590447+0000","last_clean":"2024-05-03T14:46:05.590447+0000","last_became_active":"2024-05-03T14:44:31.474336+0000","last_became_peered":"2024-05-03T14:44:31.474336+0000","last_unstale":"2024-05-03T14:46:05.590447+0000","last_undegraded":"2024-05-03T14:46:05.590447+0000","last_fullsized":"2024-05-03T14:46:05.590447+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:22:48.742222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109630+0000","last_change":"2024-05-03T14:45:05.616983+0000","last_active":"2024-05-03T14:46:06.109630+0000","last_peered":"2024-05-03T14:46:06.109630+0000","last_clean":"2024-05-03T14:46:06.109630+0000","last_became_active":"2024-05-03T14:45:05.616475+0000","last_became_peered":"2024-05-03T14:45:05.616475+0000","last_unstale":"2024-05-03T14:46:06.109630+0000","last_undegraded":"2024-05-03T14:46:06.109630+0000","last_fullsized":"2024-05-03T14:46:06.109630+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:01:02.124133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.108605+0000","last_change":"2024-05-03T14:44:27.417092+0000","last_active":"2024-05-03T14:46:06.108605+0000","last_peered":"2024-05-03T14:46:06.108605+0000","last_clean":"2024-05-03T14:46:06.108605+0000","last_became_active":"2024-05-03T14:44:27.416765+0000","last_became_peered":"2024-05-03T14:44:27.416765+0000","last_unstale":"2024-05-03T14:46:06.108605+0000","last_undegraded":"2024-05-03T14:46:06.108605+0000","last_fullsized":"2024-05-03T14:46:06.108605+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T16:08:01.882063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.593593+0000","last_change":"2024-05-03T14:44:31.448394+0000","last_active":"2024-05-03T14:46:05.593593+0000","last_peered":"2024-05-03T14:46:05.593593+0000","last_clean":"2024-05-03T14:46:05.593593+0000","last_became_active":"2024-05-03T14:44:31.447752+0000","last_became_peered":"2024-05-03T14:44:31.447752+0000","last_unstale":"2024-05-03T14:46:05.593593+0000","last_undegraded":"2024-05-03T14:46:05.593593+0000","last_fullsized":"2024-05-03T14:46:05.593593+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:01:24.601259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.591530+0000","last_change":"2024-05-03T14:44:44.587688+0000","last_active":"2024-05-03T14:46:05.591530+0000","last_peered":"2024-05-03T14:46:05.591530+0000","last_clean":"2024-05-03T14:46:05.591530+0000","last_became_active":"2024-05-03T14:44:44.586982+0000","last_became_peered":"2024-05-03T14:44:44.586982+0000","last_unstale":"2024-05-03T14:46:05.591530+0000","last_undegraded":"2024-05-03T14:46:05.591530+0000","last_fullsized":"2024-05-03T14:46:05.591530+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:22:54.456635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.113165+0000","last_change":"2024-05-03T14:44:27.411828+0000","last_active":"2024-05-03T14:46:06.113165+0000","last_peered":"2024-05-03T14:46:06.113165+0000","last_clean":"2024-05-03T14:46:06.113165+0000","last_became_active":"2024-05-03T14:44:27.410490+0000","last_became_peered":"2024-05-03T14:44:27.410490+0000","last_unstale":"2024-05-03T14:46:06.113165+0000","last_undegraded":"2024-05-03T14:46:06.113165+0000","last_fullsized":"2024-05-03T14:46:06.113165+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:34:00.061773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109045+0000","last_change":"2024-05-03T14:44:31.448582+0000","last_active":"2024-05-03T14:46:06.109045+0000","last_peered":"2024-05-03T14:46:06.109045+0000","last_clean":"2024-05-03T14:46:06.109045+0000","last_became_active":"2024-05-03T14:44:31.448246+0000","last_became_peered":"2024-05-03T14:44:31.448246+0000","last_unstale":"2024-05-03T14:46:06.109045+0000","last_undegraded":"2024-05-03T14:46:06.109045+0000","last_fullsized":"2024-05-03T14:46:06.109045+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:12:19.856995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109106+0000","last_change":"2024-05-03T14:45:05.616413+0000","last_active":"2024-05-03T14:46:06.109106+0000","last_peered":"2024-05-03T14:46:06.109106+0000","last_clean":"2024-05-03T14:46:06.109106+0000","last_became_active":"2024-05-03T14:45:05.615930+0000","last_became_peered":"2024-05-03T14:45:05.615930+0000","last_unstale":"2024-05-03T14:46:06.109106+0000","last_undegraded":"2024-05-03T14:46:06.109106+0000","last_fullsized":"2024-05-03T14:46:06.109106+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:32:43.344452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.591057+0000","last_change":"2024-05-03T14:44:27.417625+0000","last_active":"2024-05-03T14:46:05.591057+0000","last_peered":"2024-05-03T14:46:05.591057+0000","last_clean":"2024-05-03T14:46:05.591057+0000","last_became_active":"2024-05-03T14:44:27.417230+0000","last_became_peered":"2024-05-03T14:44:27.417230+0000","last_unstale":"2024-05-03T14:46:05.591057+0000","last_undegraded":"2024-05-03T14:46:05.591057+0000","last_fullsized":"2024-05-03T14:46:05.591057+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:56:13.661544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.114328+0000","last_change":"2024-05-03T14:44:31.475539+0000","last_active":"2024-05-03T14:46:06.114328+0000","last_peered":"2024-05-03T14:46:06.114328+0000","last_clean":"2024-05-03T14:46:06.114328+0000","last_became_active":"2024-05-03T14:44:31.474743+0000","last_became_peered":"2024-05-03T14:44:31.474743+0000","last_unstale":"2024-05-03T14:46:06.114328+0000","last_undegraded":"2024-05-03T14:46:06.114328+0000","last_fullsized":"2024-05-03T14:46:06.114328+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T15:52:19.705304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109430+0000","last_change":"2024-05-03T14:44:45.208377+0000","last_active":"2024-05-03T14:46:06.109430+0000","last_peered":"2024-05-03T14:46:06.109430+0000","last_clean":"2024-05-03T14:46:06.109430+0000","last_became_active":"2024-05-03T14:44:45.207778+0000","last_became_peered":"2024-05-03T14:44:45.207778+0000","last_unstale":"2024-05-03T14:46:06.109430+0000","last_undegraded":"2024-05-03T14:46:06.109430+0000","last_fullsized":"2024-05-03T14:46:06.109430+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T15:51:36.843263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.593812+0000","last_change":"2024-05-03T14:44:27.403571+0000","last_active":"2024-05-03T14:46:05.593812+0000","last_peered":"2024-05-03T14:46:05.593812+0000","last_clean":"2024-05-03T14:46:05.593812+0000","last_became_active":"2024-05-03T14:44:27.403173+0000","last_became_peered":"2024-05-03T14:44:27.403173+0000","last_unstale":"2024-05-03T14:46:05.593812+0000","last_undegraded":"2024-05-03T14:46:05.593812+0000","last_fullsized":"2024-05-03T14:46:05.593812+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T15:32:07.406763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.594783+0000","last_change":"2024-05-03T14:44:31.459573+0000","last_active":"2024-05-03T14:46:05.594783+0000","last_peered":"2024-05-03T14:46:05.594783+0000","last_clean":"2024-05-03T14:46:05.594783+0000","last_became_active":"2024-05-03T14:44:31.459200+0000","last_became_peered":"2024-05-03T14:44:31.459200+0000","last_unstale":"2024-05-03T14:46:05.594783+0000","last_undegraded":"2024-05-03T14:46:05.594783+0000","last_fullsized":"2024-05-03T14:46:05.594783+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:58:04.841787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.108722+0000","last_change":"2024-05-03T14:44:44.579130+0000","last_active":"2024-05-03T14:46:06.108722+0000","last_peered":"2024-05-03T14:46:06.108722+0000","last_clean":"2024-05-03T14:46:06.108722+0000","last_became_active":"2024-05-03T14:44:44.578696+0000","last_became_peered":"2024-05-03T14:44:44.578696+0000","last_unstale":"2024-05-03T14:46:06.108722+0000","last_undegraded":"2024-05-03T14:46:06.108722+0000","last_fullsized":"2024-05-03T14:46:06.108722+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:42:48.407127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'5","reported_seq":83,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:27.590321+0000","last_change":"2024-05-03T14:44:28.279181+0000","last_active":"2024-05-03T14:46:27.590321+0000","last_peered":"2024-05-03T14:46:27.590321+0000","last_clean":"2024-05-03T14:46:27.590321+0000","last_became_active":"2024-05-03T14:44:28.278790+0000","last_became_peered":"2024-05-03T14:44:28.278790+0000","last_unstale":"2024-05-03T14:46:27.590321+0000","last_undegraded":"2024-05-03T14:46:27.590321+0000","last_fullsized":"2024-05-03T14:46:27.590321+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:16:30.170750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":34,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.594856+0000","last_change":"2024-05-03T14:44:31.451646+0000","last_active":"2024-05-03T14:46:05.594856+0000","last_peered":"2024-05-03T14:46:05.594856+0000","last_clean":"2024-05-03T14:46:05.594856+0000","last_became_active":"2024-05-03T14:44:31.451319+0000","last_became_peered":"2024-05-03T14:44:31.451319+0000","last_unstale":"2024-05-03T14:46:05.594856+0000","last_undegraded":"2024-05-03T14:46:05.594856+0000","last_fullsized":"2024-05-03T14:46:05.594856+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:17:51.946028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.590565+0000","last_change":"2024-05-03T14:45:05.619282+0000","last_active":"2024-05-03T14:46:05.590565+0000","last_peered":"2024-05-03T14:46:05.590565+0000","last_clean":"2024-05-03T14:46:05.590565+0000","last_became_active":"2024-05-03T14:45:05.617776+0000","last_became_peered":"2024-05-03T14:45:05.617776+0000","last_unstale":"2024-05-03T14:46:05.590565+0000","last_undegraded":"2024-05-03T14:46:05.590565+0000","last_fullsized":"2024-05-03T14:46:05.590565+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:36:53.105226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110420+0000","last_change":"2024-05-03T14:44:27.415277+0000","last_active":"2024-05-03T14:46:06.110420+0000","last_peered":"2024-05-03T14:46:06.110420+0000","last_clean":"2024-05-03T14:46:06.110420+0000","last_became_active":"2024-05-03T14:44:27.414900+0000","last_became_peered":"2024-05-03T14:44:27.414900+0000","last_unstale":"2024-05-03T14:46:06.110420+0000","last_undegraded":"2024-05-03T14:46:06.110420+0000","last_fullsized":"2024-05-03T14:46:06.110420+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:41:00.065933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.594532+0000","last_change":"2024-05-03T14:44:31.467592+0000","last_active":"2024-05-03T14:46:05.594532+0000","last_peered":"2024-05-03T14:46:05.594532+0000","last_clean":"2024-05-03T14:46:05.594532+0000","last_became_active":"2024-05-03T14:44:31.467265+0000","last_became_peered":"2024-05-03T14:44:31.467265+0000","last_unstale":"2024-05-03T14:46:05.594532+0000","last_undegraded":"2024-05-03T14:46:05.594532+0000","last_fullsized":"2024-05-03T14:46:05.594532+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:28:26.672800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109064+0000","last_change":"2024-05-03T14:44:44.566154+0000","last_active":"2024-05-03T14:46:06.109064+0000","last_peered":"2024-05-03T14:46:06.109064+0000","last_clean":"2024-05-03T14:46:06.109064+0000","last_became_active":"2024-05-03T14:44:44.565714+0000","last_became_peered":"2024-05-03T14:44:44.565714+0000","last_unstale":"2024-05-03T14:46:06.109064+0000","last_undegraded":"2024-05-03T14:46:06.109064+0000","last_fullsized":"2024-05-03T14:46:06.109064+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:54:48.448838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.113551+0000","last_change":"2024-05-03T14:44:27.419293+0000","last_active":"2024-05-03T14:46:06.113551+0000","last_peered":"2024-05-03T14:46:06.113551+0000","last_clean":"2024-05-03T14:46:06.113551+0000","last_became_active":"2024-05-03T14:44:27.418902+0000","last_became_peered":"2024-05-03T14:44:27.418902+0000","last_unstale":"2024-05-03T14:46:06.113551+0000","last_undegraded":"2024-05-03T14:46:06.113551+0000","last_fullsized":"2024-05-03T14:46:06.113551+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:17:54.890553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.595057+0000","last_change":"2024-05-03T14:44:31.449682+0000","last_active":"2024-05-03T14:46:05.595057+0000","last_peered":"2024-05-03T14:46:05.595057+0000","last_clean":"2024-05-03T14:46:05.595057+0000","last_became_active":"2024-05-03T14:44:31.449192+0000","last_became_peered":"2024-05-03T14:44:31.449192+0000","last_unstale":"2024-05-03T14:46:05.595057+0000","last_undegraded":"2024-05-03T14:46:05.595057+0000","last_fullsized":"2024-05-03T14:46:05.595057+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:48:46.249154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109124+0000","last_change":"2024-05-03T14:45:05.422527+0000","last_active":"2024-05-03T14:46:06.109124+0000","last_peered":"2024-05-03T14:46:06.109124+0000","last_clean":"2024-05-03T14:46:06.109124+0000","last_became_active":"2024-05-03T14:45:05.422014+0000","last_became_peered":"2024-05-03T14:45:05.422014+0000","last_unstale":"2024-05-03T14:46:06.109124+0000","last_undegraded":"2024-05-03T14:46:06.109124+0000","last_fullsized":"2024-05-03T14:46:06.109124+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T15:13:47.216960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.591649+0000","last_change":"2024-05-03T14:44:31.450447+0000","last_active":"2024-05-03T14:46:05.591649+0000","last_peered":"2024-05-03T14:46:05.591649+0000","last_clean":"2024-05-03T14:46:05.591649+0000","last_became_active":"2024-05-03T14:44:31.450103+0000","last_became_peered":"2024-05-03T14:44:31.450103+0000","last_unstale":"2024-05-03T14:46:05.591649+0000","last_undegraded":"2024-05-03T14:46:05.591649+0000","last_fullsized":"2024-05-03T14:46:05.591649+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:43:06.840375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.591177+0000","last_change":"2024-05-03T14:44:31.467676+0000","last_active":"2024-05-03T14:46:05.591177+0000","last_peered":"2024-05-03T14:46:05.591177+0000","last_clean":"2024-05-03T14:46:05.591177+0000","last_became_active":"2024-05-03T14:44:31.467285+0000","last_became_peered":"2024-05-03T14:44:31.467285+0000","last_unstale":"2024-05-03T14:46:05.591177+0000","last_undegraded":"2024-05-03T14:46:05.591177+0000","last_fullsized":"2024-05-03T14:46:05.591177+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:39:37.400178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.591210+0000","last_change":"2024-05-03T14:44:31.469379+0000","last_active":"2024-05-03T14:46:05.591210+0000","last_peered":"2024-05-03T14:46:05.591210+0000","last_clean":"2024-05-03T14:46:05.591210+0000","last_became_active":"2024-05-03T14:44:31.469022+0000","last_became_peered":"2024-05-03T14:44:31.469022+0000","last_unstale":"2024-05-03T14:46:05.591210+0000","last_undegraded":"2024-05-03T14:46:05.591210+0000","last_fullsized":"2024-05-03T14:46:05.591210+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:33:54.227652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.111018+0000","last_change":"2024-05-03T14:44:31.463709+0000","last_active":"2024-05-03T14:46:06.111018+0000","last_peered":"2024-05-03T14:46:06.111018+0000","last_clean":"2024-05-03T14:46:06.111018+0000","last_became_active":"2024-05-03T14:44:31.463302+0000","last_became_peered":"2024-05-03T14:44:31.463302+0000","last_unstale":"2024-05-03T14:46:06.111018+0000","last_undegraded":"2024-05-03T14:46:06.111018+0000","last_fullsized":"2024-05-03T14:46:06.111018+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:44:39.692253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109781+0000","last_change":"2024-05-03T14:44:31.443381+0000","last_active":"2024-05-03T14:46:06.109781+0000","last_peered":"2024-05-03T14:46:06.109781+0000","last_clean":"2024-05-03T14:46:06.109781+0000","last_became_active":"2024-05-03T14:44:31.442981+0000","last_became_peered":"2024-05-03T14:44:31.442981+0000","last_unstale":"2024-05-03T14:46:06.109781+0000","last_undegraded":"2024-05-03T14:46:06.109781+0000","last_fullsized":"2024-05-03T14:46:06.109781+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:58:56.653006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.591515+0000","last_change":"2024-05-03T14:44:31.449229+0000","last_active":"2024-05-03T14:46:05.591515+0000","last_peered":"2024-05-03T14:46:05.591515+0000","last_clean":"2024-05-03T14:46:05.591515+0000","last_became_active":"2024-05-03T14:44:31.448870+0000","last_became_peered":"2024-05-03T14:44:31.448870+0000","last_unstale":"2024-05-03T14:46:05.591515+0000","last_undegraded":"2024-05-03T14:46:05.591515+0000","last_fullsized":"2024-05-03T14:46:05.591515+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:48:29.213680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.115390+0000","last_change":"2024-05-03T14:44:31.468266+0000","last_active":"2024-05-03T14:46:06.115390+0000","last_peered":"2024-05-03T14:46:06.115390+0000","last_clean":"2024-05-03T14:46:06.115390+0000","last_became_active":"2024-05-03T14:44:31.467784+0000","last_became_peered":"2024-05-03T14:44:31.467784+0000","last_unstale":"2024-05-03T14:46:06.115390+0000","last_undegraded":"2024-05-03T14:46:06.115390+0000","last_fullsized":"2024-05-03T14:46:06.115390+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:46:18.979331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109201+0000","last_change":"2024-05-03T14:44:31.467526+0000","last_active":"2024-05-03T14:46:06.109201+0000","last_peered":"2024-05-03T14:46:06.109201+0000","last_clean":"2024-05-03T14:46:06.109201+0000","last_became_active":"2024-05-03T14:44:31.466849+0000","last_became_peered":"2024-05-03T14:44:31.466849+0000","last_unstale":"2024-05-03T14:46:06.109201+0000","last_undegraded":"2024-05-03T14:46:06.109201+0000","last_fullsized":"2024-05-03T14:46:06.109201+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:57:11.147130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.113215+0000","last_change":"2024-05-03T14:44:31.468251+0000","last_active":"2024-05-03T14:46:06.113215+0000","last_peered":"2024-05-03T14:46:06.113215+0000","last_clean":"2024-05-03T14:46:06.113215+0000","last_became_active":"2024-05-03T14:44:31.467812+0000","last_became_peered":"2024-05-03T14:44:31.467812+0000","last_unstale":"2024-05-03T14:46:06.113215+0000","last_undegraded":"2024-05-03T14:46:06.113215+0000","last_fullsized":"2024-05-03T14:46:06.113215+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:52:45.585331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.108865+0000","last_change":"2024-05-03T14:44:31.466378+0000","last_active":"2024-05-03T14:46:06.108865+0000","last_peered":"2024-05-03T14:46:06.108865+0000","last_clean":"2024-05-03T14:46:06.108865+0000","last_became_active":"2024-05-03T14:44:31.466052+0000","last_became_peered":"2024-05-03T14:44:31.466052+0000","last_unstale":"2024-05-03T14:46:06.108865+0000","last_undegraded":"2024-05-03T14:46:06.108865+0000","last_fullsized":"2024-05-03T14:46:06.108865+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:34:48.401934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110416+0000","last_change":"2024-05-03T14:44:31.463771+0000","last_active":"2024-05-03T14:46:06.110416+0000","last_peered":"2024-05-03T14:46:06.110416+0000","last_clean":"2024-05-03T14:46:06.110416+0000","last_became_active":"2024-05-03T14:44:31.463204+0000","last_became_peered":"2024-05-03T14:44:31.463204+0000","last_unstale":"2024-05-03T14:46:06.110416+0000","last_undegraded":"2024-05-03T14:46:06.110416+0000","last_fullsized":"2024-05-03T14:46:06.110416+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:47:33.044835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109493+0000","last_change":"2024-05-03T14:44:31.450950+0000","last_active":"2024-05-03T14:46:06.109493+0000","last_peered":"2024-05-03T14:46:06.109493+0000","last_clean":"2024-05-03T14:46:06.109493+0000","last_became_active":"2024-05-03T14:44:31.450595+0000","last_became_peered":"2024-05-03T14:44:31.450595+0000","last_unstale":"2024-05-03T14:46:06.109493+0000","last_undegraded":"2024-05-03T14:46:06.109493+0000","last_fullsized":"2024-05-03T14:46:06.109493+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:42:41.240430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":250062,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65338,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":361,"num_read_kb":252,"num_write":244,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":196014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":211,"ondisk_log_size":211,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710761,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":516321,"data_compressed":32309,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906990,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28128,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717024,"statfs":{"total":95995035648,"available":95966232576,"internally_reserved":0,"allocated":1138688,"data_stored":1165365,"data_compressed":68497,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070516,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":515467,"data_compressed":32309,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266747,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":999424,"data_stored":515733,"data_compressed":32309,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462975,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":1111279,"data_compressed":68497,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626501,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":987136,"data_stored":510971,"data_compressed":32309,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574945,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":571539,"data_compressed":32309,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738472,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":512427,"data_compressed":32309,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411422,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1064960,"data_stored":570515,"data_compressed":32309,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182486,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":510679,"data_compressed":32309,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986257,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":995328,"data_stored":515045,"data_compressed":32309,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822732,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1069056,"data_stored":1107533,"data_compressed":68497,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34561,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34561,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36188,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36188,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36188,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36188,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36188,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":61290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":60266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":60652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-03T14:46:33.301 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-03T14:46:33.586 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:33 smithi060 ceph-mon[28873]: pgmap v393: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-05-03T14:46:33.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:33 smithi060 ceph-mon[28873]: from='client.24733 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T14:46:33.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:33 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:46:33.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:33 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:46:33.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:33 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:46:33.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:33 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:46:33.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:33 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:46:33.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:33 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:46:33.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:33 smithi090 ceph-mon[33568]: pgmap v393: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-05-03T14:46:33.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:33 smithi090 ceph-mon[33568]: from='client.24733 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T14:46:33.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:33 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:46:33.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:33 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:46:33.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:33 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:46:33.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:33 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:46:33.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:33 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:46:33.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:33 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:46:33.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:33 smithi149 ceph-mon[33332]: pgmap v393: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-05-03T14:46:33.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:33 smithi149 ceph-mon[33332]: from='client.24733 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T14:46:33.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:33 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:46:33.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:33 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:46:33.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:33 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:46:33.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:33 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:46:33.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:33 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:46:33.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:33 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:46:34.527 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:46:35.767 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:35 smithi060 ceph-mon[28873]: pgmap v394: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-05-03T14:46:35.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:35 smithi090 ceph-mon[33568]: pgmap v394: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-05-03T14:46:35.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:35 smithi149 ceph-mon[33332]: pgmap v394: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-05-03T14:46:36.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:36.684+0000 7f36e7a00700 1 -- 172.21.15.60:0/1457253926 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36e006b1b0 msgr2=0x7f36e006b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:36.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:36.684+0000 7f36e7a00700 1 --2- 172.21.15.60:0/1457253926 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36e006b1b0 0x7f36e006b590 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f36d40098f0 tx=0x7f36d4009c00 comp rx=0 tx=0).stop 2024-05-03T14:46:36.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:36.684+0000 7f36e7a00700 1 -- 172.21.15.60:0/1457253926 shutdown_connections 2024-05-03T14:46:36.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:36.684+0000 7f36e7a00700 1 --2- 172.21.15.60:0/1457253926 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36e010ff50 0x7f36e0112340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:36.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:36.684+0000 7f36e7a00700 1 --2- 172.21.15.60:0/1457253926 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36e006b1b0 0x7f36e006b590 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:36.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:36.684+0000 7f36e7a00700 1 --2- 172.21.15.60:0/1457253926 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36e006bad0 0x7f36e010fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:36.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:36.684+0000 7f36e7a00700 1 -- 172.21.15.60:0/1457253926 >> 172.21.15.60:0/1457253926 conn(0x7f36e0079350 msgr2=0x7f36e0079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:46:36.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:36.684+0000 7f36e7a00700 1 -- 172.21.15.60:0/1457253926 shutdown_connections 2024-05-03T14:46:36.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:36.684+0000 7f36e7a00700 1 -- 172.21.15.60:0/1457253926 wait complete. 2024-05-03T14:46:36.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:36.685+0000 7f36e7a00700 1 Processor -- start 2024-05-03T14:46:36.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:36.685+0000 7f36e7a00700 1 -- start start 2024-05-03T14:46:36.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:36.686+0000 7f36e7a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36e006b1b0 0x7f36e019bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:46:36.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:36.686+0000 7f36e7a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36e006bad0 0x7f36e019c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:46:36.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:36.686+0000 7f36e7a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36e010ff50 0x7f36e01a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:46:36.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:36.686+0000 7f36e7a00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f36e0115b90 con 0x7f36e010ff50 2024-05-03T14:46:36.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:36.686+0000 7f36e7a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f36e0115a10 con 0x7f36e006bad0 2024-05-03T14:46:36.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:36.686+0000 7f36e7a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f36e0115d10 con 0x7f36e006b1b0 2024-05-03T14:46:36.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:36.686+0000 7f36e5000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36e006b1b0 0x7f36e019bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:46:36.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:36.686+0000 7f36dfe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36e006bad0 0x7f36e019c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:46:36.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:36.686+0000 7f36e5a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36e010ff50 0x7f36e01a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:46:36.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:36.686+0000 7f36e5000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36e006b1b0 0x7f36e019bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.60:37942/0 (socket says 172.21.15.60:37942) 2024-05-03T14:46:36.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:36.686+0000 7f36e5000700 1 -- 172.21.15.60:0/3740004214 learned_addr learned my addr 172.21.15.60:0/3740004214 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:46:36.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:36.686+0000 7f36dfe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36e006bad0 0x7f36e019c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.60:44598/0 (socket says 172.21.15.60:44598) 2024-05-03T14:46:36.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:36.687+0000 7f36e5000700 1 -- 172.21.15.60:0/3740004214 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36e006bad0 msgr2=0x7f36e019c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:36.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:36.687+0000 7f36e5000700 1 --2- 172.21.15.60:0/3740004214 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36e006bad0 0x7f36e019c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:36.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:36.687+0000 7f36e5000700 1 -- 172.21.15.60:0/3740004214 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36e010ff50 msgr2=0x7f36e01a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:36.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:36.687+0000 7f36e5000700 1 --2- 172.21.15.60:0/3740004214 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36e010ff50 0x7f36e01a0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:36.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:36.687+0000 7f36e5000700 1 -- 172.21.15.60:0/3740004214 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36e01a0fd0 con 0x7f36e006b1b0 2024-05-03T14:46:36.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:36.687+0000 7f36e5a00700 1 --2- 172.21.15.60:0/3740004214 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36e010ff50 0x7f36e01a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:46:36.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:36.687+0000 7f36e5000700 1 --2- 172.21.15.60:0/3740004214 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36e006b1b0 0x7f36e019bfc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f36d4010f80 tx=0x7f36d4010fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:46:36.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:36.687+0000 7f36dd600700 1 -- 172.21.15.60:0/3740004214 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d4027070 con 0x7f36e006b1b0 2024-05-03T14:46:36.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:36.688+0000 7f36e7a00700 1 -- 172.21.15.60:0/3740004214 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36e01a1260 con 0x7f36e006b1b0 2024-05-03T14:46:36.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:36.688+0000 7f36dd600700 1 -- 172.21.15.60:0/3740004214 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f36d400f460 con 0x7f36e006b1b0 2024-05-03T14:46:36.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:36.688+0000 7f36dd600700 1 -- 172.21.15.60:0/3740004214 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d4004af0 con 0x7f36e006b1b0 2024-05-03T14:46:36.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:36.688+0000 7f36e7a00700 1 -- 172.21.15.60:0/3740004214 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36e01a8b40 con 0x7f36e006b1b0 2024-05-03T14:46:36.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:36.689+0000 7f36e7a00700 1 -- 172.21.15.60:0/3740004214 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36e006a190 con 0x7f36e006b1b0 2024-05-03T14:46:36.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:36.692+0000 7f36dd600700 1 -- 172.21.15.60:0/3740004214 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f36d402a070 con 0x7f36e006b1b0 2024-05-03T14:46:36.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:36.695+0000 7f36dd600700 1 --2- 172.21.15.60:0/3740004214 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f36c80b2910 0x7f36c80b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:46:36.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:36.696+0000 7f36dd600700 1 -- 172.21.15.60:0/3740004214 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f36d40e35c0 con 0x7f36e006b1b0 2024-05-03T14:46:36.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:36.696+0000 7f36dfe00700 1 --2- 172.21.15.60:0/3740004214 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f36c80b2910 0x7f36c80b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:46:36.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:36.696+0000 7f36dd600700 1 -- 172.21.15.60:0/3740004214 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f36d4031080 con 0x7f36e006b1b0 2024-05-03T14:46:36.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:36.696+0000 7f36dfe00700 1 --2- 172.21.15.60:0/3740004214 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f36c80b2910 0x7f36c80b4dd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f36e0103960 tx=0x7f36cc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:46:36.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:36.947+0000 7f36e7a00700 1 -- 172.21.15.60:0/3740004214 --> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f36e019d320 con 0x7f36c80b2910 2024-05-03T14:46:36.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:36.963+0000 7f36dd600700 1 -- 172.21.15.60:0/3740004214 <== mgr.14152 v2:172.21.15.60:6800/3066142956 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340651 (secure 0 0 0) 0x7f36e019d320 con 0x7f36c80b2910 2024-05-03T14:46:36.971 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:46:36.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:36.969+0000 7f36e7a00700 1 -- 172.21.15.60:0/3740004214 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f36c80b2910 msgr2=0x7f36c80b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:36.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:36.969+0000 7f36e7a00700 1 --2- 172.21.15.60:0/3740004214 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f36c80b2910 0x7f36c80b4dd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f36e0103960 tx=0x7f36cc009260 comp rx=0 tx=0).stop 2024-05-03T14:46:36.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:36.969+0000 7f36e7a00700 1 -- 172.21.15.60:0/3740004214 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36e006b1b0 msgr2=0x7f36e019bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:36.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:36.969+0000 7f36e7a00700 1 --2- 172.21.15.60:0/3740004214 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36e006b1b0 0x7f36e019bfc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f36d4010f80 tx=0x7f36d4010fb0 comp rx=0 tx=0).stop 2024-05-03T14:46:36.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:36.969+0000 7f36e7a00700 1 -- 172.21.15.60:0/3740004214 shutdown_connections 2024-05-03T14:46:36.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:36.969+0000 7f36e7a00700 1 --2- 172.21.15.60:0/3740004214 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f36c80b2910 0x7f36c80b4dd0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:36.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:36.969+0000 7f36e7a00700 1 --2- 172.21.15.60:0/3740004214 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36e010ff50 0x7f36e01a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:36.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:36.969+0000 7f36e7a00700 1 --2- 172.21.15.60:0/3740004214 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36e006b1b0 0x7f36e019bfc0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:36.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:36.969+0000 7f36e7a00700 1 --2- 172.21.15.60:0/3740004214 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36e006bad0 0x7f36e019c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:36.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:36.969+0000 7f36e7a00700 1 -- 172.21.15.60:0/3740004214 >> 172.21.15.60:0/3740004214 conn(0x7f36e0079350 msgr2=0x7f36e0111400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:46:36.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:36.969+0000 7f36e7a00700 1 -- 172.21.15.60:0/3740004214 shutdown_connections 2024-05-03T14:46:36.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:36.969+0000 7f36e7a00700 1 -- 172.21.15.60:0/3740004214 wait complete. 2024-05-03T14:46:36.975 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-05-03T14:46:37.737 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:37 smithi060 ceph-mon[28873]: pgmap v395: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-05-03T14:46:37.737 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:37 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:46:37.737 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:37 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:46:37.790 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":395,"stamp":"2024-05-03T14:46:36.366043+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655940,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":487,"num_read_kb":341,"num_write":369,"num_write_kb":2288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":2952512,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":305,"ondisk_log_size":305,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361264,"kb_used_data":12104,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580560,"statfs":{"total":1151940427776,"available":1151570493440,"internally_reserved":0,"allocated":12394496,"data_stored":8122874,"data_compressed":496272,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19140,"internal_metadata":357414204},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":11051,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":137,"num_read_kb":82,"num_write":9,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005967"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110403+0000","last_change":"2024-05-03T14:44:31.452444+0000","last_active":"2024-05-03T14:46:06.110403+0000","last_peered":"2024-05-03T14:46:06.110403+0000","last_clean":"2024-05-03T14:46:06.110403+0000","last_became_active":"2024-05-03T14:44:31.451574+0000","last_became_peered":"2024-05-03T14:44:31.451574+0000","last_unstale":"2024-05-03T14:46:06.110403+0000","last_undegraded":"2024-05-03T14:46:06.110403+0000","last_fullsized":"2024-05-03T14:46:06.110403+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:14:31.009833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.590338+0000","last_change":"2024-05-03T14:44:31.461736+0000","last_active":"2024-05-03T14:46:05.590338+0000","last_peered":"2024-05-03T14:46:05.590338+0000","last_clean":"2024-05-03T14:46:05.590338+0000","last_became_active":"2024-05-03T14:44:31.457963+0000","last_became_peered":"2024-05-03T14:44:31.457963+0000","last_unstale":"2024-05-03T14:46:05.590338+0000","last_undegraded":"2024-05-03T14:46:05.590338+0000","last_fullsized":"2024-05-03T14:46:05.590338+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:59:16.029675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.589959+0000","last_change":"2024-05-03T14:44:31.457163+0000","last_active":"2024-05-03T14:46:05.589959+0000","last_peered":"2024-05-03T14:46:05.589959+0000","last_clean":"2024-05-03T14:46:05.589959+0000","last_became_active":"2024-05-03T14:44:31.455564+0000","last_became_peered":"2024-05-03T14:44:31.455564+0000","last_unstale":"2024-05-03T14:46:05.589959+0000","last_undegraded":"2024-05-03T14:46:05.589959+0000","last_fullsized":"2024-05-03T14:46:05.589959+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T16:14:00.761333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109879+0000","last_change":"2024-05-03T14:44:31.450623+0000","last_active":"2024-05-03T14:46:06.109879+0000","last_peered":"2024-05-03T14:46:06.109879+0000","last_clean":"2024-05-03T14:46:06.109879+0000","last_became_active":"2024-05-03T14:44:31.449944+0000","last_became_peered":"2024-05-03T14:44:31.449944+0000","last_unstale":"2024-05-03T14:46:06.109879+0000","last_undegraded":"2024-05-03T14:46:06.109879+0000","last_fullsized":"2024-05-03T14:46:06.109879+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:40:24.155342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.593903+0000","last_change":"2024-05-03T14:44:31.473624+0000","last_active":"2024-05-03T14:46:05.593903+0000","last_peered":"2024-05-03T14:46:05.593903+0000","last_clean":"2024-05-03T14:46:05.593903+0000","last_became_active":"2024-05-03T14:44:31.473339+0000","last_became_peered":"2024-05-03T14:44:31.473339+0000","last_unstale":"2024-05-03T14:46:05.593903+0000","last_undegraded":"2024-05-03T14:46:05.593903+0000","last_fullsized":"2024-05-03T14:46:05.593903+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:21:42.683546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.608065+0000","last_change":"2024-05-03T14:44:31.466037+0000","last_active":"2024-05-03T14:46:05.608065+0000","last_peered":"2024-05-03T14:46:05.608065+0000","last_clean":"2024-05-03T14:46:05.608065+0000","last_became_active":"2024-05-03T14:44:31.465202+0000","last_became_peered":"2024-05-03T14:44:31.465202+0000","last_unstale":"2024-05-03T14:46:05.608065+0000","last_undegraded":"2024-05-03T14:46:05.608065+0000","last_fullsized":"2024-05-03T14:46:05.608065+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:46:04.189133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.594232+0000","last_change":"2024-05-03T14:44:31.458708+0000","last_active":"2024-05-03T14:46:05.594232+0000","last_peered":"2024-05-03T14:46:05.594232+0000","last_clean":"2024-05-03T14:46:05.594232+0000","last_became_active":"2024-05-03T14:44:31.458292+0000","last_became_peered":"2024-05-03T14:44:31.458292+0000","last_unstale":"2024-05-03T14:46:05.594232+0000","last_undegraded":"2024-05-03T14:46:05.594232+0000","last_fullsized":"2024-05-03T14:46:05.594232+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:15:28.850661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.113279+0000","last_change":"2024-05-03T14:44:31.466051+0000","last_active":"2024-05-03T14:46:06.113279+0000","last_peered":"2024-05-03T14:46:06.113279+0000","last_clean":"2024-05-03T14:46:06.113279+0000","last_became_active":"2024-05-03T14:44:31.465620+0000","last_became_peered":"2024-05-03T14:44:31.465620+0000","last_unstale":"2024-05-03T14:46:06.113279+0000","last_undegraded":"2024-05-03T14:46:06.113279+0000","last_fullsized":"2024-05-03T14:46:06.113279+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:27:02.749377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109419+0000","last_change":"2024-05-03T14:44:31.451507+0000","last_active":"2024-05-03T14:46:06.109419+0000","last_peered":"2024-05-03T14:46:06.109419+0000","last_clean":"2024-05-03T14:46:06.109419+0000","last_became_active":"2024-05-03T14:44:31.451163+0000","last_became_peered":"2024-05-03T14:44:31.451163+0000","last_unstale":"2024-05-03T14:46:06.109419+0000","last_undegraded":"2024-05-03T14:46:06.109419+0000","last_fullsized":"2024-05-03T14:46:06.109419+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:56:50.415619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.608433+0000","last_change":"2024-05-03T14:45:05.421817+0000","last_active":"2024-05-03T14:46:05.608433+0000","last_peered":"2024-05-03T14:46:05.608433+0000","last_clean":"2024-05-03T14:46:05.608433+0000","last_became_active":"2024-05-03T14:45:05.421446+0000","last_became_peered":"2024-05-03T14:45:05.421446+0000","last_unstale":"2024-05-03T14:46:05.608433+0000","last_undegraded":"2024-05-03T14:46:05.608433+0000","last_fullsized":"2024-05-03T14:46:05.608433+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:31:36.867429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.594745+0000","last_change":"2024-05-03T14:44:31.454474+0000","last_active":"2024-05-03T14:46:05.594745+0000","last_peered":"2024-05-03T14:46:05.594745+0000","last_clean":"2024-05-03T14:46:05.594745+0000","last_became_active":"2024-05-03T14:44:31.454089+0000","last_became_peered":"2024-05-03T14:44:31.454089+0000","last_unstale":"2024-05-03T14:46:05.594745+0000","last_undegraded":"2024-05-03T14:46:05.594745+0000","last_fullsized":"2024-05-03T14:46:05.594745+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:54:59.739325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.592878+0000","last_change":"2024-05-03T14:45:05.617724+0000","last_active":"2024-05-03T14:46:05.592878+0000","last_peered":"2024-05-03T14:46:05.592878+0000","last_clean":"2024-05-03T14:46:05.592878+0000","last_became_active":"2024-05-03T14:45:05.616554+0000","last_became_peered":"2024-05-03T14:45:05.616554+0000","last_unstale":"2024-05-03T14:46:05.592878+0000","last_undegraded":"2024-05-03T14:46:05.592878+0000","last_fullsized":"2024-05-03T14:46:05.592878+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:24:20.890688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110600+0000","last_change":"2024-05-03T14:44:31.462676+0000","last_active":"2024-05-03T14:46:06.110600+0000","last_peered":"2024-05-03T14:46:06.110600+0000","last_clean":"2024-05-03T14:46:06.110600+0000","last_became_active":"2024-05-03T14:44:31.459326+0000","last_became_peered":"2024-05-03T14:44:31.459326+0000","last_unstale":"2024-05-03T14:46:06.110600+0000","last_undegraded":"2024-05-03T14:46:06.110600+0000","last_fullsized":"2024-05-03T14:46:06.110600+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:36:22.975112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.589702+0000","last_change":"2024-05-03T14:44:31.447665+0000","last_active":"2024-05-03T14:46:05.589702+0000","last_peered":"2024-05-03T14:46:05.589702+0000","last_clean":"2024-05-03T14:46:05.589702+0000","last_became_active":"2024-05-03T14:44:31.447204+0000","last_became_peered":"2024-05-03T14:44:31.447204+0000","last_unstale":"2024-05-03T14:46:05.589702+0000","last_undegraded":"2024-05-03T14:46:05.589702+0000","last_fullsized":"2024-05-03T14:46:05.589702+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:23:41.313986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.590107+0000","last_change":"2024-05-03T14:44:31.458869+0000","last_active":"2024-05-03T14:46:05.590107+0000","last_peered":"2024-05-03T14:46:05.590107+0000","last_clean":"2024-05-03T14:46:05.590107+0000","last_became_active":"2024-05-03T14:44:31.453157+0000","last_became_peered":"2024-05-03T14:44:31.453157+0000","last_unstale":"2024-05-03T14:46:05.590107+0000","last_undegraded":"2024-05-03T14:46:05.590107+0000","last_fullsized":"2024-05-03T14:46:05.590107+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:26:15.764522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.108515+0000","last_change":"2024-05-03T14:44:31.461320+0000","last_active":"2024-05-03T14:46:06.108515+0000","last_peered":"2024-05-03T14:46:06.108515+0000","last_clean":"2024-05-03T14:46:06.108515+0000","last_became_active":"2024-05-03T14:44:31.460935+0000","last_became_peered":"2024-05-03T14:44:31.460935+0000","last_unstale":"2024-05-03T14:46:06.108515+0000","last_undegraded":"2024-05-03T14:46:06.108515+0000","last_fullsized":"2024-05-03T14:46:06.108515+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:32:22.368935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110099+0000","last_change":"2024-05-03T14:44:31.452694+0000","last_active":"2024-05-03T14:46:06.110099+0000","last_peered":"2024-05-03T14:46:06.110099+0000","last_clean":"2024-05-03T14:46:06.110099+0000","last_became_active":"2024-05-03T14:44:31.452012+0000","last_became_peered":"2024-05-03T14:44:31.452012+0000","last_unstale":"2024-05-03T14:46:06.110099+0000","last_undegraded":"2024-05-03T14:46:06.110099+0000","last_fullsized":"2024-05-03T14:46:06.110099+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:59:34.721990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.591501+0000","last_change":"2024-05-03T14:44:31.461029+0000","last_active":"2024-05-03T14:46:05.591501+0000","last_peered":"2024-05-03T14:46:05.591501+0000","last_clean":"2024-05-03T14:46:05.591501+0000","last_became_active":"2024-05-03T14:44:31.460506+0000","last_became_peered":"2024-05-03T14:44:31.460506+0000","last_unstale":"2024-05-03T14:46:05.591501+0000","last_undegraded":"2024-05-03T14:46:05.591501+0000","last_fullsized":"2024-05-03T14:46:05.591501+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:09:23.903606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.591580+0000","last_change":"2024-05-03T14:44:31.467556+0000","last_active":"2024-05-03T14:46:05.591580+0000","last_peered":"2024-05-03T14:46:05.591580+0000","last_clean":"2024-05-03T14:46:05.591580+0000","last_became_active":"2024-05-03T14:44:31.467191+0000","last_became_peered":"2024-05-03T14:44:31.467191+0000","last_unstale":"2024-05-03T14:46:05.591580+0000","last_undegraded":"2024-05-03T14:46:05.591580+0000","last_fullsized":"2024-05-03T14:46:05.591580+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:33:42.980620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.591049+0000","last_change":"2024-05-03T14:45:05.618172+0000","last_active":"2024-05-03T14:46:05.591049+0000","last_peered":"2024-05-03T14:46:05.591049+0000","last_clean":"2024-05-03T14:46:05.591049+0000","last_became_active":"2024-05-03T14:45:05.617555+0000","last_became_peered":"2024-05-03T14:45:05.617555+0000","last_unstale":"2024-05-03T14:46:05.591049+0000","last_undegraded":"2024-05-03T14:46:05.591049+0000","last_fullsized":"2024-05-03T14:46:05.591049+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T16:44:40.610265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'13","reported_seq":114,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:32.176164+0000","last_change":"2024-05-03T14:46:06.129243+0000","last_active":"2024-05-03T14:46:32.176164+0000","last_peered":"2024-05-03T14:46:32.176164+0000","last_clean":"2024-05-03T14:46:32.176164+0000","last_became_active":"2024-05-03T14:46:06.112952+0000","last_became_peered":"2024-05-03T14:46:06.112952+0000","last_unstale":"2024-05-03T14:46:32.176164+0000","last_undegraded":"2024-05-03T14:46:32.176164+0000","last_fullsized":"2024-05-03T14:46:32.176164+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:12:22.401171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":34,"num_read_kb":34,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.113593+0000","last_change":"2024-05-03T14:44:31.467383+0000","last_active":"2024-05-03T14:46:06.113593+0000","last_peered":"2024-05-03T14:46:06.113593+0000","last_clean":"2024-05-03T14:46:06.113593+0000","last_became_active":"2024-05-03T14:44:31.466993+0000","last_became_peered":"2024-05-03T14:44:31.466993+0000","last_unstale":"2024-05-03T14:46:06.113593+0000","last_undegraded":"2024-05-03T14:46:06.113593+0000","last_fullsized":"2024-05-03T14:46:06.113593+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T16:43:50.237083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.108877+0000","last_change":"2024-05-03T14:44:44.583870+0000","last_active":"2024-05-03T14:46:06.108877+0000","last_peered":"2024-05-03T14:46:06.108877+0000","last_clean":"2024-05-03T14:46:06.108877+0000","last_became_active":"2024-05-03T14:44:44.583296+0000","last_became_peered":"2024-05-03T14:44:44.583296+0000","last_unstale":"2024-05-03T14:46:06.108877+0000","last_undegraded":"2024-05-03T14:46:06.108877+0000","last_fullsized":"2024-05-03T14:46:06.108877+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:55:54.345707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.115090+0000","last_change":"2024-05-03T14:44:27.415797+0000","last_active":"2024-05-03T14:46:06.115090+0000","last_peered":"2024-05-03T14:46:06.115090+0000","last_clean":"2024-05-03T14:46:06.115090+0000","last_became_active":"2024-05-03T14:44:27.414960+0000","last_became_peered":"2024-05-03T14:44:27.414960+0000","last_unstale":"2024-05-03T14:46:06.115090+0000","last_undegraded":"2024-05-03T14:46:06.115090+0000","last_fullsized":"2024-05-03T14:46:06.115090+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:15:54.374392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.590908+0000","last_change":"2024-05-03T14:44:31.470241+0000","last_active":"2024-05-03T14:46:05.590908+0000","last_peered":"2024-05-03T14:46:05.590908+0000","last_clean":"2024-05-03T14:46:05.590908+0000","last_became_active":"2024-05-03T14:44:31.469882+0000","last_became_peered":"2024-05-03T14:44:31.469882+0000","last_unstale":"2024-05-03T14:46:05.590908+0000","last_undegraded":"2024-05-03T14:46:05.590908+0000","last_fullsized":"2024-05-03T14:46:05.590908+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:40:43.970897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109037+0000","last_change":"2024-05-03T14:44:44.583776+0000","last_active":"2024-05-03T14:46:06.109037+0000","last_peered":"2024-05-03T14:46:06.109037+0000","last_clean":"2024-05-03T14:46:06.109037+0000","last_became_active":"2024-05-03T14:44:44.583117+0000","last_became_peered":"2024-05-03T14:44:44.583117+0000","last_unstale":"2024-05-03T14:46:06.109037+0000","last_undegraded":"2024-05-03T14:46:06.109037+0000","last_fullsized":"2024-05-03T14:46:06.109037+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:58:49.428720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":57,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.594170+0000","last_change":"2024-05-03T14:44:27.404806+0000","last_active":"2024-05-03T14:46:05.594170+0000","last_peered":"2024-05-03T14:46:05.594170+0000","last_clean":"2024-05-03T14:46:05.594170+0000","last_became_active":"2024-05-03T14:44:27.404415+0000","last_became_peered":"2024-05-03T14:44:27.404415+0000","last_unstale":"2024-05-03T14:46:05.594170+0000","last_undegraded":"2024-05-03T14:46:05.594170+0000","last_fullsized":"2024-05-03T14:46:05.594170+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:04:40.569591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.108822+0000","last_change":"2024-05-03T14:44:31.465714+0000","last_active":"2024-05-03T14:46:06.108822+0000","last_peered":"2024-05-03T14:46:06.108822+0000","last_clean":"2024-05-03T14:46:06.108822+0000","last_became_active":"2024-05-03T14:44:31.465264+0000","last_became_peered":"2024-05-03T14:44:31.465264+0000","last_unstale":"2024-05-03T14:46:06.108822+0000","last_undegraded":"2024-05-03T14:46:06.108822+0000","last_fullsized":"2024-05-03T14:46:06.108822+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:19:38.206564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109120+0000","last_change":"2024-05-03T14:44:44.583611+0000","last_active":"2024-05-03T14:46:06.109120+0000","last_peered":"2024-05-03T14:46:06.109120+0000","last_clean":"2024-05-03T14:46:06.109120+0000","last_became_active":"2024-05-03T14:44:44.583149+0000","last_became_peered":"2024-05-03T14:44:44.583149+0000","last_unstale":"2024-05-03T14:46:06.109120+0000","last_undegraded":"2024-05-03T14:46:06.109120+0000","last_fullsized":"2024-05-03T14:46:06.109120+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:25:23.550168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.593940+0000","last_change":"2024-05-03T14:44:27.405942+0000","last_active":"2024-05-03T14:46:05.593940+0000","last_peered":"2024-05-03T14:46:05.593940+0000","last_clean":"2024-05-03T14:46:05.593940+0000","last_became_active":"2024-05-03T14:44:27.405556+0000","last_became_peered":"2024-05-03T14:44:27.405556+0000","last_unstale":"2024-05-03T14:46:05.593940+0000","last_undegraded":"2024-05-03T14:46:05.593940+0000","last_fullsized":"2024-05-03T14:46:05.593940+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:16:07.798364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110278+0000","last_change":"2024-05-03T14:44:31.451421+0000","last_active":"2024-05-03T14:46:06.110278+0000","last_peered":"2024-05-03T14:46:06.110278+0000","last_clean":"2024-05-03T14:46:06.110278+0000","last_became_active":"2024-05-03T14:44:31.451072+0000","last_became_peered":"2024-05-03T14:44:31.451072+0000","last_unstale":"2024-05-03T14:46:06.110278+0000","last_undegraded":"2024-05-03T14:46:06.110278+0000","last_fullsized":"2024-05-03T14:46:06.110278+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:12:31.040518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.591370+0000","last_change":"2024-05-03T14:44:44.567926+0000","last_active":"2024-05-03T14:46:05.591370+0000","last_peered":"2024-05-03T14:46:05.591370+0000","last_clean":"2024-05-03T14:46:05.591370+0000","last_became_active":"2024-05-03T14:44:44.567494+0000","last_became_peered":"2024-05-03T14:44:44.567494+0000","last_unstale":"2024-05-03T14:46:05.591370+0000","last_undegraded":"2024-05-03T14:46:05.591370+0000","last_fullsized":"2024-05-03T14:46:05.591370+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:51:11.144925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'5","reported_seq":94,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:32.597275+0000","last_change":"2024-05-03T14:45:05.614704+0000","last_active":"2024-05-03T14:46:32.597275+0000","last_peered":"2024-05-03T14:46:32.597275+0000","last_clean":"2024-05-03T14:46:32.597275+0000","last_became_active":"2024-05-03T14:45:05.614173+0000","last_became_peered":"2024-05-03T14:45:05.614173+0000","last_unstale":"2024-05-03T14:46:32.597275+0000","last_undegraded":"2024-05-03T14:46:32.597275+0000","last_fullsized":"2024-05-03T14:46:32.597275+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:50:42.956239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":39,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.593674+0000","last_change":"2024-05-03T14:44:31.464666+0000","last_active":"2024-05-03T14:46:05.593674+0000","last_peered":"2024-05-03T14:46:05.593674+0000","last_clean":"2024-05-03T14:46:05.593674+0000","last_became_active":"2024-05-03T14:44:31.464298+0000","last_became_peered":"2024-05-03T14:44:31.464298+0000","last_unstale":"2024-05-03T14:46:05.593674+0000","last_undegraded":"2024-05-03T14:46:05.593674+0000","last_fullsized":"2024-05-03T14:46:05.593674+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:26:35.425908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.591268+0000","last_change":"2024-05-03T14:44:44.580268+0000","last_active":"2024-05-03T14:46:05.591268+0000","last_peered":"2024-05-03T14:46:05.591268+0000","last_clean":"2024-05-03T14:46:05.591268+0000","last_became_active":"2024-05-03T14:44:44.579617+0000","last_became_peered":"2024-05-03T14:44:44.579617+0000","last_unstale":"2024-05-03T14:46:05.591268+0000","last_undegraded":"2024-05-03T14:46:05.591268+0000","last_fullsized":"2024-05-03T14:46:05.591268+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:16:25.580698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.608061+0000","last_change":"2024-05-03T14:44:27.418578+0000","last_active":"2024-05-03T14:46:05.608061+0000","last_peered":"2024-05-03T14:46:05.608061+0000","last_clean":"2024-05-03T14:46:05.608061+0000","last_became_active":"2024-05-03T14:44:27.418206+0000","last_became_peered":"2024-05-03T14:44:27.418206+0000","last_unstale":"2024-05-03T14:46:05.608061+0000","last_undegraded":"2024-05-03T14:46:05.608061+0000","last_fullsized":"2024-05-03T14:46:05.608061+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:48:47.085451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110188+0000","last_change":"2024-05-03T14:44:31.467353+0000","last_active":"2024-05-03T14:46:06.110188+0000","last_peered":"2024-05-03T14:46:06.110188+0000","last_clean":"2024-05-03T14:46:06.110188+0000","last_became_active":"2024-05-03T14:44:31.466606+0000","last_became_peered":"2024-05-03T14:44:31.466606+0000","last_unstale":"2024-05-03T14:46:06.110188+0000","last_undegraded":"2024-05-03T14:46:06.110188+0000","last_fullsized":"2024-05-03T14:46:06.110188+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T14:50:22.624015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109972+0000","last_change":"2024-05-03T14:44:44.567367+0000","last_active":"2024-05-03T14:46:06.109972+0000","last_peered":"2024-05-03T14:46:06.109972+0000","last_clean":"2024-05-03T14:46:06.109972+0000","last_became_active":"2024-05-03T14:44:44.566916+0000","last_became_peered":"2024-05-03T14:44:44.566916+0000","last_unstale":"2024-05-03T14:46:06.109972+0000","last_undegraded":"2024-05-03T14:46:06.109972+0000","last_fullsized":"2024-05-03T14:46:06.109972+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:07:39.966644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110169+0000","last_change":"2024-05-03T14:44:27.424704+0000","last_active":"2024-05-03T14:46:06.110169+0000","last_peered":"2024-05-03T14:46:06.110169+0000","last_clean":"2024-05-03T14:46:06.110169+0000","last_became_active":"2024-05-03T14:44:27.424231+0000","last_became_peered":"2024-05-03T14:44:27.424231+0000","last_unstale":"2024-05-03T14:46:06.110169+0000","last_undegraded":"2024-05-03T14:46:06.110169+0000","last_fullsized":"2024-05-03T14:46:06.110169+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:00:22.076458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.113124+0000","last_change":"2024-05-03T14:44:31.474599+0000","last_active":"2024-05-03T14:46:06.113124+0000","last_peered":"2024-05-03T14:46:06.113124+0000","last_clean":"2024-05-03T14:46:06.113124+0000","last_became_active":"2024-05-03T14:44:31.473970+0000","last_became_peered":"2024-05-03T14:44:31.473970+0000","last_unstale":"2024-05-03T14:46:06.113124+0000","last_undegraded":"2024-05-03T14:46:06.113124+0000","last_fullsized":"2024-05-03T14:46:06.113124+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:20:14.259534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.590230+0000","last_change":"2024-05-03T14:44:44.580126+0000","last_active":"2024-05-03T14:46:05.590230+0000","last_peered":"2024-05-03T14:46:05.590230+0000","last_clean":"2024-05-03T14:46:05.590230+0000","last_became_active":"2024-05-03T14:44:44.578708+0000","last_became_peered":"2024-05-03T14:44:44.578708+0000","last_unstale":"2024-05-03T14:46:05.590230+0000","last_undegraded":"2024-05-03T14:46:05.590230+0000","last_fullsized":"2024-05-03T14:46:05.590230+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:42:43.279786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109173+0000","last_change":"2024-05-03T14:44:27.405332+0000","last_active":"2024-05-03T14:46:06.109173+0000","last_peered":"2024-05-03T14:46:06.109173+0000","last_clean":"2024-05-03T14:46:06.109173+0000","last_became_active":"2024-05-03T14:44:27.404994+0000","last_became_peered":"2024-05-03T14:44:27.404994+0000","last_unstale":"2024-05-03T14:46:06.109173+0000","last_undegraded":"2024-05-03T14:46:06.109173+0000","last_fullsized":"2024-05-03T14:46:06.109173+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:18:23.716014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.590783+0000","last_change":"2024-05-03T14:44:31.470396+0000","last_active":"2024-05-03T14:46:05.590783+0000","last_peered":"2024-05-03T14:46:05.590783+0000","last_clean":"2024-05-03T14:46:05.590783+0000","last_became_active":"2024-05-03T14:44:31.467973+0000","last_became_peered":"2024-05-03T14:44:31.467973+0000","last_unstale":"2024-05-03T14:46:05.590783+0000","last_undegraded":"2024-05-03T14:46:05.590783+0000","last_fullsized":"2024-05-03T14:46:05.590783+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:44:21.266589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.113100+0000","last_change":"2024-05-03T14:44:44.582678+0000","last_active":"2024-05-03T14:46:06.113100+0000","last_peered":"2024-05-03T14:46:06.113100+0000","last_clean":"2024-05-03T14:46:06.113100+0000","last_became_active":"2024-05-03T14:44:44.582155+0000","last_became_peered":"2024-05-03T14:44:44.582155+0000","last_unstale":"2024-05-03T14:46:06.113100+0000","last_undegraded":"2024-05-03T14:46:06.113100+0000","last_fullsized":"2024-05-03T14:46:06.113100+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T15:21:05.986627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":124,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:34.599009+0000","last_change":"2024-05-03T14:44:27.415880+0000","last_active":"2024-05-03T14:46:34.599009+0000","last_peered":"2024-05-03T14:46:34.599009+0000","last_clean":"2024-05-03T14:46:34.599009+0000","last_became_active":"2024-05-03T14:44:27.415354+0000","last_became_peered":"2024-05-03T14:44:27.415354+0000","last_unstale":"2024-05-03T14:46:34.599009+0000","last_undegraded":"2024-05-03T14:46:34.599009+0000","last_fullsized":"2024-05-03T14:46:34.599009+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T16:55:41.808314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":76,"num_read_kb":76,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109516+0000","last_change":"2024-05-03T14:44:31.462685+0000","last_active":"2024-05-03T14:46:06.109516+0000","last_peered":"2024-05-03T14:46:06.109516+0000","last_clean":"2024-05-03T14:46:06.109516+0000","last_became_active":"2024-05-03T14:44:31.462325+0000","last_became_peered":"2024-05-03T14:44:31.462325+0000","last_unstale":"2024-05-03T14:46:06.109516+0000","last_undegraded":"2024-05-03T14:46:06.109516+0000","last_fullsized":"2024-05-03T14:46:06.109516+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:59:54.715648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.590180+0000","last_change":"2024-05-03T14:44:44.573855+0000","last_active":"2024-05-03T14:46:05.590180+0000","last_peered":"2024-05-03T14:46:05.590180+0000","last_clean":"2024-05-03T14:46:05.590180+0000","last_became_active":"2024-05-03T14:44:44.573314+0000","last_became_peered":"2024-05-03T14:44:44.573314+0000","last_unstale":"2024-05-03T14:46:05.590180+0000","last_undegraded":"2024-05-03T14:46:05.590180+0000","last_fullsized":"2024-05-03T14:46:05.590180+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:04:24.282503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.594812+0000","last_change":"2024-05-03T14:45:05.613037+0000","last_active":"2024-05-03T14:46:05.594812+0000","last_peered":"2024-05-03T14:46:05.594812+0000","last_clean":"2024-05-03T14:46:05.594812+0000","last_became_active":"2024-05-03T14:45:05.612581+0000","last_became_peered":"2024-05-03T14:45:05.612581+0000","last_unstale":"2024-05-03T14:46:05.594812+0000","last_undegraded":"2024-05-03T14:46:05.594812+0000","last_fullsized":"2024-05-03T14:46:05.594812+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:34:07.365447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.590800+0000","last_change":"2024-05-03T14:44:31.451912+0000","last_active":"2024-05-03T14:46:05.590800+0000","last_peered":"2024-05-03T14:46:05.590800+0000","last_clean":"2024-05-03T14:46:05.590800+0000","last_became_active":"2024-05-03T14:44:31.449029+0000","last_became_peered":"2024-05-03T14:44:31.449029+0000","last_unstale":"2024-05-03T14:46:05.590800+0000","last_undegraded":"2024-05-03T14:46:05.590800+0000","last_fullsized":"2024-05-03T14:46:05.590800+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:27:13.018719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109400+0000","last_change":"2024-05-03T14:44:44.577001+0000","last_active":"2024-05-03T14:46:06.109400+0000","last_peered":"2024-05-03T14:46:06.109400+0000","last_clean":"2024-05-03T14:46:06.109400+0000","last_became_active":"2024-05-03T14:44:44.576224+0000","last_became_peered":"2024-05-03T14:44:44.576224+0000","last_unstale":"2024-05-03T14:46:06.109400+0000","last_undegraded":"2024-05-03T14:46:06.109400+0000","last_fullsized":"2024-05-03T14:46:06.109400+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T15:15:30.497801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110238+0000","last_change":"2024-05-03T14:44:27.402496+0000","last_active":"2024-05-03T14:46:06.110238+0000","last_peered":"2024-05-03T14:46:06.110238+0000","last_clean":"2024-05-03T14:46:06.110238+0000","last_became_active":"2024-05-03T14:44:27.402062+0000","last_became_peered":"2024-05-03T14:44:27.402062+0000","last_unstale":"2024-05-03T14:46:06.110238+0000","last_undegraded":"2024-05-03T14:46:06.110238+0000","last_fullsized":"2024-05-03T14:46:06.110238+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:47:10.090902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.108858+0000","last_change":"2024-05-03T14:44:31.452654+0000","last_active":"2024-05-03T14:46:06.108858+0000","last_peered":"2024-05-03T14:46:06.108858+0000","last_clean":"2024-05-03T14:46:06.108858+0000","last_became_active":"2024-05-03T14:44:31.452305+0000","last_became_peered":"2024-05-03T14:44:31.452305+0000","last_unstale":"2024-05-03T14:46:06.108858+0000","last_undegraded":"2024-05-03T14:46:06.108858+0000","last_fullsized":"2024-05-03T14:46:06.108858+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:20:45.816113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.591172+0000","last_change":"2024-05-03T14:44:44.576004+0000","last_active":"2024-05-03T14:46:05.591172+0000","last_peered":"2024-05-03T14:46:05.591172+0000","last_clean":"2024-05-03T14:46:05.591172+0000","last_became_active":"2024-05-03T14:44:44.572486+0000","last_became_peered":"2024-05-03T14:44:44.572486+0000","last_unstale":"2024-05-03T14:46:05.591172+0000","last_undegraded":"2024-05-03T14:46:05.591172+0000","last_fullsized":"2024-05-03T14:46:05.591172+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T15:11:51.125404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'36","reported_seq":165,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:34.599056+0000","last_change":"2024-05-03T14:44:27.405937+0000","last_active":"2024-05-03T14:46:34.599056+0000","last_peered":"2024-05-03T14:46:34.599056+0000","last_clean":"2024-05-03T14:46:34.599056+0000","last_became_active":"2024-05-03T14:44:27.405511+0000","last_became_peered":"2024-05-03T14:44:27.405511+0000","last_unstale":"2024-05-03T14:46:34.599056+0000","last_undegraded":"2024-05-03T14:46:34.599056+0000","last_fullsized":"2024-05-03T14:46:34.599056+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:00:56.824015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58916,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":85,"num_read_kb":100,"num_write":38,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.594060+0000","last_change":"2024-05-03T14:44:31.448712+0000","last_active":"2024-05-03T14:46:05.594060+0000","last_peered":"2024-05-03T14:46:05.594060+0000","last_clean":"2024-05-03T14:46:05.594060+0000","last_became_active":"2024-05-03T14:44:31.448346+0000","last_became_peered":"2024-05-03T14:44:31.448346+0000","last_unstale":"2024-05-03T14:46:05.594060+0000","last_undegraded":"2024-05-03T14:46:05.594060+0000","last_fullsized":"2024-05-03T14:46:05.594060+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T16:32:40.770487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110612+0000","last_change":"2024-05-03T14:44:44.585600+0000","last_active":"2024-05-03T14:46:06.110612+0000","last_peered":"2024-05-03T14:46:06.110612+0000","last_clean":"2024-05-03T14:46:06.110612+0000","last_became_active":"2024-05-03T14:44:44.585221+0000","last_became_peered":"2024-05-03T14:44:44.585221+0000","last_unstale":"2024-05-03T14:46:06.110612+0000","last_undegraded":"2024-05-03T14:46:06.110612+0000","last_fullsized":"2024-05-03T14:46:06.110612+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:28:45.804470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.591282+0000","last_change":"2024-05-03T14:44:27.416524+0000","last_active":"2024-05-03T14:46:05.591282+0000","last_peered":"2024-05-03T14:46:05.591282+0000","last_clean":"2024-05-03T14:46:05.591282+0000","last_became_active":"2024-05-03T14:44:27.416211+0000","last_became_peered":"2024-05-03T14:44:27.416211+0000","last_unstale":"2024-05-03T14:46:05.591282+0000","last_undegraded":"2024-05-03T14:46:05.591282+0000","last_fullsized":"2024-05-03T14:46:05.591282+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:00:26.361267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.111040+0000","last_change":"2024-05-03T14:44:31.468973+0000","last_active":"2024-05-03T14:46:06.111040+0000","last_peered":"2024-05-03T14:46:06.111040+0000","last_clean":"2024-05-03T14:46:06.111040+0000","last_became_active":"2024-05-03T14:44:31.468547+0000","last_became_peered":"2024-05-03T14:44:31.468547+0000","last_unstale":"2024-05-03T14:46:06.111040+0000","last_undegraded":"2024-05-03T14:46:06.111040+0000","last_fullsized":"2024-05-03T14:46:06.111040+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:16:16.223859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109286+0000","last_change":"2024-05-03T14:44:44.579157+0000","last_active":"2024-05-03T14:46:06.109286+0000","last_peered":"2024-05-03T14:46:06.109286+0000","last_clean":"2024-05-03T14:46:06.109286+0000","last_became_active":"2024-05-03T14:44:44.578718+0000","last_became_peered":"2024-05-03T14:44:44.578718+0000","last_unstale":"2024-05-03T14:46:06.109286+0000","last_undegraded":"2024-05-03T14:46:06.109286+0000","last_fullsized":"2024-05-03T14:46:06.109286+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:03:31.574958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110029+0000","last_change":"2024-05-03T14:44:27.415290+0000","last_active":"2024-05-03T14:46:06.110029+0000","last_peered":"2024-05-03T14:46:06.110029+0000","last_clean":"2024-05-03T14:46:06.110029+0000","last_became_active":"2024-05-03T14:44:27.414930+0000","last_became_peered":"2024-05-03T14:44:27.414930+0000","last_unstale":"2024-05-03T14:46:06.110029+0000","last_undegraded":"2024-05-03T14:46:06.110029+0000","last_fullsized":"2024-05-03T14:46:06.110029+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:06:40.675902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.594341+0000","last_change":"2024-05-03T14:44:31.464631+0000","last_active":"2024-05-03T14:46:05.594341+0000","last_peered":"2024-05-03T14:46:05.594341+0000","last_clean":"2024-05-03T14:46:05.594341+0000","last_became_active":"2024-05-03T14:44:31.464325+0000","last_became_peered":"2024-05-03T14:44:31.464325+0000","last_unstale":"2024-05-03T14:46:05.594341+0000","last_undegraded":"2024-05-03T14:46:05.594341+0000","last_fullsized":"2024-05-03T14:46:05.594341+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:09:39.140391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.591327+0000","last_change":"2024-05-03T14:44:44.583495+0000","last_active":"2024-05-03T14:46:05.591327+0000","last_peered":"2024-05-03T14:46:05.591327+0000","last_clean":"2024-05-03T14:46:05.591327+0000","last_became_active":"2024-05-03T14:44:44.582971+0000","last_became_peered":"2024-05-03T14:44:44.582971+0000","last_unstale":"2024-05-03T14:46:05.591327+0000","last_undegraded":"2024-05-03T14:46:05.591327+0000","last_fullsized":"2024-05-03T14:46:05.591327+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:23:12.086610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"87'5","reported_seq":88,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:32.620760+0000","last_change":"2024-05-03T14:44:27.403319+0000","last_active":"2024-05-03T14:46:32.620760+0000","last_peered":"2024-05-03T14:46:32.620760+0000","last_clean":"2024-05-03T14:46:32.620760+0000","last_became_active":"2024-05-03T14:44:27.402936+0000","last_became_peered":"2024-05-03T14:44:27.402936+0000","last_unstale":"2024-05-03T14:46:32.620760+0000","last_undegraded":"2024-05-03T14:46:32.620760+0000","last_fullsized":"2024-05-03T14:46:32.620760+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:06:24.851356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":40,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.594223+0000","last_change":"2024-05-03T14:44:31.453514+0000","last_active":"2024-05-03T14:46:05.594223+0000","last_peered":"2024-05-03T14:46:05.594223+0000","last_clean":"2024-05-03T14:46:05.594223+0000","last_became_active":"2024-05-03T14:44:31.452755+0000","last_became_peered":"2024-05-03T14:44:31.452755+0000","last_unstale":"2024-05-03T14:46:05.594223+0000","last_undegraded":"2024-05-03T14:46:05.594223+0000","last_fullsized":"2024-05-03T14:46:05.594223+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:29:45.095487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.113797+0000","last_change":"2024-05-03T14:44:44.587105+0000","last_active":"2024-05-03T14:46:06.113797+0000","last_peered":"2024-05-03T14:46:06.113797+0000","last_clean":"2024-05-03T14:46:06.113797+0000","last_became_active":"2024-05-03T14:44:44.586669+0000","last_became_peered":"2024-05-03T14:44:44.586669+0000","last_unstale":"2024-05-03T14:46:06.113797+0000","last_undegraded":"2024-05-03T14:46:06.113797+0000","last_fullsized":"2024-05-03T14:46:06.113797+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:18:16.364544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":62,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.593578+0000","last_change":"2024-05-03T14:44:27.412481+0000","last_active":"2024-05-03T14:46:05.593578+0000","last_peered":"2024-05-03T14:46:05.593578+0000","last_clean":"2024-05-03T14:46:05.593578+0000","last_became_active":"2024-05-03T14:44:27.412146+0000","last_became_peered":"2024-05-03T14:44:27.412146+0000","last_unstale":"2024-05-03T14:46:05.593578+0000","last_undegraded":"2024-05-03T14:46:05.593578+0000","last_fullsized":"2024-05-03T14:46:05.593578+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T16:03:40.205892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109727+0000","last_change":"2024-05-03T14:44:31.453208+0000","last_active":"2024-05-03T14:46:06.109727+0000","last_peered":"2024-05-03T14:46:06.109727+0000","last_clean":"2024-05-03T14:46:06.109727+0000","last_became_active":"2024-05-03T14:44:31.452788+0000","last_became_peered":"2024-05-03T14:44:31.452788+0000","last_unstale":"2024-05-03T14:46:06.109727+0000","last_undegraded":"2024-05-03T14:46:06.109727+0000","last_fullsized":"2024-05-03T14:46:06.109727+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:43:31.554510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109677+0000","last_change":"2024-05-03T14:44:44.582993+0000","last_active":"2024-05-03T14:46:06.109677+0000","last_peered":"2024-05-03T14:46:06.109677+0000","last_clean":"2024-05-03T14:46:06.109677+0000","last_became_active":"2024-05-03T14:44:44.582511+0000","last_became_peered":"2024-05-03T14:44:44.582511+0000","last_unstale":"2024-05-03T14:46:06.109677+0000","last_undegraded":"2024-05-03T14:46:06.109677+0000","last_fullsized":"2024-05-03T14:46:06.109677+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:22:26.380344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.594298+0000","last_change":"2024-05-03T14:44:27.401742+0000","last_active":"2024-05-03T14:46:05.594298+0000","last_peered":"2024-05-03T14:46:05.594298+0000","last_clean":"2024-05-03T14:46:05.594298+0000","last_became_active":"2024-05-03T14:44:27.401347+0000","last_became_peered":"2024-05-03T14:44:27.401347+0000","last_unstale":"2024-05-03T14:46:05.594298+0000","last_undegraded":"2024-05-03T14:46:05.594298+0000","last_fullsized":"2024-05-03T14:46:05.594298+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T15:39:41.144999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109323+0000","last_change":"2024-05-03T14:44:31.453480+0000","last_active":"2024-05-03T14:46:06.109323+0000","last_peered":"2024-05-03T14:46:06.109323+0000","last_clean":"2024-05-03T14:46:06.109323+0000","last_became_active":"2024-05-03T14:44:31.453171+0000","last_became_peered":"2024-05-03T14:44:31.453171+0000","last_unstale":"2024-05-03T14:46:06.109323+0000","last_undegraded":"2024-05-03T14:46:06.109323+0000","last_fullsized":"2024-05-03T14:46:06.109323+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:30:16.567938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.593599+0000","last_change":"2024-05-03T14:44:45.208587+0000","last_active":"2024-05-03T14:46:05.593599+0000","last_peered":"2024-05-03T14:46:05.593599+0000","last_clean":"2024-05-03T14:46:05.593599+0000","last_became_active":"2024-05-03T14:44:45.207987+0000","last_became_peered":"2024-05-03T14:44:45.207987+0000","last_unstale":"2024-05-03T14:46:05.593599+0000","last_undegraded":"2024-05-03T14:46:05.593599+0000","last_fullsized":"2024-05-03T14:46:05.593599+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:22:14.854390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.593134+0000","last_change":"2024-05-03T14:44:27.421333+0000","last_active":"2024-05-03T14:46:05.593134+0000","last_peered":"2024-05-03T14:46:05.593134+0000","last_clean":"2024-05-03T14:46:05.593134+0000","last_became_active":"2024-05-03T14:44:27.419745+0000","last_became_peered":"2024-05-03T14:44:27.419745+0000","last_unstale":"2024-05-03T14:46:05.593134+0000","last_undegraded":"2024-05-03T14:46:05.593134+0000","last_fullsized":"2024-05-03T14:46:05.593134+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:40:57.258260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.608091+0000","last_change":"2024-05-03T14:44:31.467871+0000","last_active":"2024-05-03T14:46:05.608091+0000","last_peered":"2024-05-03T14:46:05.608091+0000","last_clean":"2024-05-03T14:46:05.608091+0000","last_became_active":"2024-05-03T14:44:31.467408+0000","last_became_peered":"2024-05-03T14:44:31.467408+0000","last_unstale":"2024-05-03T14:46:05.608091+0000","last_undegraded":"2024-05-03T14:46:05.608091+0000","last_fullsized":"2024-05-03T14:46:05.608091+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:19:24.843591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110417+0000","last_change":"2024-05-03T14:44:44.583116+0000","last_active":"2024-05-03T14:46:06.110417+0000","last_peered":"2024-05-03T14:46:06.110417+0000","last_clean":"2024-05-03T14:46:06.110417+0000","last_became_active":"2024-05-03T14:44:44.581838+0000","last_became_peered":"2024-05-03T14:44:44.581838+0000","last_unstale":"2024-05-03T14:46:06.110417+0000","last_undegraded":"2024-05-03T14:46:06.110417+0000","last_fullsized":"2024-05-03T14:46:06.110417+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:14:24.608777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'5","reported_seq":89,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:35.184476+0000","last_change":"2024-05-03T14:44:27.416068+0000","last_active":"2024-05-03T14:46:35.184476+0000","last_peered":"2024-05-03T14:46:35.184476+0000","last_clean":"2024-05-03T14:46:35.184476+0000","last_became_active":"2024-05-03T14:44:27.415574+0000","last_became_peered":"2024-05-03T14:44:27.415574+0000","last_unstale":"2024-05-03T14:46:35.184476+0000","last_undegraded":"2024-05-03T14:46:35.184476+0000","last_fullsized":"2024-05-03T14:46:35.184476+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T15:31:40.186754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":39,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.608007+0000","last_change":"2024-05-03T14:44:31.453935+0000","last_active":"2024-05-03T14:46:05.608007+0000","last_peered":"2024-05-03T14:46:05.608007+0000","last_clean":"2024-05-03T14:46:05.608007+0000","last_became_active":"2024-05-03T14:44:31.453565+0000","last_became_peered":"2024-05-03T14:44:31.453565+0000","last_unstale":"2024-05-03T14:46:05.608007+0000","last_undegraded":"2024-05-03T14:46:05.608007+0000","last_fullsized":"2024-05-03T14:46:05.608007+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:34:04.855830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.111343+0000","last_change":"2024-05-03T14:44:45.210274+0000","last_active":"2024-05-03T14:46:06.111343+0000","last_peered":"2024-05-03T14:46:06.111343+0000","last_clean":"2024-05-03T14:46:06.111343+0000","last_became_active":"2024-05-03T14:44:45.208951+0000","last_became_peered":"2024-05-03T14:44:45.208951+0000","last_unstale":"2024-05-03T14:46:06.111343+0000","last_undegraded":"2024-05-03T14:46:06.111343+0000","last_fullsized":"2024-05-03T14:46:06.111343+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:56:30.953019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109801+0000","last_change":"2024-05-03T14:44:31.449470+0000","last_active":"2024-05-03T14:46:06.109801+0000","last_peered":"2024-05-03T14:46:06.109801+0000","last_clean":"2024-05-03T14:46:06.109801+0000","last_became_active":"2024-05-03T14:44:31.449031+0000","last_became_peered":"2024-05-03T14:44:31.449031+0000","last_unstale":"2024-05-03T14:46:06.109801+0000","last_undegraded":"2024-05-03T14:46:06.109801+0000","last_fullsized":"2024-05-03T14:46:06.109801+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T14:51:54.576685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":89,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:34.623093+0000","last_change":"2024-05-03T14:44:27.422378+0000","last_active":"2024-05-03T14:46:34.623093+0000","last_peered":"2024-05-03T14:46:34.623093+0000","last_clean":"2024-05-03T14:46:34.623093+0000","last_became_active":"2024-05-03T14:44:27.421988+0000","last_became_peered":"2024-05-03T14:44:27.421988+0000","last_unstale":"2024-05-03T14:46:34.623093+0000","last_undegraded":"2024-05-03T14:46:34.623093+0000","last_fullsized":"2024-05-03T14:46:34.623093+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:20:10.371418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":39,"num_read_kb":39,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.594688+0000","last_change":"2024-05-03T14:44:44.576084+0000","last_active":"2024-05-03T14:46:05.594688+0000","last_peered":"2024-05-03T14:46:05.594688+0000","last_clean":"2024-05-03T14:46:05.594688+0000","last_became_active":"2024-05-03T14:44:44.575633+0000","last_became_peered":"2024-05-03T14:44:44.575633+0000","last_unstale":"2024-05-03T14:46:05.594688+0000","last_undegraded":"2024-05-03T14:46:05.594688+0000","last_fullsized":"2024-05-03T14:46:05.594688+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:35:38.570413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.113711+0000","last_change":"2024-05-03T14:44:31.467024+0000","last_active":"2024-05-03T14:46:06.113711+0000","last_peered":"2024-05-03T14:46:06.113711+0000","last_clean":"2024-05-03T14:46:06.113711+0000","last_became_active":"2024-05-03T14:44:31.466545+0000","last_became_peered":"2024-05-03T14:44:31.466545+0000","last_unstale":"2024-05-03T14:46:06.113711+0000","last_undegraded":"2024-05-03T14:46:06.113711+0000","last_fullsized":"2024-05-03T14:46:06.113711+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:34:41.289638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109321+0000","last_change":"2024-05-03T14:44:27.410463+0000","last_active":"2024-05-03T14:46:06.109321+0000","last_peered":"2024-05-03T14:46:06.109321+0000","last_clean":"2024-05-03T14:46:06.109321+0000","last_became_active":"2024-05-03T14:44:27.410017+0000","last_became_peered":"2024-05-03T14:44:27.410017+0000","last_unstale":"2024-05-03T14:46:06.109321+0000","last_undegraded":"2024-05-03T14:46:06.109321+0000","last_fullsized":"2024-05-03T14:46:06.109321+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T16:44:40.376214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":119,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.593994+0000","last_change":"2024-05-03T14:42:51.347913+0000","last_active":"2024-05-03T14:46:05.593994+0000","last_peered":"2024-05-03T14:46:05.593994+0000","last_clean":"2024-05-03T14:46:05.593994+0000","last_became_active":"2024-05-03T14:42:51.341302+0000","last_became_peered":"2024-05-03T14:42:51.341302+0000","last_unstale":"2024-05-03T14:46:05.593994+0000","last_undegraded":"2024-05-03T14:46:05.593994+0000","last_fullsized":"2024-05-03T14:46:05.593994+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:38:55.096374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:38:55.096374+0000","last_clean_scrub_stamp":"2024-05-03T14:38:55.096374+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:35:46.708686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110529+0000","last_change":"2024-05-03T14:44:45.209369+0000","last_active":"2024-05-03T14:46:06.110529+0000","last_peered":"2024-05-03T14:46:06.110529+0000","last_clean":"2024-05-03T14:46:06.110529+0000","last_became_active":"2024-05-03T14:44:45.208835+0000","last_became_peered":"2024-05-03T14:44:45.208835+0000","last_unstale":"2024-05-03T14:46:06.110529+0000","last_undegraded":"2024-05-03T14:46:06.110529+0000","last_fullsized":"2024-05-03T14:46:06.110529+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:35:35.343030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109853+0000","last_change":"2024-05-03T14:44:31.462199+0000","last_active":"2024-05-03T14:46:06.109853+0000","last_peered":"2024-05-03T14:46:06.109853+0000","last_clean":"2024-05-03T14:46:06.109853+0000","last_became_active":"2024-05-03T14:44:31.460972+0000","last_became_peered":"2024-05-03T14:44:31.460972+0000","last_unstale":"2024-05-03T14:46:06.109853+0000","last_undegraded":"2024-05-03T14:46:06.109853+0000","last_fullsized":"2024-05-03T14:46:06.109853+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:21:49.619023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.594437+0000","last_change":"2024-05-03T14:44:27.403303+0000","last_active":"2024-05-03T14:46:05.594437+0000","last_peered":"2024-05-03T14:46:05.594437+0000","last_clean":"2024-05-03T14:46:05.594437+0000","last_became_active":"2024-05-03T14:44:27.402870+0000","last_became_peered":"2024-05-03T14:44:27.402870+0000","last_unstale":"2024-05-03T14:46:05.594437+0000","last_undegraded":"2024-05-03T14:46:05.594437+0000","last_fullsized":"2024-05-03T14:46:05.594437+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:57:23.209298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109237+0000","last_change":"2024-05-03T14:44:44.579532+0000","last_active":"2024-05-03T14:46:06.109237+0000","last_peered":"2024-05-03T14:46:06.109237+0000","last_clean":"2024-05-03T14:46:06.109237+0000","last_became_active":"2024-05-03T14:44:44.579052+0000","last_became_peered":"2024-05-03T14:44:44.579052+0000","last_unstale":"2024-05-03T14:46:06.109237+0000","last_undegraded":"2024-05-03T14:46:06.109237+0000","last_fullsized":"2024-05-03T14:46:06.109237+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:07:43.567880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109477+0000","last_change":"2024-05-03T14:45:05.422044+0000","last_active":"2024-05-03T14:46:06.109477+0000","last_peered":"2024-05-03T14:46:06.109477+0000","last_clean":"2024-05-03T14:46:06.109477+0000","last_became_active":"2024-05-03T14:45:05.421557+0000","last_became_peered":"2024-05-03T14:45:05.421557+0000","last_unstale":"2024-05-03T14:46:06.109477+0000","last_undegraded":"2024-05-03T14:46:06.109477+0000","last_fullsized":"2024-05-03T14:46:06.109477+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:44:03.074990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110826+0000","last_change":"2024-05-03T14:44:31.460150+0000","last_active":"2024-05-03T14:46:06.110826+0000","last_peered":"2024-05-03T14:46:06.110826+0000","last_clean":"2024-05-03T14:46:06.110826+0000","last_became_active":"2024-05-03T14:44:31.459550+0000","last_became_peered":"2024-05-03T14:44:31.459550+0000","last_unstale":"2024-05-03T14:46:06.110826+0000","last_undegraded":"2024-05-03T14:46:06.110826+0000","last_fullsized":"2024-05-03T14:46:06.110826+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:28:17.556068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.112955+0000","last_change":"2024-05-03T14:44:45.215035+0000","last_active":"2024-05-03T14:46:06.112955+0000","last_peered":"2024-05-03T14:46:06.112955+0000","last_clean":"2024-05-03T14:46:06.112955+0000","last_became_active":"2024-05-03T14:44:45.208667+0000","last_became_peered":"2024-05-03T14:44:45.208667+0000","last_unstale":"2024-05-03T14:46:06.112955+0000","last_undegraded":"2024-05-03T14:46:06.112955+0000","last_fullsized":"2024-05-03T14:46:06.112955+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:11:47.512884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'13","reported_seq":57,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:17.491776+0000","last_change":"2024-05-03T14:44:27.417377+0000","last_active":"2024-05-03T14:46:17.491776+0000","last_peered":"2024-05-03T14:46:17.491776+0000","last_clean":"2024-05-03T14:46:17.491776+0000","last_became_active":"2024-05-03T14:44:27.416983+0000","last_became_peered":"2024-05-03T14:44:27.416983+0000","last_unstale":"2024-05-03T14:46:17.491776+0000","last_undegraded":"2024-05-03T14:46:17.491776+0000","last_fullsized":"2024-05-03T14:46:17.491776+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T15:37:31.610517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.115229+0000","last_change":"2024-05-03T14:44:31.467316+0000","last_active":"2024-05-03T14:46:06.115229+0000","last_peered":"2024-05-03T14:46:06.115229+0000","last_clean":"2024-05-03T14:46:06.115229+0000","last_became_active":"2024-05-03T14:44:31.466025+0000","last_became_peered":"2024-05-03T14:44:31.466025+0000","last_unstale":"2024-05-03T14:46:06.115229+0000","last_undegraded":"2024-05-03T14:46:06.115229+0000","last_fullsized":"2024-05-03T14:46:06.115229+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:13:00.130689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.590957+0000","last_change":"2024-05-03T14:44:44.567031+0000","last_active":"2024-05-03T14:46:05.590957+0000","last_peered":"2024-05-03T14:46:05.590957+0000","last_clean":"2024-05-03T14:46:05.590957+0000","last_became_active":"2024-05-03T14:44:44.566562+0000","last_became_peered":"2024-05-03T14:44:44.566562+0000","last_unstale":"2024-05-03T14:46:05.590957+0000","last_undegraded":"2024-05-03T14:46:05.590957+0000","last_fullsized":"2024-05-03T14:46:05.590957+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T15:47:12.335283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110029+0000","last_change":"2024-05-03T14:44:28.279115+0000","last_active":"2024-05-03T14:46:06.110029+0000","last_peered":"2024-05-03T14:46:06.110029+0000","last_clean":"2024-05-03T14:46:06.110029+0000","last_became_active":"2024-05-03T14:44:28.278781+0000","last_became_peered":"2024-05-03T14:44:28.278781+0000","last_unstale":"2024-05-03T14:46:06.110029+0000","last_undegraded":"2024-05-03T14:46:06.110029+0000","last_fullsized":"2024-05-03T14:46:06.110029+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:40:41.273763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.590447+0000","last_change":"2024-05-03T14:44:31.475575+0000","last_active":"2024-05-03T14:46:05.590447+0000","last_peered":"2024-05-03T14:46:05.590447+0000","last_clean":"2024-05-03T14:46:05.590447+0000","last_became_active":"2024-05-03T14:44:31.474336+0000","last_became_peered":"2024-05-03T14:44:31.474336+0000","last_unstale":"2024-05-03T14:46:05.590447+0000","last_undegraded":"2024-05-03T14:46:05.590447+0000","last_fullsized":"2024-05-03T14:46:05.590447+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:22:48.742222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109630+0000","last_change":"2024-05-03T14:45:05.616983+0000","last_active":"2024-05-03T14:46:06.109630+0000","last_peered":"2024-05-03T14:46:06.109630+0000","last_clean":"2024-05-03T14:46:06.109630+0000","last_became_active":"2024-05-03T14:45:05.616475+0000","last_became_peered":"2024-05-03T14:45:05.616475+0000","last_unstale":"2024-05-03T14:46:06.109630+0000","last_undegraded":"2024-05-03T14:46:06.109630+0000","last_fullsized":"2024-05-03T14:46:06.109630+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:01:02.124133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.108605+0000","last_change":"2024-05-03T14:44:27.417092+0000","last_active":"2024-05-03T14:46:06.108605+0000","last_peered":"2024-05-03T14:46:06.108605+0000","last_clean":"2024-05-03T14:46:06.108605+0000","last_became_active":"2024-05-03T14:44:27.416765+0000","last_became_peered":"2024-05-03T14:44:27.416765+0000","last_unstale":"2024-05-03T14:46:06.108605+0000","last_undegraded":"2024-05-03T14:46:06.108605+0000","last_fullsized":"2024-05-03T14:46:06.108605+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T16:08:01.882063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.593593+0000","last_change":"2024-05-03T14:44:31.448394+0000","last_active":"2024-05-03T14:46:05.593593+0000","last_peered":"2024-05-03T14:46:05.593593+0000","last_clean":"2024-05-03T14:46:05.593593+0000","last_became_active":"2024-05-03T14:44:31.447752+0000","last_became_peered":"2024-05-03T14:44:31.447752+0000","last_unstale":"2024-05-03T14:46:05.593593+0000","last_undegraded":"2024-05-03T14:46:05.593593+0000","last_fullsized":"2024-05-03T14:46:05.593593+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:01:24.601259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.591530+0000","last_change":"2024-05-03T14:44:44.587688+0000","last_active":"2024-05-03T14:46:05.591530+0000","last_peered":"2024-05-03T14:46:05.591530+0000","last_clean":"2024-05-03T14:46:05.591530+0000","last_became_active":"2024-05-03T14:44:44.586982+0000","last_became_peered":"2024-05-03T14:44:44.586982+0000","last_unstale":"2024-05-03T14:46:05.591530+0000","last_undegraded":"2024-05-03T14:46:05.591530+0000","last_fullsized":"2024-05-03T14:46:05.591530+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:22:54.456635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.113165+0000","last_change":"2024-05-03T14:44:27.411828+0000","last_active":"2024-05-03T14:46:06.113165+0000","last_peered":"2024-05-03T14:46:06.113165+0000","last_clean":"2024-05-03T14:46:06.113165+0000","last_became_active":"2024-05-03T14:44:27.410490+0000","last_became_peered":"2024-05-03T14:44:27.410490+0000","last_unstale":"2024-05-03T14:46:06.113165+0000","last_undegraded":"2024-05-03T14:46:06.113165+0000","last_fullsized":"2024-05-03T14:46:06.113165+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:34:00.061773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109045+0000","last_change":"2024-05-03T14:44:31.448582+0000","last_active":"2024-05-03T14:46:06.109045+0000","last_peered":"2024-05-03T14:46:06.109045+0000","last_clean":"2024-05-03T14:46:06.109045+0000","last_became_active":"2024-05-03T14:44:31.448246+0000","last_became_peered":"2024-05-03T14:44:31.448246+0000","last_unstale":"2024-05-03T14:46:06.109045+0000","last_undegraded":"2024-05-03T14:46:06.109045+0000","last_fullsized":"2024-05-03T14:46:06.109045+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:12:19.856995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109106+0000","last_change":"2024-05-03T14:45:05.616413+0000","last_active":"2024-05-03T14:46:06.109106+0000","last_peered":"2024-05-03T14:46:06.109106+0000","last_clean":"2024-05-03T14:46:06.109106+0000","last_became_active":"2024-05-03T14:45:05.615930+0000","last_became_peered":"2024-05-03T14:45:05.615930+0000","last_unstale":"2024-05-03T14:46:06.109106+0000","last_undegraded":"2024-05-03T14:46:06.109106+0000","last_fullsized":"2024-05-03T14:46:06.109106+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:32:43.344452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.591057+0000","last_change":"2024-05-03T14:44:27.417625+0000","last_active":"2024-05-03T14:46:05.591057+0000","last_peered":"2024-05-03T14:46:05.591057+0000","last_clean":"2024-05-03T14:46:05.591057+0000","last_became_active":"2024-05-03T14:44:27.417230+0000","last_became_peered":"2024-05-03T14:44:27.417230+0000","last_unstale":"2024-05-03T14:46:05.591057+0000","last_undegraded":"2024-05-03T14:46:05.591057+0000","last_fullsized":"2024-05-03T14:46:05.591057+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:56:13.661544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.114328+0000","last_change":"2024-05-03T14:44:31.475539+0000","last_active":"2024-05-03T14:46:06.114328+0000","last_peered":"2024-05-03T14:46:06.114328+0000","last_clean":"2024-05-03T14:46:06.114328+0000","last_became_active":"2024-05-03T14:44:31.474743+0000","last_became_peered":"2024-05-03T14:44:31.474743+0000","last_unstale":"2024-05-03T14:46:06.114328+0000","last_undegraded":"2024-05-03T14:46:06.114328+0000","last_fullsized":"2024-05-03T14:46:06.114328+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T15:52:19.705304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109430+0000","last_change":"2024-05-03T14:44:45.208377+0000","last_active":"2024-05-03T14:46:06.109430+0000","last_peered":"2024-05-03T14:46:06.109430+0000","last_clean":"2024-05-03T14:46:06.109430+0000","last_became_active":"2024-05-03T14:44:45.207778+0000","last_became_peered":"2024-05-03T14:44:45.207778+0000","last_unstale":"2024-05-03T14:46:06.109430+0000","last_undegraded":"2024-05-03T14:46:06.109430+0000","last_fullsized":"2024-05-03T14:46:06.109430+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T15:51:36.843263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.593812+0000","last_change":"2024-05-03T14:44:27.403571+0000","last_active":"2024-05-03T14:46:05.593812+0000","last_peered":"2024-05-03T14:46:05.593812+0000","last_clean":"2024-05-03T14:46:05.593812+0000","last_became_active":"2024-05-03T14:44:27.403173+0000","last_became_peered":"2024-05-03T14:44:27.403173+0000","last_unstale":"2024-05-03T14:46:05.593812+0000","last_undegraded":"2024-05-03T14:46:05.593812+0000","last_fullsized":"2024-05-03T14:46:05.593812+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T15:32:07.406763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.594783+0000","last_change":"2024-05-03T14:44:31.459573+0000","last_active":"2024-05-03T14:46:05.594783+0000","last_peered":"2024-05-03T14:46:05.594783+0000","last_clean":"2024-05-03T14:46:05.594783+0000","last_became_active":"2024-05-03T14:44:31.459200+0000","last_became_peered":"2024-05-03T14:44:31.459200+0000","last_unstale":"2024-05-03T14:46:05.594783+0000","last_undegraded":"2024-05-03T14:46:05.594783+0000","last_fullsized":"2024-05-03T14:46:05.594783+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:58:04.841787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.108722+0000","last_change":"2024-05-03T14:44:44.579130+0000","last_active":"2024-05-03T14:46:06.108722+0000","last_peered":"2024-05-03T14:46:06.108722+0000","last_clean":"2024-05-03T14:46:06.108722+0000","last_became_active":"2024-05-03T14:44:44.578696+0000","last_became_peered":"2024-05-03T14:44:44.578696+0000","last_unstale":"2024-05-03T14:46:06.108722+0000","last_undegraded":"2024-05-03T14:46:06.108722+0000","last_fullsized":"2024-05-03T14:46:06.108722+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:42:48.407127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'5","reported_seq":88,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:32.597237+0000","last_change":"2024-05-03T14:44:28.279181+0000","last_active":"2024-05-03T14:46:32.597237+0000","last_peered":"2024-05-03T14:46:32.597237+0000","last_clean":"2024-05-03T14:46:32.597237+0000","last_became_active":"2024-05-03T14:44:28.278790+0000","last_became_peered":"2024-05-03T14:44:28.278790+0000","last_unstale":"2024-05-03T14:46:32.597237+0000","last_undegraded":"2024-05-03T14:46:32.597237+0000","last_fullsized":"2024-05-03T14:46:32.597237+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:16:30.170750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":39,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.594856+0000","last_change":"2024-05-03T14:44:31.451646+0000","last_active":"2024-05-03T14:46:05.594856+0000","last_peered":"2024-05-03T14:46:05.594856+0000","last_clean":"2024-05-03T14:46:05.594856+0000","last_became_active":"2024-05-03T14:44:31.451319+0000","last_became_peered":"2024-05-03T14:44:31.451319+0000","last_unstale":"2024-05-03T14:46:05.594856+0000","last_undegraded":"2024-05-03T14:46:05.594856+0000","last_fullsized":"2024-05-03T14:46:05.594856+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:17:51.946028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.590565+0000","last_change":"2024-05-03T14:45:05.619282+0000","last_active":"2024-05-03T14:46:05.590565+0000","last_peered":"2024-05-03T14:46:05.590565+0000","last_clean":"2024-05-03T14:46:05.590565+0000","last_became_active":"2024-05-03T14:45:05.617776+0000","last_became_peered":"2024-05-03T14:45:05.617776+0000","last_unstale":"2024-05-03T14:46:05.590565+0000","last_undegraded":"2024-05-03T14:46:05.590565+0000","last_fullsized":"2024-05-03T14:46:05.590565+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:36:53.105226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110420+0000","last_change":"2024-05-03T14:44:27.415277+0000","last_active":"2024-05-03T14:46:06.110420+0000","last_peered":"2024-05-03T14:46:06.110420+0000","last_clean":"2024-05-03T14:46:06.110420+0000","last_became_active":"2024-05-03T14:44:27.414900+0000","last_became_peered":"2024-05-03T14:44:27.414900+0000","last_unstale":"2024-05-03T14:46:06.110420+0000","last_undegraded":"2024-05-03T14:46:06.110420+0000","last_fullsized":"2024-05-03T14:46:06.110420+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:41:00.065933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.594532+0000","last_change":"2024-05-03T14:44:31.467592+0000","last_active":"2024-05-03T14:46:05.594532+0000","last_peered":"2024-05-03T14:46:05.594532+0000","last_clean":"2024-05-03T14:46:05.594532+0000","last_became_active":"2024-05-03T14:44:31.467265+0000","last_became_peered":"2024-05-03T14:44:31.467265+0000","last_unstale":"2024-05-03T14:46:05.594532+0000","last_undegraded":"2024-05-03T14:46:05.594532+0000","last_fullsized":"2024-05-03T14:46:05.594532+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:28:26.672800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109064+0000","last_change":"2024-05-03T14:44:44.566154+0000","last_active":"2024-05-03T14:46:06.109064+0000","last_peered":"2024-05-03T14:46:06.109064+0000","last_clean":"2024-05-03T14:46:06.109064+0000","last_became_active":"2024-05-03T14:44:44.565714+0000","last_became_peered":"2024-05-03T14:44:44.565714+0000","last_unstale":"2024-05-03T14:46:06.109064+0000","last_undegraded":"2024-05-03T14:46:06.109064+0000","last_fullsized":"2024-05-03T14:46:06.109064+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:54:48.448838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.113551+0000","last_change":"2024-05-03T14:44:27.419293+0000","last_active":"2024-05-03T14:46:06.113551+0000","last_peered":"2024-05-03T14:46:06.113551+0000","last_clean":"2024-05-03T14:46:06.113551+0000","last_became_active":"2024-05-03T14:44:27.418902+0000","last_became_peered":"2024-05-03T14:44:27.418902+0000","last_unstale":"2024-05-03T14:46:06.113551+0000","last_undegraded":"2024-05-03T14:46:06.113551+0000","last_fullsized":"2024-05-03T14:46:06.113551+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:17:54.890553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.595057+0000","last_change":"2024-05-03T14:44:31.449682+0000","last_active":"2024-05-03T14:46:05.595057+0000","last_peered":"2024-05-03T14:46:05.595057+0000","last_clean":"2024-05-03T14:46:05.595057+0000","last_became_active":"2024-05-03T14:44:31.449192+0000","last_became_peered":"2024-05-03T14:44:31.449192+0000","last_unstale":"2024-05-03T14:46:05.595057+0000","last_undegraded":"2024-05-03T14:46:05.595057+0000","last_fullsized":"2024-05-03T14:46:05.595057+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:48:46.249154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109183+0000","last_change":"2024-05-03T14:45:05.422527+0000","last_active":"2024-05-03T14:46:06.109183+0000","last_peered":"2024-05-03T14:46:06.109183+0000","last_clean":"2024-05-03T14:46:06.109183+0000","last_became_active":"2024-05-03T14:45:05.422014+0000","last_became_peered":"2024-05-03T14:45:05.422014+0000","last_unstale":"2024-05-03T14:46:06.109183+0000","last_undegraded":"2024-05-03T14:46:06.109183+0000","last_fullsized":"2024-05-03T14:46:06.109183+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T15:13:47.216960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.591649+0000","last_change":"2024-05-03T14:44:31.450447+0000","last_active":"2024-05-03T14:46:05.591649+0000","last_peered":"2024-05-03T14:46:05.591649+0000","last_clean":"2024-05-03T14:46:05.591649+0000","last_became_active":"2024-05-03T14:44:31.450103+0000","last_became_peered":"2024-05-03T14:44:31.450103+0000","last_unstale":"2024-05-03T14:46:05.591649+0000","last_undegraded":"2024-05-03T14:46:05.591649+0000","last_fullsized":"2024-05-03T14:46:05.591649+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:43:06.840375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.591177+0000","last_change":"2024-05-03T14:44:31.467676+0000","last_active":"2024-05-03T14:46:05.591177+0000","last_peered":"2024-05-03T14:46:05.591177+0000","last_clean":"2024-05-03T14:46:05.591177+0000","last_became_active":"2024-05-03T14:44:31.467285+0000","last_became_peered":"2024-05-03T14:44:31.467285+0000","last_unstale":"2024-05-03T14:46:05.591177+0000","last_undegraded":"2024-05-03T14:46:05.591177+0000","last_fullsized":"2024-05-03T14:46:05.591177+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:39:37.400178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.591210+0000","last_change":"2024-05-03T14:44:31.469379+0000","last_active":"2024-05-03T14:46:05.591210+0000","last_peered":"2024-05-03T14:46:05.591210+0000","last_clean":"2024-05-03T14:46:05.591210+0000","last_became_active":"2024-05-03T14:44:31.469022+0000","last_became_peered":"2024-05-03T14:44:31.469022+0000","last_unstale":"2024-05-03T14:46:05.591210+0000","last_undegraded":"2024-05-03T14:46:05.591210+0000","last_fullsized":"2024-05-03T14:46:05.591210+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:33:54.227652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.111018+0000","last_change":"2024-05-03T14:44:31.463709+0000","last_active":"2024-05-03T14:46:06.111018+0000","last_peered":"2024-05-03T14:46:06.111018+0000","last_clean":"2024-05-03T14:46:06.111018+0000","last_became_active":"2024-05-03T14:44:31.463302+0000","last_became_peered":"2024-05-03T14:44:31.463302+0000","last_unstale":"2024-05-03T14:46:06.111018+0000","last_undegraded":"2024-05-03T14:46:06.111018+0000","last_fullsized":"2024-05-03T14:46:06.111018+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:44:39.692253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109781+0000","last_change":"2024-05-03T14:44:31.443381+0000","last_active":"2024-05-03T14:46:06.109781+0000","last_peered":"2024-05-03T14:46:06.109781+0000","last_clean":"2024-05-03T14:46:06.109781+0000","last_became_active":"2024-05-03T14:44:31.442981+0000","last_became_peered":"2024-05-03T14:44:31.442981+0000","last_unstale":"2024-05-03T14:46:06.109781+0000","last_undegraded":"2024-05-03T14:46:06.109781+0000","last_fullsized":"2024-05-03T14:46:06.109781+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:58:56.653006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.591515+0000","last_change":"2024-05-03T14:44:31.449229+0000","last_active":"2024-05-03T14:46:05.591515+0000","last_peered":"2024-05-03T14:46:05.591515+0000","last_clean":"2024-05-03T14:46:05.591515+0000","last_became_active":"2024-05-03T14:44:31.448870+0000","last_became_peered":"2024-05-03T14:44:31.448870+0000","last_unstale":"2024-05-03T14:46:05.591515+0000","last_undegraded":"2024-05-03T14:46:05.591515+0000","last_fullsized":"2024-05-03T14:46:05.591515+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:48:29.213680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.115390+0000","last_change":"2024-05-03T14:44:31.468266+0000","last_active":"2024-05-03T14:46:06.115390+0000","last_peered":"2024-05-03T14:46:06.115390+0000","last_clean":"2024-05-03T14:46:06.115390+0000","last_became_active":"2024-05-03T14:44:31.467784+0000","last_became_peered":"2024-05-03T14:44:31.467784+0000","last_unstale":"2024-05-03T14:46:06.115390+0000","last_undegraded":"2024-05-03T14:46:06.115390+0000","last_fullsized":"2024-05-03T14:46:06.115390+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:46:18.979331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109201+0000","last_change":"2024-05-03T14:44:31.467526+0000","last_active":"2024-05-03T14:46:06.109201+0000","last_peered":"2024-05-03T14:46:06.109201+0000","last_clean":"2024-05-03T14:46:06.109201+0000","last_became_active":"2024-05-03T14:44:31.466849+0000","last_became_peered":"2024-05-03T14:44:31.466849+0000","last_unstale":"2024-05-03T14:46:06.109201+0000","last_undegraded":"2024-05-03T14:46:06.109201+0000","last_fullsized":"2024-05-03T14:46:06.109201+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:57:11.147130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.113215+0000","last_change":"2024-05-03T14:44:31.468251+0000","last_active":"2024-05-03T14:46:06.113215+0000","last_peered":"2024-05-03T14:46:06.113215+0000","last_clean":"2024-05-03T14:46:06.113215+0000","last_became_active":"2024-05-03T14:44:31.467812+0000","last_became_peered":"2024-05-03T14:44:31.467812+0000","last_unstale":"2024-05-03T14:46:06.113215+0000","last_undegraded":"2024-05-03T14:46:06.113215+0000","last_fullsized":"2024-05-03T14:46:06.113215+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:52:45.585331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.108865+0000","last_change":"2024-05-03T14:44:31.466378+0000","last_active":"2024-05-03T14:46:06.108865+0000","last_peered":"2024-05-03T14:46:06.108865+0000","last_clean":"2024-05-03T14:46:06.108865+0000","last_became_active":"2024-05-03T14:44:31.466052+0000","last_became_peered":"2024-05-03T14:44:31.466052+0000","last_unstale":"2024-05-03T14:46:06.108865+0000","last_undegraded":"2024-05-03T14:46:06.108865+0000","last_fullsized":"2024-05-03T14:46:06.108865+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:34:48.401934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110416+0000","last_change":"2024-05-03T14:44:31.463771+0000","last_active":"2024-05-03T14:46:06.110416+0000","last_peered":"2024-05-03T14:46:06.110416+0000","last_clean":"2024-05-03T14:46:06.110416+0000","last_became_active":"2024-05-03T14:44:31.463204+0000","last_became_peered":"2024-05-03T14:44:31.463204+0000","last_unstale":"2024-05-03T14:46:06.110416+0000","last_undegraded":"2024-05-03T14:46:06.110416+0000","last_fullsized":"2024-05-03T14:46:06.110416+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:47:33.044835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109493+0000","last_change":"2024-05-03T14:44:31.450950+0000","last_active":"2024-05-03T14:46:06.109493+0000","last_peered":"2024-05-03T14:46:06.109493+0000","last_clean":"2024-05-03T14:46:06.109493+0000","last_became_active":"2024-05-03T14:44:31.450595+0000","last_became_peered":"2024-05-03T14:44:31.450595+0000","last_unstale":"2024-05-03T14:46:06.109493+0000","last_undegraded":"2024-05-03T14:46:06.109493+0000","last_fullsized":"2024-05-03T14:46:06.109493+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:42:41.240430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":250062,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65338,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":411,"num_read_kb":277,"num_write":244,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":196014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":211,"ondisk_log_size":211,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710762,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":516321,"data_compressed":32309,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906991,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28128,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717024,"statfs":{"total":95995035648,"available":95966232576,"internally_reserved":0,"allocated":1138688,"data_stored":1165365,"data_compressed":68497,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070517,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":515467,"data_compressed":32309,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266747,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":999424,"data_stored":515733,"data_compressed":32309,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462976,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":1111279,"data_compressed":68497,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626502,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":987136,"data_stored":510971,"data_compressed":32309,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574946,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":571539,"data_compressed":32309,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738473,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":512427,"data_compressed":32309,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411423,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1064960,"data_stored":570515,"data_compressed":32309,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182487,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":510679,"data_compressed":32309,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986258,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":995328,"data_stored":515045,"data_compressed":32309,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822733,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1069056,"data_stored":1107533,"data_compressed":68497,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34561,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34561,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36188,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36188,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36188,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36188,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36188,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":61290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":60266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":60652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-03T14:46:37.796 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-03T14:46:37.796 INFO:tasks.cephadm:Setting up client nodes... 2024-05-03T14:46:37.796 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-03T14:46:37.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:37 smithi090 ceph-mon[33568]: pgmap v395: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-05-03T14:46:37.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:37 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:46:37.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:37 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:46:37.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:37 smithi149 ceph-mon[33332]: pgmap v395: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-05-03T14:46:37.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:37 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:46:37.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:37 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:46:38.304 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:46:38.783 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:38 smithi060 ceph-mon[28873]: from='client.24739 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T14:46:38.783 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:38 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:46:38.783 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:38 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:46:38.783 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:38 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:46:38.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:38 smithi090 ceph-mon[33568]: from='client.24739 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T14:46:38.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:38 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:46:38.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:38 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:46:38.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:38 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:46:38.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:38 smithi149 ceph-mon[33332]: from='client.24739 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T14:46:38.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:38 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:46:38.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:38 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:46:38.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:38 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:46:39.702 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:39 smithi060 ceph-mon[28873]: pgmap v396: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 9 op/s 2024-05-03T14:46:39.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:39 smithi090 ceph-mon[33568]: pgmap v396: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 9 op/s 2024-05-03T14:46:39.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:39 smithi149 ceph-mon[33332]: pgmap v396: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 9 op/s 2024-05-03T14:46:40.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:40.573+0000 7f05ff200700 1 -- 172.21.15.60:0/2337610728 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0600101be0 msgr2=0x7f0600104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:40.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:40.573+0000 7f05ff200700 1 --2- 172.21.15.60:0/2337610728 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0600101be0 0x7f0600104010 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f05f0009990 tx=0x7f05f0009ca0 comp rx=0 tx=0).stop 2024-05-03T14:46:40.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:40.573+0000 7f05ff200700 1 -- 172.21.15.60:0/2337610728 shutdown_connections 2024-05-03T14:46:40.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:40.573+0000 7f05ff200700 1 --2- 172.21.15.60:0/2337610728 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0600106f20 0x7f06001073a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:40.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:40.573+0000 7f05ff200700 1 --2- 172.21.15.60:0/2337610728 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0600101be0 0x7f0600104010 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:40.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:40.573+0000 7f05ff200700 1 --2- 172.21.15.60:0/2337610728 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0600104550 0x7f06001069e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:40.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:40.573+0000 7f05ff200700 1 -- 172.21.15.60:0/2337610728 >> 172.21.15.60:0/2337610728 conn(0x7f06000fd820 msgr2=0x7f06000ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:46:40.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:40.573+0000 7f05ff200700 1 -- 172.21.15.60:0/2337610728 shutdown_connections 2024-05-03T14:46:40.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:40.573+0000 7f05ff200700 1 -- 172.21.15.60:0/2337610728 wait complete. 2024-05-03T14:46:40.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:40.574+0000 7f05ff200700 1 Processor -- start 2024-05-03T14:46:40.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:40.575+0000 7f05ff200700 1 -- start start 2024-05-03T14:46:40.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:40.575+0000 7f05ff200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0600101be0 0x7f060019fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:46:40.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:40.575+0000 7f05ff200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0600104550 0x7f060019fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:46:40.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:40.575+0000 7f05ff200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0600106f20 0x7f06001a7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:46:40.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:40.575+0000 7f05ff200700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f0600115600 con 0x7f0600106f20 2024-05-03T14:46:40.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:40.575+0000 7f05ff200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0600115480 con 0x7f0600101be0 2024-05-03T14:46:40.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:40.575+0000 7f05ff200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0600115780 con 0x7f0600104550 2024-05-03T14:46:40.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:40.575+0000 7f05fe800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0600106f20 0x7f06001a7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:46:40.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:40.575+0000 7f05fe800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0600106f20 0x7f06001a7070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50908/0 (socket says 172.21.15.60:50908) 2024-05-03T14:46:40.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:40.575+0000 7f05fe800700 1 -- 172.21.15.60:0/1639333718 learned_addr learned my addr 172.21.15.60:0/1639333718 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:46:40.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:40.575+0000 7f05f7e00700 1 --2- 172.21.15.60:0/1639333718 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0600104550 0x7f060019fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:46:40.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:40.576+0000 7f05fde00700 1 --2- 172.21.15.60:0/1639333718 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0600101be0 0x7f060019fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:46:40.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:40.576+0000 7f05fe800700 1 -- 172.21.15.60:0/1639333718 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0600104550 msgr2=0x7f060019fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:40.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:40.576+0000 7f05fe800700 1 --2- 172.21.15.60:0/1639333718 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0600104550 0x7f060019fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:40.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:40.576+0000 7f05fe800700 1 -- 172.21.15.60:0/1639333718 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0600101be0 msgr2=0x7f060019fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:40.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:40.576+0000 7f05fe800700 1 --2- 172.21.15.60:0/1639333718 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0600101be0 0x7f060019fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:40.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:40.576+0000 7f05fe800700 1 -- 172.21.15.60:0/1639333718 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06001a7710 con 0x7f0600106f20 2024-05-03T14:46:40.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:40.576+0000 7f05fde00700 1 --2- 172.21.15.60:0/1639333718 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0600101be0 0x7f060019fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:46:40.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:40.576+0000 7f05fe800700 1 --2- 172.21.15.60:0/1639333718 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0600106f20 0x7f06001a7070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f05f800ef60 tx=0x7f05f800ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:46:40.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:40.577+0000 7f05f6a00700 1 -- 172.21.15.60:0/1639333718 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f80092e0 con 0x7f0600106f20 2024-05-03T14:46:40.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:40.577+0000 7f05f6a00700 1 -- 172.21.15.60:0/1639333718 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f05f8004bb0 con 0x7f0600106f20 2024-05-03T14:46:40.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:40.577+0000 7f05f6a00700 1 -- 172.21.15.60:0/1639333718 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f8004280 con 0x7f0600106f20 2024-05-03T14:46:40.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:40.577+0000 7f05ff200700 1 -- 172.21.15.60:0/1639333718 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06001a79a0 con 0x7f0600106f20 2024-05-03T14:46:40.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:40.577+0000 7f05ff200700 1 -- 172.21.15.60:0/1639333718 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06001a7e90 con 0x7f0600106f20 2024-05-03T14:46:40.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:40.578+0000 7f05ff200700 1 -- 172.21.15.60:0/1639333718 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0600064b00 con 0x7f0600106f20 2024-05-03T14:46:40.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:40.580+0000 7f05f6a00700 1 -- 172.21.15.60:0/1639333718 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f05f801b030 con 0x7f0600106f20 2024-05-03T14:46:40.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:40.582+0000 7f05f6a00700 1 --2- 172.21.15.60:0/1639333718 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f05e40b29b0 0x7f05e40b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:46:40.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:40.582+0000 7f05f6a00700 1 -- 172.21.15.60:0/1639333718 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f05f80e2170 con 0x7f0600106f20 2024-05-03T14:46:40.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:40.582+0000 7f05fde00700 1 --2- 172.21.15.60:0/1639333718 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f05e40b29b0 0x7f05e40b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:46:40.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:40.583+0000 7f05fde00700 1 --2- 172.21.15.60:0/1639333718 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f05e40b29b0 0x7f05e40b4e70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f05f000b2e0 tx=0x7f05f001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:46:40.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:40.587+0000 7f05f6a00700 1 -- 172.21.15.60:0/1639333718 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f05f8022020 con 0x7f0600106f20 2024-05-03T14:46:40.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:40.925+0000 7f05ff200700 1 -- 172.21.15.60:0/1639333718 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f06000640b0 con 0x7f0600106f20 2024-05-03T14:46:40.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:40.930+0000 7f05f6a00700 1 -- 172.21.15.60:0/1639333718 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f05f8010040 con 0x7f0600106f20 2024-05-03T14:46:40.933 INFO:teuthology.orchestra.run.smithi060.stdout:[client.0] 2024-05-03T14:46:40.933 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQDQ+DRmLGhRNxAAkpbxLPUuh8Rc+ppr2HgqPg== 2024-05-03T14:46:40.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:40.933+0000 7f05ff200700 1 -- 172.21.15.60:0/1639333718 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f05e40b29b0 msgr2=0x7f05e40b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:40.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:40.933+0000 7f05ff200700 1 --2- 172.21.15.60:0/1639333718 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f05e40b29b0 0x7f05e40b4e70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f05f000b2e0 tx=0x7f05f001a040 comp rx=0 tx=0).stop 2024-05-03T14:46:40.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:40.933+0000 7f05ff200700 1 -- 172.21.15.60:0/1639333718 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0600106f20 msgr2=0x7f06001a7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:40.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:40.933+0000 7f05ff200700 1 --2- 172.21.15.60:0/1639333718 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0600106f20 0x7f06001a7070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f05f800ef60 tx=0x7f05f800ef90 comp rx=0 tx=0).stop 2024-05-03T14:46:40.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:40.933+0000 7f05ff200700 1 -- 172.21.15.60:0/1639333718 shutdown_connections 2024-05-03T14:46:40.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:40.933+0000 7f05ff200700 1 --2- 172.21.15.60:0/1639333718 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f05e40b29b0 0x7f05e40b4e70 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:40.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:40.933+0000 7f05ff200700 1 --2- 172.21.15.60:0/1639333718 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0600106f20 0x7f06001a7070 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:40.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:40.933+0000 7f05ff200700 1 --2- 172.21.15.60:0/1639333718 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0600101be0 0x7f060019fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:40.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:40.933+0000 7f05ff200700 1 --2- 172.21.15.60:0/1639333718 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0600104550 0x7f060019fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:40.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:40.933+0000 7f05ff200700 1 -- 172.21.15.60:0/1639333718 >> 172.21.15.60:0/1639333718 conn(0x7f06000fd820 msgr2=0x7f0600105260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:46:40.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:40.934+0000 7f05ff200700 1 -- 172.21.15.60:0/1639333718 shutdown_connections 2024-05-03T14:46:40.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:40.934+0000 7f05ff200700 1 -- 172.21.15.60:0/1639333718 wait complete. 2024-05-03T14:46:41.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:41 smithi060 ceph-mon[28873]: pgmap v397: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 938 B/s wr, 12 op/s 2024-05-03T14:46:41.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:41 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/1639333718' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-03T14:46:41.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:41 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/1639333718' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-03T14:46:41.775 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-05-03T14:46:41.776 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-03T14:46:41.776 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-03T14:46:41.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:41 smithi090 ceph-mon[33568]: pgmap v397: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 938 B/s wr, 12 op/s 2024-05-03T14:46:41.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:41 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/1639333718' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-03T14:46:41.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:41 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/1639333718' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-03T14:46:41.911 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-03T14:46:41.911 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-03T14:46:41.911 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph mgr dump --format=json 2024-05-03T14:46:41.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:41 smithi149 ceph-mon[33332]: pgmap v397: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 938 B/s wr, 12 op/s 2024-05-03T14:46:41.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:41 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/1639333718' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-03T14:46:41.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:41 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/1639333718' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-03T14:46:42.444 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:46:43.728 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:43 smithi060 ceph-mon[28873]: pgmap v398: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-03T14:46:43.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:43 smithi090 ceph-mon[33568]: pgmap v398: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-03T14:46:43.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:43 smithi149 ceph-mon[33332]: pgmap v398: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-03T14:46:44.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:44.615+0000 7f0a1ee00700 1 -- 172.21.15.60:0/1736749741 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a18103ef0 msgr2=0x7f0a181103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:44.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:44.615+0000 7f0a1ee00700 1 --2- 172.21.15.60:0/1736749741 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a18103ef0 0x7f0a181103d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0a1000b0a0 tx=0x7f0a1000b3b0 comp rx=0 tx=0).stop 2024-05-03T14:46:44.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:44.615+0000 7f0a1ee00700 1 -- 172.21.15.60:0/1736749741 shutdown_connections 2024-05-03T14:46:44.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:44.615+0000 7f0a1ee00700 1 --2- 172.21.15.60:0/1736749741 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a18103ef0 0x7f0a181103d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:44.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:44.615+0000 7f0a1ee00700 1 --2- 172.21.15.60:0/1736749741 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0a18108460 0x7f0a18108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:44.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:44.615+0000 7f0a1ee00700 1 --2- 172.21.15.60:0/1736749741 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a18103550 0x7f0a181039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:44.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:44.615+0000 7f0a1ee00700 1 -- 172.21.15.60:0/1736749741 >> 172.21.15.60:0/1736749741 conn(0x7f0a1807a600 msgr2=0x7f0a1807aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:46:44.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:44.616+0000 7f0a1ee00700 1 -- 172.21.15.60:0/1736749741 shutdown_connections 2024-05-03T14:46:44.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:44.616+0000 7f0a1ee00700 1 -- 172.21.15.60:0/1736749741 wait complete. 2024-05-03T14:46:44.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:44.617+0000 7f0a1ee00700 1 Processor -- start 2024-05-03T14:46:44.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:44.617+0000 7f0a1ee00700 1 -- start start 2024-05-03T14:46:44.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:44.618+0000 7f0a1ee00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a18103550 0x7f0a1819f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:46:44.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:44.618+0000 7f0a1ee00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a18103ef0 0x7f0a1819fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:46:44.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:44.618+0000 7f0a1ee00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0a18108460 0x7f0a18199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:46:44.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:44.618+0000 7f0a1ee00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f0a18113930 con 0x7f0a18108460 2024-05-03T14:46:44.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:44.618+0000 7f0a1ee00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0a181137b0 con 0x7f0a18103550 2024-05-03T14:46:44.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:44.618+0000 7f0a1ee00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0a18113ab0 con 0x7f0a18103ef0 2024-05-03T14:46:44.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:44.618+0000 7f0a17e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a18103550 0x7f0a1819f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:46:44.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:44.618+0000 7f0a1ce00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0a18108460 0x7f0a18199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:46:44.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:44.618+0000 7f0a17e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a18103550 0x7f0a1819f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.60:40472/0 (socket says 172.21.15.60:40472) 2024-05-03T14:46:44.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:44.618+0000 7f0a17e00700 1 -- 172.21.15.60:0/3993245292 learned_addr learned my addr 172.21.15.60:0/3993245292 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:46:44.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:44.618+0000 7f0a17400700 1 --2- 172.21.15.60:0/3993245292 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a18103ef0 0x7f0a1819fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:46:44.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:44.619+0000 7f0a17e00700 1 -- 172.21.15.60:0/3993245292 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a18103ef0 msgr2=0x7f0a1819fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:44.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:44.619+0000 7f0a17e00700 1 --2- 172.21.15.60:0/3993245292 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a18103ef0 0x7f0a1819fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:44.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:44.619+0000 7f0a17e00700 1 -- 172.21.15.60:0/3993245292 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0a18108460 msgr2=0x7f0a18199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:44.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:44.619+0000 7f0a17e00700 1 --2- 172.21.15.60:0/3993245292 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0a18108460 0x7f0a18199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:44.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:44.619+0000 7f0a17e00700 1 -- 172.21.15.60:0/3993245292 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a1819a250 con 0x7f0a18103550 2024-05-03T14:46:44.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:44.619+0000 7f0a1ce00700 1 --2- 172.21.15.60:0/3993245292 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0a18108460 0x7f0a18199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:46:44.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:44.619+0000 7f0a17e00700 1 --2- 172.21.15.60:0/3993245292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a18103550 0x7f0a1819f8e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0a0800ea50 tx=0x7f0a0800ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:46:44.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:44.619+0000 7f0a14c00700 1 -- 172.21.15.60:0/3993245292 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0800cbd0 con 0x7f0a18103550 2024-05-03T14:46:44.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:44.619+0000 7f0a14c00700 1 -- 172.21.15.60:0/3993245292 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f0a0800cd70 con 0x7f0a18103550 2024-05-03T14:46:44.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:44.619+0000 7f0a14c00700 1 -- 172.21.15.60:0/3993245292 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a08018820 con 0x7f0a18103550 2024-05-03T14:46:44.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:44.619+0000 7f0a1ee00700 1 -- 172.21.15.60:0/3993245292 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a1819a540 con 0x7f0a18103550 2024-05-03T14:46:44.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:44.620+0000 7f0a1ee00700 1 -- 172.21.15.60:0/3993245292 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a181a67b0 con 0x7f0a18103550 2024-05-03T14:46:44.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:44.621+0000 7f0a1ee00700 1 -- 172.21.15.60:0/3993245292 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a1810ba70 con 0x7f0a18103550 2024-05-03T14:46:44.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:44.623+0000 7f0a14c00700 1 -- 172.21.15.60:0/3993245292 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f0a0801f020 con 0x7f0a18103550 2024-05-03T14:46:44.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:44.625+0000 7f0a14c00700 1 --2- 172.21.15.60:0/3993245292 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f09fc0b2990 0x7f09fc0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:46:44.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:44.625+0000 7f0a14c00700 1 -- 172.21.15.60:0/3993245292 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f0a080e2980 con 0x7f0a18103550 2024-05-03T14:46:44.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:44.625+0000 7f0a17400700 1 --2- 172.21.15.60:0/3993245292 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f09fc0b2990 0x7f09fc0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:46:44.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:44.626+0000 7f0a17400700 1 --2- 172.21.15.60:0/3993245292 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f09fc0b2990 0x7f09fc0b4e50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0a1807ab20 tx=0x7f0a04009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:46:44.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:44.627+0000 7f0a14c00700 1 -- 172.21.15.60:0/3993245292 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f0a080aa160 con 0x7f0a18103550 2024-05-03T14:46:44.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:44.927+0000 7f0a1ee00700 1 -- 172.21.15.60:0/3993245292 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f0a1810b450 con 0x7f0a18103550 2024-05-03T14:46:44.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:44.934+0000 7f0a14c00700 1 -- 172.21.15.60:0/3993245292 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+288647 (secure 0 0 0) 0x7f0a080b21e0 con 0x7f0a18103550 2024-05-03T14:46:44.943 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:46:44.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:44.939+0000 7f0a1ee00700 1 -- 172.21.15.60:0/3993245292 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f09fc0b2990 msgr2=0x7f09fc0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:44.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:44.940+0000 7f0a1ee00700 1 --2- 172.21.15.60:0/3993245292 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f09fc0b2990 0x7f09fc0b4e50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0a1807ab20 tx=0x7f0a04009260 comp rx=0 tx=0).stop 2024-05-03T14:46:44.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:44.940+0000 7f0a1ee00700 1 -- 172.21.15.60:0/3993245292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a18103550 msgr2=0x7f0a1819f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:44.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:44.940+0000 7f0a1ee00700 1 --2- 172.21.15.60:0/3993245292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a18103550 0x7f0a1819f8e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0a0800ea50 tx=0x7f0a0800ed60 comp rx=0 tx=0).stop 2024-05-03T14:46:44.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:44.940+0000 7f0a1ee00700 1 -- 172.21.15.60:0/3993245292 shutdown_connections 2024-05-03T14:46:44.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:44.940+0000 7f0a1ee00700 1 --2- 172.21.15.60:0/3993245292 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f09fc0b2990 0x7f09fc0b4e50 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:44.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:44.940+0000 7f0a1ee00700 1 --2- 172.21.15.60:0/3993245292 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0a18108460 0x7f0a18199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:44.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:44.940+0000 7f0a1ee00700 1 --2- 172.21.15.60:0/3993245292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a18103550 0x7f0a1819f8e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:44.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:44.940+0000 7f0a1ee00700 1 --2- 172.21.15.60:0/3993245292 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a18103ef0 0x7f0a1819fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:44.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:44.940+0000 7f0a1ee00700 1 -- 172.21.15.60:0/3993245292 >> 172.21.15.60:0/3993245292 conn(0x7f0a1807a600 msgr2=0x7f0a18101500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:46:44.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:44.940+0000 7f0a1ee00700 1 -- 172.21.15.60:0/3993245292 shutdown_connections 2024-05-03T14:46:44.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:44.940+0000 7f0a1ee00700 1 -- 172.21.15.60:0/3993245292 wait complete. 2024-05-03T14:46:45.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:45 smithi060 ceph-mon[28873]: pgmap v399: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-03T14:46:45.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:45 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/3993245292' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-03T14:46:45.768 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3066142956},{"type":"v1","addr":"172.21.15.60:6801","nonce":3066142956}]},"active_addr":"172.21.15.60:6801/3066142956","active_change":"2024-05-03T14:36:04.154691+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24110,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.60:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.60:0","nonce":1912780771}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.60:0","nonce":4189048076}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.60:0","nonce":1886662085}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.60:0","nonce":3370991107}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.60:0","nonce":3537086186}]}]} 2024-05-03T14:46:45.774 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-03T14:46:45.774 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-03T14:46:45.774 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-03T14:46:45.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:45 smithi090 ceph-mon[33568]: pgmap v399: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-03T14:46:45.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:45 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/3993245292' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-03T14:46:45.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:45 smithi149 ceph-mon[33332]: pgmap v399: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-03T14:46:45.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:45 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/3993245292' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-03T14:46:46.229 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:46:47.725 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:47 smithi149 ceph-mon[33332]: pgmap v400: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-03T14:46:47.836 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:47 smithi060 ceph-mon[28873]: pgmap v400: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-03T14:46:47.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:47 smithi090 ceph-mon[33568]: pgmap v400: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-03T14:46:48.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:48.474+0000 7fa2eba00700 1 -- 172.21.15.60:0/1470125058 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2e410a490 msgr2=0x7fa2e41169a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:48.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:48.474+0000 7fa2eba00700 1 --2- 172.21.15.60:0/1470125058 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2e410a490 0x7fa2e41169a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc009990 tx=0x7fa2dc009ca0 comp rx=0 tx=0).stop 2024-05-03T14:46:48.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:48.474+0000 7fa2eba00700 1 -- 172.21.15.60:0/1470125058 shutdown_connections 2024-05-03T14:46:48.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:48.474+0000 7fa2eba00700 1 --2- 172.21.15.60:0/1470125058 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2e410a490 0x7fa2e41169a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:48.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:48.474+0000 7fa2eba00700 1 --2- 172.21.15.60:0/1470125058 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa2e410fb60 0x7fa2e410ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:48.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:48.474+0000 7fa2eba00700 1 --2- 172.21.15.60:0/1470125058 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2e4109ad0 0x7fa2e4109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:48.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:48.474+0000 7fa2eba00700 1 -- 172.21.15.60:0/1470125058 >> 172.21.15.60:0/1470125058 conn(0x7fa2e4076c70 msgr2=0x7fa2e41082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:46:48.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:48.475+0000 7fa2eba00700 1 -- 172.21.15.60:0/1470125058 shutdown_connections 2024-05-03T14:46:48.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:48.475+0000 7fa2eba00700 1 -- 172.21.15.60:0/1470125058 wait complete. 2024-05-03T14:46:48.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:48.476+0000 7fa2eba00700 1 Processor -- start 2024-05-03T14:46:48.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:48.476+0000 7fa2eba00700 1 -- start start 2024-05-03T14:46:48.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:48.477+0000 7fa2eba00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2e4109ad0 0x7fa2e41a0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:46:48.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:48.477+0000 7fa2eba00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2e410a490 0x7fa2e41a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:46:48.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:48.477+0000 7fa2eba00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa2e410fb60 0x7fa2e41a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:46:48.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:48.477+0000 7fa2eba00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e4119f90 con 0x7fa2e4109ad0 2024-05-03T14:46:48.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:48.477+0000 7fa2eba00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e4119e10 con 0x7fa2e410fb60 2024-05-03T14:46:48.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:48.477+0000 7fa2eba00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e411a110 con 0x7fa2e410a490 2024-05-03T14:46:48.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:48.477+0000 7fa2e9a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa2e410fb60 0x7fa2e41a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:46:48.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:48.477+0000 7fa2e9a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa2e410fb60 0x7fa2e41a4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.60:40486/0 (socket says 172.21.15.60:40486) 2024-05-03T14:46:48.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:48.477+0000 7fa2e9a00700 1 -- 172.21.15.60:0/3448582298 learned_addr learned my addr 172.21.15.60:0/3448582298 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:46:48.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:48.477+0000 7fa2e9000700 1 --2- 172.21.15.60:0/3448582298 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2e4109ad0 0x7fa2e41a0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:46:48.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:48.477+0000 7fa2e3e00700 1 --2- 172.21.15.60:0/3448582298 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2e410a490 0x7fa2e41a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:46:48.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:48.478+0000 7fa2e9a00700 1 -- 172.21.15.60:0/3448582298 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2e410a490 msgr2=0x7fa2e41a08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:48.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:48.478+0000 7fa2e9a00700 1 --2- 172.21.15.60:0/3448582298 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2e410a490 0x7fa2e41a08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:48.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:48.478+0000 7fa2e9a00700 1 -- 172.21.15.60:0/3448582298 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2e4109ad0 msgr2=0x7fa2e41a0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:48.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:48.478+0000 7fa2e9a00700 1 --2- 172.21.15.60:0/3448582298 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2e4109ad0 0x7fa2e41a0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:48.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:48.478+0000 7fa2e9a00700 1 -- 172.21.15.60:0/3448582298 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2e41a5370 con 0x7fa2e410fb60 2024-05-03T14:46:48.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:48.478+0000 7fa2e9a00700 1 --2- 172.21.15.60:0/3448582298 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa2e410fb60 0x7fa2e41a4c30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc00b2e0 tx=0x7fa2dc005590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:46:48.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:48.479+0000 7fa2e1600700 1 -- 172.21.15.60:0/3448582298 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2dc026070 con 0x7fa2e410fb60 2024-05-03T14:46:48.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:48.479+0000 7fa2e1600700 1 -- 172.21.15.60:0/3448582298 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7fa2dc0056b0 con 0x7fa2e410fb60 2024-05-03T14:46:48.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:48.479+0000 7fa2eba00700 1 -- 172.21.15.60:0/3448582298 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2e41a5600 con 0x7fa2e410fb60 2024-05-03T14:46:48.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:48.479+0000 7fa2e1600700 1 -- 172.21.15.60:0/3448582298 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2dc017650 con 0x7fa2e410fb60 2024-05-03T14:46:48.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:48.479+0000 7fa2eba00700 1 -- 172.21.15.60:0/3448582298 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2e41acee0 con 0x7fa2e410fb60 2024-05-03T14:46:48.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:48.480+0000 7fa2eba00700 1 -- 172.21.15.60:0/3448582298 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2e406a190 con 0x7fa2e410fb60 2024-05-03T14:46:48.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:48.482+0000 7fa2e1600700 1 -- 172.21.15.60:0/3448582298 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fa2dc029070 con 0x7fa2e410fb60 2024-05-03T14:46:48.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:48.484+0000 7fa2e1600700 1 --2- 172.21.15.60:0/3448582298 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fa2cc0b2990 0x7fa2cc0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:46:48.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:48.484+0000 7fa2e1600700 1 -- 172.21.15.60:0/3448582298 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7fa2dc0e35b0 con 0x7fa2e410fb60 2024-05-03T14:46:48.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:48.484+0000 7fa2e9000700 1 --2- 172.21.15.60:0/3448582298 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fa2cc0b2990 0x7fa2cc0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:46:48.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:48.485+0000 7fa2e9000700 1 --2- 172.21.15.60:0/3448582298 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fa2cc0b2990 0x7fa2cc0b4e50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa2d800f540 tx=0x7fa2d800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:46:48.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:48.489+0000 7fa2e1600700 1 -- 172.21.15.60:0/3448582298 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fa2dc013070 con 0x7fa2e410fb60 2024-05-03T14:46:48.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:48.731+0000 7fa2eba00700 1 -- 172.21.15.60:0/3448582298 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa2e41ad0b0 con 0x7fa2e410fb60 2024-05-03T14:46:48.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:48.733+0000 7fa2e1600700 1 -- 172.21.15.60:0/3448582298 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24674 (secure 0 0 0) 0x7fa2dc0aae20 con 0x7fa2e410fb60 2024-05-03T14:46:48.736 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:46:48.736 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":89,"fsid":"285d64b2-095a-11ef-bc97-c7b262605968","created":"2024-05-03T14:34:53.560460+0000","modified":"2024-05-03T14:46:05.582395+0000","last_up_change":"2024-05-03T14:43:14.877043+0000","last_in_change":"2024-05-03T14:42:55.085959+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-03T14:38:54.333714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-03T14:44:25.608599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-03T14:44:30.154009+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-03T14:44:42.549502+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f0797730-c4c1-47ea-9dce-9270d5ebb4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2952930162},{"type":"v1","addr":"172.21.15.60:6803","nonce":2952930162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2952930162},{"type":"v1","addr":"172.21.15.60:6805","nonce":2952930162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":2952930162},{"type":"v1","addr":"172.21.15.60:6809","nonce":2952930162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2952930162},{"type":"v1","addr":"172.21.15.60:6807","nonce":2952930162}]},"public_addr":"172.21.15.60:6803/2952930162","cluster_addr":"172.21.15.60:6805/2952930162","heartbeat_back_addr":"172.21.15.60:6809/2952930162","heartbeat_front_addr":"172.21.15.60:6807/2952930162","state":["exists","up"]},{"osd":1,"uuid":"96d0ad08-cda4-4824-bad4-e64973508c0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":720063242},{"type":"v1","addr":"172.21.15.90:6801","nonce":720063242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":720063242},{"type":"v1","addr":"172.21.15.90:6803","nonce":720063242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":720063242},{"type":"v1","addr":"172.21.15.90:6807","nonce":720063242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":720063242},{"type":"v1","addr":"172.21.15.90:6805","nonce":720063242}]},"public_addr":"172.21.15.90:6801/720063242","cluster_addr":"172.21.15.90:6803/720063242","heartbeat_back_addr":"172.21.15.90:6807/720063242","heartbeat_front_addr":"172.21.15.90:6805/720063242","state":["exists","up"]},{"osd":2,"uuid":"ce95fb7c-1611-4dd1-aee7-97c420d6a466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1359662077},{"type":"v1","addr":"172.21.15.149:6801","nonce":1359662077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1359662077},{"type":"v1","addr":"172.21.15.149:6803","nonce":1359662077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1359662077},{"type":"v1","addr":"172.21.15.149:6807","nonce":1359662077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1359662077},{"type":"v1","addr":"172.21.15.149:6805","nonce":1359662077}]},"public_addr":"172.21.15.149:6801/1359662077","cluster_addr":"172.21.15.149:6803/1359662077","heartbeat_back_addr":"172.21.15.149:6807/1359662077","heartbeat_front_addr":"172.21.15.149:6805/1359662077","state":["exists","up"]},{"osd":3,"uuid":"ef31136a-9b36-4b87-8389-f53d311bc5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3226133334},{"type":"v1","addr":"172.21.15.60:6811","nonce":3226133334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3226133334},{"type":"v1","addr":"172.21.15.60:6813","nonce":3226133334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3226133334},{"type":"v1","addr":"172.21.15.60:6817","nonce":3226133334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3226133334},{"type":"v1","addr":"172.21.15.60:6815","nonce":3226133334}]},"public_addr":"172.21.15.60:6811/3226133334","cluster_addr":"172.21.15.60:6813/3226133334","heartbeat_back_addr":"172.21.15.60:6817/3226133334","heartbeat_front_addr":"172.21.15.60:6815/3226133334","state":["exists","up"]},{"osd":4,"uuid":"73db7408-a2ff-47e9-b13d-d31688a93610","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2201391883},{"type":"v1","addr":"172.21.15.90:6809","nonce":2201391883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":2201391883},{"type":"v1","addr":"172.21.15.90:6811","nonce":2201391883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":2201391883},{"type":"v1","addr":"172.21.15.90:6815","nonce":2201391883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":2201391883},{"type":"v1","addr":"172.21.15.90:6813","nonce":2201391883}]},"public_addr":"172.21.15.90:6809/2201391883","cluster_addr":"172.21.15.90:6811/2201391883","heartbeat_back_addr":"172.21.15.90:6815/2201391883","heartbeat_front_addr":"172.21.15.90:6813/2201391883","state":["exists","up"]},{"osd":5,"uuid":"aefb78a2-522a-4b35-92b8-e5c04fa47b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2139732836},{"type":"v1","addr":"172.21.15.149:6809","nonce":2139732836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":2139732836},{"type":"v1","addr":"172.21.15.149:6811","nonce":2139732836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":2139732836},{"type":"v1","addr":"172.21.15.149:6815","nonce":2139732836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":2139732836},{"type":"v1","addr":"172.21.15.149:6813","nonce":2139732836}]},"public_addr":"172.21.15.149:6809/2139732836","cluster_addr":"172.21.15.149:6811/2139732836","heartbeat_back_addr":"172.21.15.149:6815/2139732836","heartbeat_front_addr":"172.21.15.149:6813/2139732836","state":["exists","up"]},{"osd":6,"uuid":"7c887819-6182-4636-8cfd-d845c30bd13c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3738521002},{"type":"v1","addr":"172.21.15.60:6819","nonce":3738521002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3738521002},{"type":"v1","addr":"172.21.15.60:6821","nonce":3738521002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3738521002},{"type":"v1","addr":"172.21.15.60:6825","nonce":3738521002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3738521002},{"type":"v1","addr":"172.21.15.60:6823","nonce":3738521002}]},"public_addr":"172.21.15.60:6819/3738521002","cluster_addr":"172.21.15.60:6821/3738521002","heartbeat_back_addr":"172.21.15.60:6825/3738521002","heartbeat_front_addr":"172.21.15.60:6823/3738521002","state":["exists","up"]},{"osd":7,"uuid":"ebce6acf-8873-4b35-bd5f-80de2163a307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":1686298661},{"type":"v1","addr":"172.21.15.90:6817","nonce":1686298661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":1686298661},{"type":"v1","addr":"172.21.15.90:6819","nonce":1686298661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":1686298661},{"type":"v1","addr":"172.21.15.90:6823","nonce":1686298661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1686298661},{"type":"v1","addr":"172.21.15.90:6821","nonce":1686298661}]},"public_addr":"172.21.15.90:6817/1686298661","cluster_addr":"172.21.15.90:6819/1686298661","heartbeat_back_addr":"172.21.15.90:6823/1686298661","heartbeat_front_addr":"172.21.15.90:6821/1686298661","state":["exists","up"]},{"osd":8,"uuid":"4ae04b98-e9b2-4956-83d9-17e5ac3c963f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":2608348248},{"type":"v1","addr":"172.21.15.149:6817","nonce":2608348248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2608348248},{"type":"v1","addr":"172.21.15.149:6819","nonce":2608348248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2608348248},{"type":"v1","addr":"172.21.15.149:6823","nonce":2608348248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2608348248},{"type":"v1","addr":"172.21.15.149:6821","nonce":2608348248}]},"public_addr":"172.21.15.149:6817/2608348248","cluster_addr":"172.21.15.149:6819/2608348248","heartbeat_back_addr":"172.21.15.149:6823/2608348248","heartbeat_front_addr":"172.21.15.149:6821/2608348248","state":["exists","up"]},{"osd":9,"uuid":"d0c0a70d-aa76-47e4-91e8-c343c8c41197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":2156130999},{"type":"v1","addr":"172.21.15.60:6827","nonce":2156130999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":2156130999},{"type":"v1","addr":"172.21.15.60:6829","nonce":2156130999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":2156130999},{"type":"v1","addr":"172.21.15.60:6833","nonce":2156130999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":2156130999},{"type":"v1","addr":"172.21.15.60:6831","nonce":2156130999}]},"public_addr":"172.21.15.60:6827/2156130999","cluster_addr":"172.21.15.60:6829/2156130999","heartbeat_back_addr":"172.21.15.60:6833/2156130999","heartbeat_front_addr":"172.21.15.60:6831/2156130999","state":["exists","up"]},{"osd":10,"uuid":"cdc3d7b8-33c7-4cc4-be4e-6887d15a3d8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":2492593590},{"type":"v1","addr":"172.21.15.90:6825","nonce":2492593590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":2492593590},{"type":"v1","addr":"172.21.15.90:6827","nonce":2492593590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":2492593590},{"type":"v1","addr":"172.21.15.90:6831","nonce":2492593590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":2492593590},{"type":"v1","addr":"172.21.15.90:6829","nonce":2492593590}]},"public_addr":"172.21.15.90:6825/2492593590","cluster_addr":"172.21.15.90:6827/2492593590","heartbeat_back_addr":"172.21.15.90:6831/2492593590","heartbeat_front_addr":"172.21.15.90:6829/2492593590","state":["exists","up"]},{"osd":11,"uuid":"8d17ec1a-bdd5-41f1-9b59-c2b6a527bfbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":1063245126},{"type":"v1","addr":"172.21.15.149:6825","nonce":1063245126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":1063245126},{"type":"v1","addr":"172.21.15.149:6827","nonce":1063245126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":1063245126},{"type":"v1","addr":"172.21.15.149:6831","nonce":1063245126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":1063245126},{"type":"v1","addr":"172.21.15.149:6829","nonce":1063245126}]},"public_addr":"172.21.15.149:6825/1063245126","cluster_addr":"172.21.15.149:6827/1063245126","heartbeat_back_addr":"172.21.15.149:6831/1063245126","heartbeat_front_addr":"172.21.15.149:6829/1063245126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:38:02.056563+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:38:27.140677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:38:51.287070+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:39:25.484410+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:39:51.483133+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:40:17.200423+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:40:51.979804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:41:18.742542+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:41:44.922541+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:42:19.778451+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:42:47.438440+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:43:13.429419+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6801/424391417":"2024-05-04T14:36:04.154333+0000","172.21.15.60:6800/424391417":"2024-05-04T14:36:04.154333+0000","172.21.15.60:0/1426532561":"2024-05-04T14:36:04.154333+0000","172.21.15.60:0/3230079926":"2024-05-04T14:35:28.066961+0000","172.21.15.60:0/2334895610":"2024-05-04T14:35:28.066961+0000","172.21.15.60:6800/888447185":"2024-05-04T14:35:28.066961+0000","172.21.15.60:0/3996899937":"2024-05-04T14:35:28.066961+0000","172.21.15.60:0/2779644018":"2024-05-04T14:36:04.154333+0000","172.21.15.60:0/71440838":"2024-05-04T14:36:04.154333+0000","172.21.15.60:6801/888447185":"2024-05-04T14:35:28.066961+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-03T14:46:48.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:48.738+0000 7fa2eba00700 1 -- 172.21.15.60:0/3448582298 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fa2cc0b2990 msgr2=0x7fa2cc0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:48.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:48.738+0000 7fa2eba00700 1 --2- 172.21.15.60:0/3448582298 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fa2cc0b2990 0x7fa2cc0b4e50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa2d800f540 tx=0x7fa2d800b040 comp rx=0 tx=0).stop 2024-05-03T14:46:48.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:48.738+0000 7fa2eba00700 1 -- 172.21.15.60:0/3448582298 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa2e410fb60 msgr2=0x7fa2e41a4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:48.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:48.738+0000 7fa2eba00700 1 --2- 172.21.15.60:0/3448582298 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa2e410fb60 0x7fa2e41a4c30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc00b2e0 tx=0x7fa2dc005590 comp rx=0 tx=0).stop 2024-05-03T14:46:48.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:48.738+0000 7fa2eba00700 1 -- 172.21.15.60:0/3448582298 shutdown_connections 2024-05-03T14:46:48.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:48.738+0000 7fa2eba00700 1 --2- 172.21.15.60:0/3448582298 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fa2cc0b2990 0x7fa2cc0b4e50 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:48.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:48.738+0000 7fa2eba00700 1 --2- 172.21.15.60:0/3448582298 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa2e410fb60 0x7fa2e41a4c30 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:48.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:48.738+0000 7fa2eba00700 1 --2- 172.21.15.60:0/3448582298 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2e4109ad0 0x7fa2e41a0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:48.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:48.738+0000 7fa2eba00700 1 --2- 172.21.15.60:0/3448582298 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2e410a490 0x7fa2e41a08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:48.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:48.738+0000 7fa2eba00700 1 -- 172.21.15.60:0/3448582298 >> 172.21.15.60:0/3448582298 conn(0x7fa2e4076c70 msgr2=0x7fa2e4077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:46:48.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:48.739+0000 7fa2eba00700 1 -- 172.21.15.60:0/3448582298 shutdown_connections 2024-05-03T14:46:48.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:48.739+0000 7fa2eba00700 1 -- 172.21.15.60:0/3448582298 wait complete. 2024-05-03T14:46:49.727 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:49 smithi060 ceph-mon[28873]: pgmap v401: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-03T14:46:49.727 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:49 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/3448582298' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T14:46:49.731 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-03T14:46:49.732 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-03T14:46:49.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:49 smithi090 ceph-mon[33568]: pgmap v401: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-03T14:46:49.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:49 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/3448582298' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T14:46:49.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:49 smithi149 ceph-mon[33332]: pgmap v401: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-03T14:46:49.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:49 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/3448582298' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T14:46:50.172 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:46:51.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:51 smithi060 ceph-mon[28873]: pgmap v402: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-03T14:46:51.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:51 smithi090 ceph-mon[33568]: pgmap v402: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-03T14:46:51.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:51 smithi149 ceph-mon[33332]: pgmap v402: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-03T14:46:52.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:52.394+0000 7f234b400700 1 -- 172.21.15.60:0/2365493173 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f234406bad0 msgr2=0x7f234410fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:52.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:52.394+0000 7f234b400700 1 --2- 172.21.15.60:0/2365493173 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f234406bad0 0x7f234410fa10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2330009990 tx=0x7f2330009ca0 comp rx=0 tx=0).stop 2024-05-03T14:46:52.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:52.394+0000 7f234b400700 1 -- 172.21.15.60:0/2365493173 shutdown_connections 2024-05-03T14:46:52.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:52.394+0000 7f234b400700 1 --2- 172.21.15.60:0/2365493173 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f234410ff50 0x7f2344112340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:52.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:52.394+0000 7f234b400700 1 --2- 172.21.15.60:0/2365493173 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f234406b1b0 0x7f234406b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:52.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:52.394+0000 7f234b400700 1 --2- 172.21.15.60:0/2365493173 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f234406bad0 0x7f234410fa10 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:52.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:52.394+0000 7f234b400700 1 -- 172.21.15.60:0/2365493173 >> 172.21.15.60:0/2365493173 conn(0x7f2344079350 msgr2=0x7f2344079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:46:52.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:52.394+0000 7f234b400700 1 -- 172.21.15.60:0/2365493173 shutdown_connections 2024-05-03T14:46:52.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:52.394+0000 7f234b400700 1 -- 172.21.15.60:0/2365493173 wait complete. 2024-05-03T14:46:52.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:52.395+0000 7f234b400700 1 Processor -- start 2024-05-03T14:46:52.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:52.396+0000 7f234b400700 1 -- start start 2024-05-03T14:46:52.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:52.396+0000 7f234b400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f234406b1b0 0x7f234419bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:46:52.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:52.396+0000 7f234b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f234406bad0 0x7f234419c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:46:52.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:52.396+0000 7f234b400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f234410ff50 0x7f23441a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:46:52.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:52.396+0000 7f234b400700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f2344115b90 con 0x7f234406b1b0 2024-05-03T14:46:52.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:52.396+0000 7f234b400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2344115a10 con 0x7f234406bad0 2024-05-03T14:46:52.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:52.396+0000 7f234b400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2344115d10 con 0x7f234410ff50 2024-05-03T14:46:52.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:52.397+0000 7f2348a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f234406b1b0 0x7f234419bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:46:52.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:52.397+0000 7f2348a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f234406b1b0 0x7f234419bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38870/0 (socket says 172.21.15.60:38870) 2024-05-03T14:46:52.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:52.397+0000 7f2348a00700 1 -- 172.21.15.60:0/4011500557 learned_addr learned my addr 172.21.15.60:0/4011500557 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:46:52.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:52.397+0000 7f2343e00700 1 --2- 172.21.15.60:0/4011500557 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f234406bad0 0x7f234419c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:46:52.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:52.397+0000 7f2349400700 1 --2- 172.21.15.60:0/4011500557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f234410ff50 0x7f23441a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:46:52.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:52.397+0000 7f2343e00700 1 -- 172.21.15.60:0/4011500557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f234410ff50 msgr2=0x7f23441a08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:52.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:52.397+0000 7f2343e00700 1 --2- 172.21.15.60:0/4011500557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f234410ff50 0x7f23441a08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:52.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:52.397+0000 7f2343e00700 1 -- 172.21.15.60:0/4011500557 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f234406b1b0 msgr2=0x7f234419bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:52.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:52.397+0000 7f2343e00700 1 --2- 172.21.15.60:0/4011500557 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f234406b1b0 0x7f234419bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:52.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:52.397+0000 7f2343e00700 1 -- 172.21.15.60:0/4011500557 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23441a0ff0 con 0x7f234406bad0 2024-05-03T14:46:52.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:52.397+0000 7f2348a00700 1 --2- 172.21.15.60:0/4011500557 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f234406b1b0 0x7f234419bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:46:52.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:52.397+0000 7f2343e00700 1 --2- 172.21.15.60:0/4011500557 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f234406bad0 0x7f234419c520 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f23300098e0 tx=0x7f2330005590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:46:52.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:52.398+0000 7f2341600700 1 -- 172.21.15.60:0/4011500557 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2330027070 con 0x7f234406bad0 2024-05-03T14:46:52.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:52.398+0000 7f234b400700 1 -- 172.21.15.60:0/4011500557 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23441a1280 con 0x7f234406bad0 2024-05-03T14:46:52.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:52.398+0000 7f234b400700 1 -- 172.21.15.60:0/4011500557 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23441a8b60 con 0x7f234406bad0 2024-05-03T14:46:52.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:52.398+0000 7f2341600700 1 -- 172.21.15.60:0/4011500557 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f233000fe00 con 0x7f234406bad0 2024-05-03T14:46:52.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:52.398+0000 7f2341600700 1 -- 172.21.15.60:0/4011500557 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23300173f0 con 0x7f234406bad0 2024-05-03T14:46:52.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:52.400+0000 7f234b400700 1 -- 172.21.15.60:0/4011500557 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2324005330 con 0x7f234406bad0 2024-05-03T14:46:52.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:52.401+0000 7f2341600700 1 -- 172.21.15.60:0/4011500557 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f233002a070 con 0x7f234406bad0 2024-05-03T14:46:52.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:52.403+0000 7f2341600700 1 --2- 172.21.15.60:0/4011500557 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f232c0b2910 0x7f232c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:46:52.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:52.403+0000 7f2341600700 1 -- 172.21.15.60:0/4011500557 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f23300e32d0 con 0x7f234406bad0 2024-05-03T14:46:52.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:52.403+0000 7f2348a00700 1 --2- 172.21.15.60:0/4011500557 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f232c0b2910 0x7f232c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:46:52.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:52.404+0000 7f2348a00700 1 --2- 172.21.15.60:0/4011500557 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f232c0b2910 0x7f232c0b4dd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2338011550 tx=0x7f2338009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:46:52.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:52.408+0000 7f2341600700 1 -- 172.21.15.60:0/4011500557 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f2330013070 con 0x7f234406bad0 2024-05-03T14:46:52.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:52.664+0000 7f234b400700 1 -- 172.21.15.60:0/4011500557 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f23240058f0 con 0x7f234406bad0 2024-05-03T14:46:52.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:52.666+0000 7f2341600700 1 -- 172.21.15.60:0/4011500557 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24674 (secure 0 0 0) 0x7f23300aab40 con 0x7f234406bad0 2024-05-03T14:46:52.668 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:46:52.669 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":89,"fsid":"285d64b2-095a-11ef-bc97-c7b262605968","created":"2024-05-03T14:34:53.560460+0000","modified":"2024-05-03T14:46:05.582395+0000","last_up_change":"2024-05-03T14:43:14.877043+0000","last_in_change":"2024-05-03T14:42:55.085959+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-03T14:38:54.333714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-03T14:44:25.608599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-03T14:44:30.154009+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-03T14:44:42.549502+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f0797730-c4c1-47ea-9dce-9270d5ebb4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2952930162},{"type":"v1","addr":"172.21.15.60:6803","nonce":2952930162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2952930162},{"type":"v1","addr":"172.21.15.60:6805","nonce":2952930162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":2952930162},{"type":"v1","addr":"172.21.15.60:6809","nonce":2952930162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2952930162},{"type":"v1","addr":"172.21.15.60:6807","nonce":2952930162}]},"public_addr":"172.21.15.60:6803/2952930162","cluster_addr":"172.21.15.60:6805/2952930162","heartbeat_back_addr":"172.21.15.60:6809/2952930162","heartbeat_front_addr":"172.21.15.60:6807/2952930162","state":["exists","up"]},{"osd":1,"uuid":"96d0ad08-cda4-4824-bad4-e64973508c0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":720063242},{"type":"v1","addr":"172.21.15.90:6801","nonce":720063242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":720063242},{"type":"v1","addr":"172.21.15.90:6803","nonce":720063242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":720063242},{"type":"v1","addr":"172.21.15.90:6807","nonce":720063242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":720063242},{"type":"v1","addr":"172.21.15.90:6805","nonce":720063242}]},"public_addr":"172.21.15.90:6801/720063242","cluster_addr":"172.21.15.90:6803/720063242","heartbeat_back_addr":"172.21.15.90:6807/720063242","heartbeat_front_addr":"172.21.15.90:6805/720063242","state":["exists","up"]},{"osd":2,"uuid":"ce95fb7c-1611-4dd1-aee7-97c420d6a466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1359662077},{"type":"v1","addr":"172.21.15.149:6801","nonce":1359662077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1359662077},{"type":"v1","addr":"172.21.15.149:6803","nonce":1359662077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1359662077},{"type":"v1","addr":"172.21.15.149:6807","nonce":1359662077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1359662077},{"type":"v1","addr":"172.21.15.149:6805","nonce":1359662077}]},"public_addr":"172.21.15.149:6801/1359662077","cluster_addr":"172.21.15.149:6803/1359662077","heartbeat_back_addr":"172.21.15.149:6807/1359662077","heartbeat_front_addr":"172.21.15.149:6805/1359662077","state":["exists","up"]},{"osd":3,"uuid":"ef31136a-9b36-4b87-8389-f53d311bc5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3226133334},{"type":"v1","addr":"172.21.15.60:6811","nonce":3226133334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3226133334},{"type":"v1","addr":"172.21.15.60:6813","nonce":3226133334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3226133334},{"type":"v1","addr":"172.21.15.60:6817","nonce":3226133334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3226133334},{"type":"v1","addr":"172.21.15.60:6815","nonce":3226133334}]},"public_addr":"172.21.15.60:6811/3226133334","cluster_addr":"172.21.15.60:6813/3226133334","heartbeat_back_addr":"172.21.15.60:6817/3226133334","heartbeat_front_addr":"172.21.15.60:6815/3226133334","state":["exists","up"]},{"osd":4,"uuid":"73db7408-a2ff-47e9-b13d-d31688a93610","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2201391883},{"type":"v1","addr":"172.21.15.90:6809","nonce":2201391883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":2201391883},{"type":"v1","addr":"172.21.15.90:6811","nonce":2201391883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":2201391883},{"type":"v1","addr":"172.21.15.90:6815","nonce":2201391883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":2201391883},{"type":"v1","addr":"172.21.15.90:6813","nonce":2201391883}]},"public_addr":"172.21.15.90:6809/2201391883","cluster_addr":"172.21.15.90:6811/2201391883","heartbeat_back_addr":"172.21.15.90:6815/2201391883","heartbeat_front_addr":"172.21.15.90:6813/2201391883","state":["exists","up"]},{"osd":5,"uuid":"aefb78a2-522a-4b35-92b8-e5c04fa47b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2139732836},{"type":"v1","addr":"172.21.15.149:6809","nonce":2139732836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":2139732836},{"type":"v1","addr":"172.21.15.149:6811","nonce":2139732836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":2139732836},{"type":"v1","addr":"172.21.15.149:6815","nonce":2139732836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":2139732836},{"type":"v1","addr":"172.21.15.149:6813","nonce":2139732836}]},"public_addr":"172.21.15.149:6809/2139732836","cluster_addr":"172.21.15.149:6811/2139732836","heartbeat_back_addr":"172.21.15.149:6815/2139732836","heartbeat_front_addr":"172.21.15.149:6813/2139732836","state":["exists","up"]},{"osd":6,"uuid":"7c887819-6182-4636-8cfd-d845c30bd13c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3738521002},{"type":"v1","addr":"172.21.15.60:6819","nonce":3738521002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3738521002},{"type":"v1","addr":"172.21.15.60:6821","nonce":3738521002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3738521002},{"type":"v1","addr":"172.21.15.60:6825","nonce":3738521002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3738521002},{"type":"v1","addr":"172.21.15.60:6823","nonce":3738521002}]},"public_addr":"172.21.15.60:6819/3738521002","cluster_addr":"172.21.15.60:6821/3738521002","heartbeat_back_addr":"172.21.15.60:6825/3738521002","heartbeat_front_addr":"172.21.15.60:6823/3738521002","state":["exists","up"]},{"osd":7,"uuid":"ebce6acf-8873-4b35-bd5f-80de2163a307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":1686298661},{"type":"v1","addr":"172.21.15.90:6817","nonce":1686298661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":1686298661},{"type":"v1","addr":"172.21.15.90:6819","nonce":1686298661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":1686298661},{"type":"v1","addr":"172.21.15.90:6823","nonce":1686298661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1686298661},{"type":"v1","addr":"172.21.15.90:6821","nonce":1686298661}]},"public_addr":"172.21.15.90:6817/1686298661","cluster_addr":"172.21.15.90:6819/1686298661","heartbeat_back_addr":"172.21.15.90:6823/1686298661","heartbeat_front_addr":"172.21.15.90:6821/1686298661","state":["exists","up"]},{"osd":8,"uuid":"4ae04b98-e9b2-4956-83d9-17e5ac3c963f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":2608348248},{"type":"v1","addr":"172.21.15.149:6817","nonce":2608348248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2608348248},{"type":"v1","addr":"172.21.15.149:6819","nonce":2608348248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2608348248},{"type":"v1","addr":"172.21.15.149:6823","nonce":2608348248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2608348248},{"type":"v1","addr":"172.21.15.149:6821","nonce":2608348248}]},"public_addr":"172.21.15.149:6817/2608348248","cluster_addr":"172.21.15.149:6819/2608348248","heartbeat_back_addr":"172.21.15.149:6823/2608348248","heartbeat_front_addr":"172.21.15.149:6821/2608348248","state":["exists","up"]},{"osd":9,"uuid":"d0c0a70d-aa76-47e4-91e8-c343c8c41197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":2156130999},{"type":"v1","addr":"172.21.15.60:6827","nonce":2156130999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":2156130999},{"type":"v1","addr":"172.21.15.60:6829","nonce":2156130999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":2156130999},{"type":"v1","addr":"172.21.15.60:6833","nonce":2156130999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":2156130999},{"type":"v1","addr":"172.21.15.60:6831","nonce":2156130999}]},"public_addr":"172.21.15.60:6827/2156130999","cluster_addr":"172.21.15.60:6829/2156130999","heartbeat_back_addr":"172.21.15.60:6833/2156130999","heartbeat_front_addr":"172.21.15.60:6831/2156130999","state":["exists","up"]},{"osd":10,"uuid":"cdc3d7b8-33c7-4cc4-be4e-6887d15a3d8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":2492593590},{"type":"v1","addr":"172.21.15.90:6825","nonce":2492593590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":2492593590},{"type":"v1","addr":"172.21.15.90:6827","nonce":2492593590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":2492593590},{"type":"v1","addr":"172.21.15.90:6831","nonce":2492593590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":2492593590},{"type":"v1","addr":"172.21.15.90:6829","nonce":2492593590}]},"public_addr":"172.21.15.90:6825/2492593590","cluster_addr":"172.21.15.90:6827/2492593590","heartbeat_back_addr":"172.21.15.90:6831/2492593590","heartbeat_front_addr":"172.21.15.90:6829/2492593590","state":["exists","up"]},{"osd":11,"uuid":"8d17ec1a-bdd5-41f1-9b59-c2b6a527bfbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":1063245126},{"type":"v1","addr":"172.21.15.149:6825","nonce":1063245126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":1063245126},{"type":"v1","addr":"172.21.15.149:6827","nonce":1063245126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":1063245126},{"type":"v1","addr":"172.21.15.149:6831","nonce":1063245126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":1063245126},{"type":"v1","addr":"172.21.15.149:6829","nonce":1063245126}]},"public_addr":"172.21.15.149:6825/1063245126","cluster_addr":"172.21.15.149:6827/1063245126","heartbeat_back_addr":"172.21.15.149:6831/1063245126","heartbeat_front_addr":"172.21.15.149:6829/1063245126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:38:02.056563+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:38:27.140677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:38:51.287070+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:39:25.484410+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:39:51.483133+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:40:17.200423+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:40:51.979804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:41:18.742542+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:41:44.922541+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:42:19.778451+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:42:47.438440+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:43:13.429419+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6801/424391417":"2024-05-04T14:36:04.154333+0000","172.21.15.60:6800/424391417":"2024-05-04T14:36:04.154333+0000","172.21.15.60:0/1426532561":"2024-05-04T14:36:04.154333+0000","172.21.15.60:0/3230079926":"2024-05-04T14:35:28.066961+0000","172.21.15.60:0/2334895610":"2024-05-04T14:35:28.066961+0000","172.21.15.60:6800/888447185":"2024-05-04T14:35:28.066961+0000","172.21.15.60:0/3996899937":"2024-05-04T14:35:28.066961+0000","172.21.15.60:0/2779644018":"2024-05-04T14:36:04.154333+0000","172.21.15.60:0/71440838":"2024-05-04T14:36:04.154333+0000","172.21.15.60:6801/888447185":"2024-05-04T14:35:28.066961+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-03T14:46:52.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:52.669+0000 7f234b400700 1 -- 172.21.15.60:0/4011500557 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f232c0b2910 msgr2=0x7f232c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:52.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:52.669+0000 7f234b400700 1 --2- 172.21.15.60:0/4011500557 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f232c0b2910 0x7f232c0b4dd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2338011550 tx=0x7f2338009260 comp rx=0 tx=0).stop 2024-05-03T14:46:52.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:52.669+0000 7f234b400700 1 -- 172.21.15.60:0/4011500557 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f234406bad0 msgr2=0x7f234419c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:46:52.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:52.669+0000 7f234b400700 1 --2- 172.21.15.60:0/4011500557 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f234406bad0 0x7f234419c520 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f23300098e0 tx=0x7f2330005590 comp rx=0 tx=0).stop 2024-05-03T14:46:52.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:52.669+0000 7f234b400700 1 -- 172.21.15.60:0/4011500557 shutdown_connections 2024-05-03T14:46:52.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:52.669+0000 7f234b400700 1 --2- 172.21.15.60:0/4011500557 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f232c0b2910 0x7f232c0b4dd0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:52.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:52.669+0000 7f234b400700 1 --2- 172.21.15.60:0/4011500557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f234410ff50 0x7f23441a08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:52.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:52.669+0000 7f234b400700 1 --2- 172.21.15.60:0/4011500557 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f234406b1b0 0x7f234419bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:52.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:52.669+0000 7f234b400700 1 --2- 172.21.15.60:0/4011500557 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f234406bad0 0x7f234419c520 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:46:52.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:52.669+0000 7f234b400700 1 -- 172.21.15.60:0/4011500557 >> 172.21.15.60:0/4011500557 conn(0x7f2344079350 msgr2=0x7f23441112c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:46:52.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:52.670+0000 7f234b400700 1 -- 172.21.15.60:0/4011500557 shutdown_connections 2024-05-03T14:46:52.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:46:52.670+0000 7f234b400700 1 -- 172.21.15.60:0/4011500557 wait complete. 2024-05-03T14:46:53.416 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-03T14:46:53.417 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-03T14:46:53.417 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-03T14:46:53.417 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-03T14:46:53.418 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-03T14:46:53.418 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-03T14:46:53.419 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-05-03T14:46:53.419 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-05-03T14:46:53.419 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-05-03T14:46:53.420 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-05-03T14:46:53.420 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-05-03T14:46:53.421 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-05-03T14:46:53.597 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:53 smithi060 ceph-mon[28873]: pgmap v403: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T14:46:53.597 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:53 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/4011500557' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T14:46:53.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:53 smithi090 ceph-mon[33568]: pgmap v403: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T14:46:53.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:53 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/4011500557' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T14:46:53.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:53 smithi149 ceph-mon[33332]: pgmap v403: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T14:46:53.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:53 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/4011500557' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T14:46:54.470 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:46:54.493 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:46:54.495 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:46:54.498 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:46:54.537 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:46:54.594 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:46:54.599 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:46:54.637 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:46:54.854 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:46:54.961 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:46:55.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:55 smithi060 ceph-mon[28873]: pgmap v404: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:46:55.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:55 smithi090 ceph-mon[33568]: pgmap v404: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:46:55.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:55 smithi149 ceph-mon[33332]: pgmap v404: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:46:57.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:57 smithi060 ceph-mon[28873]: pgmap v405: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:46:57.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:57 smithi149 ceph-mon[33332]: pgmap v405: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:46:57.624 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:57 smithi090 ceph-mon[33568]: pgmap v405: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:46:59.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:46:59 smithi060 ceph-mon[28873]: pgmap v406: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:46:59.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:46:59 smithi090 ceph-mon[33568]: pgmap v406: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:46:59.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:46:59 smithi149 ceph-mon[33332]: pgmap v406: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:47:01.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:47:01 smithi060 ceph-mon[28873]: pgmap v407: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:47:01.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:47:01 smithi090 ceph-mon[33568]: pgmap v407: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:47:01.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:47:01 smithi149 ceph-mon[33332]: pgmap v407: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:47:02.734 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:47:02.742 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:47:03.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:47:03 smithi060 ceph-mon[28873]: pgmap v408: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:47:03.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:47:03 smithi090 ceph-mon[33568]: pgmap v408: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:47:03.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:47:03 smithi149 ceph-mon[33332]: pgmap v408: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:47:04.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:03.634+0000 7fc4fee00700 1 -- 172.21.15.60:0/546472744 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc4f80779d0 msgr2=0x7fc4f8077db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:04.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:03.634+0000 7fc4fee00700 1 --2- 172.21.15.60:0/546472744 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc4f80779d0 0x7fc4f8077db0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8009990 tx=0x7fc4e8009ca0 comp rx=0 tx=0).stop 2024-05-03T14:47:04.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:03.635+0000 7fc4fee00700 1 -- 172.21.15.60:0/546472744 shutdown_connections 2024-05-03T14:47:04.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:03.635+0000 7fc4fee00700 1 --2- 172.21.15.60:0/546472744 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4f8114330 0x7fc4f8116720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:04.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:03.635+0000 7fc4fee00700 1 --2- 172.21.15.60:0/546472744 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc4f80779d0 0x7fc4f8077db0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:04.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:03.635+0000 7fc4fee00700 1 --2- 172.21.15.60:0/546472744 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc4f80782f0 0x7fc4f8113df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:04.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:03.635+0000 7fc4fee00700 1 -- 172.21.15.60:0/546472744 >> 172.21.15.60:0/546472744 conn(0x7fc4f8101180 msgr2=0x7fc4f81035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:47:04.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:03.635+0000 7fc4fee00700 1 -- 172.21.15.60:0/546472744 shutdown_connections 2024-05-03T14:47:04.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:03.635+0000 7fc4fee00700 1 -- 172.21.15.60:0/546472744 wait complete. 2024-05-03T14:47:04.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:03.636+0000 7fc4fee00700 1 Processor -- start 2024-05-03T14:47:04.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:03.636+0000 7fc4fee00700 1 -- start start 2024-05-03T14:47:04.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:03.636+0000 7fc4fee00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4f80779d0 0x7fc4f81a03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:04.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:03.636+0000 7fc4fee00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc4f80782f0 0x7fc4f81a08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:04.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:03.637+0000 7fc4fee00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc4f8114330 0x7fc4f81a4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:04.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:03.637+0000 7fc4fee00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f8119f30 con 0x7fc4f80779d0 2024-05-03T14:47:04.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:03.637+0000 7fc4fee00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f8119db0 con 0x7fc4f80782f0 2024-05-03T14:47:04.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:03.637+0000 7fc4fee00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f811a0b0 con 0x7fc4f8114330 2024-05-03T14:47:04.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:03.637+0000 7fc4f7e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4f80779d0 0x7fc4f81a03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:04.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:03.637+0000 7fc4f7e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4f80779d0 0x7fc4f81a03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49030/0 (socket says 172.21.15.60:49030) 2024-05-03T14:47:04.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:03.637+0000 7fc4f7e00700 1 -- 172.21.15.60:0/97320930 learned_addr learned my addr 172.21.15.60:0/97320930 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:47:04.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:03.637+0000 7fc4fce00700 1 --2- 172.21.15.60:0/97320930 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc4f8114330 0x7fc4f81a4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:04.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:03.637+0000 7fc4f7400700 1 --2- 172.21.15.60:0/97320930 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc4f80782f0 0x7fc4f81a08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:04.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:03.637+0000 7fc4f7e00700 1 -- 172.21.15.60:0/97320930 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc4f8114330 msgr2=0x7fc4f81a4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:04.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:03.637+0000 7fc4f7e00700 1 --2- 172.21.15.60:0/97320930 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc4f8114330 0x7fc4f81a4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:04.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:03.637+0000 7fc4f7e00700 1 -- 172.21.15.60:0/97320930 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc4f80782f0 msgr2=0x7fc4f81a08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:04.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:03.637+0000 7fc4f7e00700 1 --2- 172.21.15.60:0/97320930 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc4f80782f0 0x7fc4f81a08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:04.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:03.637+0000 7fc4f7e00700 1 -- 172.21.15.60:0/97320930 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4f81a53b0 con 0x7fc4f80779d0 2024-05-03T14:47:04.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:03.638+0000 7fc4f7e00700 1 --2- 172.21.15.60:0/97320930 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4f80779d0 0x7fc4f81a03a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc4e80095c0 tx=0x7fc4e8005590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:04.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:03.638+0000 7fc4f4c00700 1 -- 172.21.15.60:0/97320930 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e8026070 con 0x7fc4f80779d0 2024-05-03T14:47:04.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:03.638+0000 7fc4f4c00700 1 -- 172.21.15.60:0/97320930 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7fc4e800fe30 con 0x7fc4f80779d0 2024-05-03T14:47:04.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:03.638+0000 7fc4f4c00700 1 -- 172.21.15.60:0/97320930 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e80173f0 con 0x7fc4f80779d0 2024-05-03T14:47:04.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:03.638+0000 7fc4fee00700 1 -- 172.21.15.60:0/97320930 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4f81a5640 con 0x7fc4f80779d0 2024-05-03T14:47:04.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:03.638+0000 7fc4fee00700 1 -- 172.21.15.60:0/97320930 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4f81acf20 con 0x7fc4f80779d0 2024-05-03T14:47:04.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:03.639+0000 7fc4fee00700 1 -- 172.21.15.60:0/97320930 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc4f81a5ad0 con 0x7fc4f80779d0 2024-05-03T14:47:04.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:03.643+0000 7fc4f4c00700 1 -- 172.21.15.60:0/97320930 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fc4e8029070 con 0x7fc4f80779d0 2024-05-03T14:47:04.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:03.645+0000 7fc4f4c00700 1 --2- 172.21.15.60:0/97320930 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fc4dc0b2990 0x7fc4dc0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:04.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:03.645+0000 7fc4f4c00700 1 -- 172.21.15.60:0/97320930 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7fc4e80e34e0 con 0x7fc4f80779d0 2024-05-03T14:47:04.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:03.645+0000 7fc4f7400700 1 --2- 172.21.15.60:0/97320930 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fc4dc0b2990 0x7fc4dc0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:04.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:03.645+0000 7fc4f4c00700 1 --2- 172.21.15.60:0/97320930 >> [v2:172.21.15.149:6808/2139732836,v1:172.21.15.149:6809/2139732836] conn(0x7fc4dc0bdb70 0x7fc4dc0c0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:04.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:03.645+0000 7fc4f4c00700 1 -- 172.21.15.60:0/97320930 --> [v2:172.21.15.149:6808/2139732836,v1:172.21.15.149:6809/2139732836] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc4e800eea0 con 0x7fc4dc0bdb70 2024-05-03T14:47:04.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:03.645+0000 7fc4f4c00700 1 -- 172.21.15.60:0/97320930 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fc4e80e3920 con 0x7fc4f80779d0 2024-05-03T14:47:04.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:03.646+0000 7fc4fce00700 1 --2- 172.21.15.60:0/97320930 >> [v2:172.21.15.149:6808/2139732836,v1:172.21.15.149:6809/2139732836] conn(0x7fc4dc0bdb70 0x7fc4dc0c0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:04.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:03.646+0000 7fc4f7400700 1 --2- 172.21.15.60:0/97320930 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fc4dc0b2990 0x7fc4dc0b4e50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4009680 tx=0x7fc4e4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:04.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:03.646+0000 7fc4fce00700 1 --2- 172.21.15.60:0/97320930 >> [v2:172.21.15.149:6808/2139732836,v1:172.21.15.149:6809/2139732836] conn(0x7fc4dc0bdb70 0x7fc4dc0c0010 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:04.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:03.647+0000 7fc4f4c00700 1 -- 172.21.15.60:0/97320930 <== osd.5 v2:172.21.15.149:6808/2139732836 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc4e800eea0 con 0x7fc4dc0bdb70 2024-05-03T14:47:04.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:03.666+0000 7fc4fee00700 1 -- 172.21.15.60:0/97320930 --> [v2:172.21.15.149:6808/2139732836,v1:172.21.15.149:6809/2139732836] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc4f8063e50 con 0x7fc4dc0bdb70 2024-05-03T14:47:04.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:03.666+0000 7fc4f4c00700 1 -- 172.21.15.60:0/97320930 <== osd.5 v2:172.21.15.149:6808/2139732836 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc4f8063e50 con 0x7fc4dc0bdb70 2024-05-03T14:47:04.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:03.667+0000 7fc4fee00700 1 -- 172.21.15.60:0/97320930 >> [v2:172.21.15.149:6808/2139732836,v1:172.21.15.149:6809/2139732836] conn(0x7fc4dc0bdb70 msgr2=0x7fc4dc0c0010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:04.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:03.667+0000 7fc4fee00700 1 --2- 172.21.15.60:0/97320930 >> [v2:172.21.15.149:6808/2139732836,v1:172.21.15.149:6809/2139732836] conn(0x7fc4dc0bdb70 0x7fc4dc0c0010 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:04.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:03.667+0000 7fc4fee00700 1 -- 172.21.15.60:0/97320930 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fc4dc0b2990 msgr2=0x7fc4dc0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:04.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:03.667+0000 7fc4fee00700 1 --2- 172.21.15.60:0/97320930 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fc4dc0b2990 0x7fc4dc0b4e50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4009680 tx=0x7fc4e4009260 comp rx=0 tx=0).stop 2024-05-03T14:47:04.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:03.667+0000 7fc4fee00700 1 -- 172.21.15.60:0/97320930 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4f80779d0 msgr2=0x7fc4f81a03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:04.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:03.667+0000 7fc4fee00700 1 --2- 172.21.15.60:0/97320930 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4f80779d0 0x7fc4f81a03a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc4e80095c0 tx=0x7fc4e8005590 comp rx=0 tx=0).stop 2024-05-03T14:47:04.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:03.667+0000 7fc4fce00700 1 -- 172.21.15.60:0/97320930 reap_dead start 2024-05-03T14:47:04.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:03.667+0000 7fc4fee00700 1 -- 172.21.15.60:0/97320930 shutdown_connections 2024-05-03T14:47:04.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:03.667+0000 7fc4fee00700 1 -- 172.21.15.60:0/97320930 >> 172.21.15.60:0/97320930 conn(0x7fc4f8101180 msgr2=0x7fc4f8103570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:47:04.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:03.667+0000 7fc4fee00700 1 -- 172.21.15.60:0/97320930 shutdown_connections 2024-05-03T14:47:04.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:03.668+0000 7fc4fee00700 1 -- 172.21.15.60:0/97320930 wait complete. 2024-05-03T14:47:05.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:47:05 smithi060 ceph-mon[28873]: pgmap v409: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:47:05.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:47:05 smithi090 ceph-mon[33568]: pgmap v409: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:47:05.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:47:05 smithi149 ceph-mon[33332]: pgmap v409: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:47:06.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:06.747+0000 7f45c0600700 1 -- 172.21.15.60:0/631799871 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45b8075ff0 msgr2=0x7f45b8114130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:06.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:06.747+0000 7f45c0600700 1 --2- 172.21.15.60:0/631799871 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45b8075ff0 0x7f45b8114130 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f45a4009990 tx=0x7f45a4009ca0 comp rx=0 tx=0).stop 2024-05-03T14:47:06.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:06.747+0000 7f45c0600700 1 -- 172.21.15.60:0/631799871 shutdown_connections 2024-05-03T14:47:06.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:06.747+0000 7f45c0600700 1 --2- 172.21.15.60:0/631799871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f45b8114670 0x7f45b8116a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:06.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:06.747+0000 7f45c0600700 1 --2- 172.21.15.60:0/631799871 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f45b80756d0 0x7f45b8075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:06.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:06.747+0000 7f45c0600700 1 --2- 172.21.15.60:0/631799871 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45b8075ff0 0x7f45b8114130 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:06.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:06.747+0000 7f45c0600700 1 -- 172.21.15.60:0/631799871 >> 172.21.15.60:0/631799871 conn(0x7f45b807a650 msgr2=0x7f45b807aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:47:06.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:06.747+0000 7f45c0600700 1 -- 172.21.15.60:0/631799871 shutdown_connections 2024-05-03T14:47:06.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:06.747+0000 7f45c0600700 1 -- 172.21.15.60:0/631799871 wait complete. 2024-05-03T14:47:06.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:06.748+0000 7f45c0600700 1 Processor -- start 2024-05-03T14:47:06.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:06.749+0000 7f45c0600700 1 -- start start 2024-05-03T14:47:06.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:06.749+0000 7f45c0600700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45b80756d0 0x7f45b81a0340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:06.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:06.749+0000 7f45c0600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f45b8075ff0 0x7f45b81a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:06.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:06.749+0000 7f45c0600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f45b8114670 0x7f45b81a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:06.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:06.749+0000 7f45c0600700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f45b8119f70 con 0x7f45b80756d0 2024-05-03T14:47:06.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:06.749+0000 7f45c0600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f45b8119df0 con 0x7f45b8114670 2024-05-03T14:47:06.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:06.749+0000 7f45c0600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f45b811a0f0 con 0x7f45b8075ff0 2024-05-03T14:47:06.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:06.749+0000 7f45bdc00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45b80756d0 0x7f45b81a0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:06.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:06.749+0000 7f45be600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f45b8114670 0x7f45b81a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:06.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:06.750+0000 7f45bdc00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45b80756d0 0x7f45b81a0340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49056/0 (socket says 172.21.15.60:49056) 2024-05-03T14:47:06.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:06.750+0000 7f45bdc00700 1 -- 172.21.15.60:0/621240249 learned_addr learned my addr 172.21.15.60:0/621240249 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:47:06.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:06.750+0000 7f45be600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f45b8114670 0x7f45b81a4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.60:40548/0 (socket says 172.21.15.60:40548) 2024-05-03T14:47:06.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:06.750+0000 7f45bd200700 1 --2- 172.21.15.60:0/621240249 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f45b8075ff0 0x7f45b81a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:06.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:06.750+0000 7f45be600700 1 -- 172.21.15.60:0/621240249 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f45b8075ff0 msgr2=0x7f45b81a0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:06.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:06.750+0000 7f45be600700 1 --2- 172.21.15.60:0/621240249 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f45b8075ff0 0x7f45b81a0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:06.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:06.750+0000 7f45be600700 1 -- 172.21.15.60:0/621240249 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45b80756d0 msgr2=0x7f45b81a0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:06.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:06.750+0000 7f45be600700 1 --2- 172.21.15.60:0/621240249 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45b80756d0 0x7f45b81a0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:06.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:06.750+0000 7f45be600700 1 -- 172.21.15.60:0/621240249 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45b81a5350 con 0x7f45b8114670 2024-05-03T14:47:06.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:06.750+0000 7f45be600700 1 --2- 172.21.15.60:0/621240249 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f45b8114670 0x7f45b81a4c10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f45b000efc0 tx=0x7f45b000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:06.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:06.751+0000 7f45b6000700 1 -- 172.21.15.60:0/621240249 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b00093a0 con 0x7f45b8114670 2024-05-03T14:47:06.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:06.751+0000 7f45c0600700 1 -- 172.21.15.60:0/621240249 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45b81a55e0 con 0x7f45b8114670 2024-05-03T14:47:06.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:06.751+0000 7f45b6000700 1 -- 172.21.15.60:0/621240249 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f45b000c940 con 0x7f45b8114670 2024-05-03T14:47:06.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:06.751+0000 7f45b6000700 1 -- 172.21.15.60:0/621240249 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b00045a0 con 0x7f45b8114670 2024-05-03T14:47:06.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:06.751+0000 7f45c0600700 1 -- 172.21.15.60:0/621240249 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45b81acec0 con 0x7f45b8114670 2024-05-03T14:47:06.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:06.752+0000 7f45c0600700 1 -- 172.21.15.60:0/621240249 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f45b8113e10 con 0x7f45b8114670 2024-05-03T14:47:06.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:06.754+0000 7f45b6000700 1 -- 172.21.15.60:0/621240249 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f45b001b030 con 0x7f45b8114670 2024-05-03T14:47:06.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:06.756+0000 7f45b6000700 1 --2- 172.21.15.60:0/621240249 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f45a00b2990 0x7f45a00b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:06.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:06.756+0000 7f45b6000700 1 -- 172.21.15.60:0/621240249 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f45b00e25d0 con 0x7f45b8114670 2024-05-03T14:47:06.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:06.756+0000 7f45bdc00700 1 --2- 172.21.15.60:0/621240249 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f45a00b2990 0x7f45a00b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:06.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:06.756+0000 7f45b6000700 1 --2- 172.21.15.60:0/621240249 >> [v2:172.21.15.60:6810/3226133334,v1:172.21.15.60:6811/3226133334] conn(0x7f45a00bdb70 0x7f45a00c0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:06.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:06.756+0000 7f45b6000700 1 -- 172.21.15.60:0/621240249 --> [v2:172.21.15.60:6810/3226133334,v1:172.21.15.60:6811/3226133334] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f45b0024ea0 con 0x7f45a00bdb70 2024-05-03T14:47:06.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:06.756+0000 7f45b6000700 1 -- 172.21.15.60:0/621240249 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f45b00e2980 con 0x7f45b8114670 2024-05-03T14:47:06.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:06.757+0000 7f45bd200700 1 --2- 172.21.15.60:0/621240249 >> [v2:172.21.15.60:6810/3226133334,v1:172.21.15.60:6811/3226133334] conn(0x7f45a00bdb70 0x7f45a00c0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:06.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:06.757+0000 7f45bdc00700 1 --2- 172.21.15.60:0/621240249 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f45a00b2990 0x7f45a00b4e50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f45ac009d20 tx=0x7f45ac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:06.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:06.757+0000 7f45bd200700 1 --2- 172.21.15.60:0/621240249 >> [v2:172.21.15.60:6810/3226133334,v1:172.21.15.60:6811/3226133334] conn(0x7f45a00bdb70 0x7f45a00c0010 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:06.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:06.759+0000 7f45b6000700 1 -- 172.21.15.60:0/621240249 <== osd.3 v2:172.21.15.60:6810/3226133334 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f45b0024ea0 con 0x7f45a00bdb70 2024-05-03T14:47:06.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:06.777+0000 7f45c0600700 1 -- 172.21.15.60:0/621240249 --> [v2:172.21.15.60:6810/3226133334,v1:172.21.15.60:6811/3226133334] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f45b8076df0 con 0x7f45a00bdb70 2024-05-03T14:47:06.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:06.778+0000 7f45b6000700 1 -- 172.21.15.60:0/621240249 <== osd.3 v2:172.21.15.60:6810/3226133334 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f45b8076df0 con 0x7f45a00bdb70 2024-05-03T14:47:06.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:06.778+0000 7f45c0600700 1 -- 172.21.15.60:0/621240249 >> [v2:172.21.15.60:6810/3226133334,v1:172.21.15.60:6811/3226133334] conn(0x7f45a00bdb70 msgr2=0x7f45a00c0010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:06.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:06.778+0000 7f45c0600700 1 --2- 172.21.15.60:0/621240249 >> [v2:172.21.15.60:6810/3226133334,v1:172.21.15.60:6811/3226133334] conn(0x7f45a00bdb70 0x7f45a00c0010 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:06.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:06.778+0000 7f45c0600700 1 -- 172.21.15.60:0/621240249 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f45a00b2990 msgr2=0x7f45a00b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:06.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:06.778+0000 7f45c0600700 1 --2- 172.21.15.60:0/621240249 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f45a00b2990 0x7f45a00b4e50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f45ac009d20 tx=0x7f45ac009260 comp rx=0 tx=0).stop 2024-05-03T14:47:06.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:06.778+0000 7f45c0600700 1 -- 172.21.15.60:0/621240249 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f45b8114670 msgr2=0x7f45b81a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:06.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:06.778+0000 7f45c0600700 1 --2- 172.21.15.60:0/621240249 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f45b8114670 0x7f45b81a4c10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f45b000efc0 tx=0x7f45b000c3a0 comp rx=0 tx=0).stop 2024-05-03T14:47:06.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:06.778+0000 7f45be600700 1 -- 172.21.15.60:0/621240249 reap_dead start 2024-05-03T14:47:06.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:06.778+0000 7f45c0600700 1 -- 172.21.15.60:0/621240249 shutdown_connections 2024-05-03T14:47:06.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:06.778+0000 7f45c0600700 1 -- 172.21.15.60:0/621240249 >> 172.21.15.60:0/621240249 conn(0x7f45b807a650 msgr2=0x7f45b8113430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:47:06.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:06.779+0000 7f45c0600700 1 -- 172.21.15.60:0/621240249 shutdown_connections 2024-05-03T14:47:06.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:06.779+0000 7f45c0600700 1 -- 172.21.15.60:0/621240249 wait complete. 2024-05-03T14:47:07.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:47:07 smithi060 ceph-mon[28873]: pgmap v410: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:47:07.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:47:07 smithi090 ceph-mon[33568]: pgmap v410: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:47:07.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:47:07 smithi149 ceph-mon[33332]: pgmap v410: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:47:08.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:08.694+0000 7f73c4c00700 1 -- 172.21.15.60:0/2065680422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73c0109ad0 msgr2=0x7f73c0109f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:08.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:08.694+0000 7f73c4c00700 1 --2- 172.21.15.60:0/2065680422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73c0109ad0 0x7f73c0109f50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f73ac009990 tx=0x7f73ac009ca0 comp rx=0 tx=0).stop 2024-05-03T14:47:08.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:08.694+0000 7f73c4c00700 1 -- 172.21.15.60:0/2065680422 shutdown_connections 2024-05-03T14:47:08.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:08.694+0000 7f73c4c00700 1 --2- 172.21.15.60:0/2065680422 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f73c010a490 0x7f73c01169a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:08.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:08.694+0000 7f73c4c00700 1 --2- 172.21.15.60:0/2065680422 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f73c010fb60 0x7f73c010ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:08.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:08.694+0000 7f73c4c00700 1 --2- 172.21.15.60:0/2065680422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73c0109ad0 0x7f73c0109f50 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:08.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:08.694+0000 7f73c4c00700 1 -- 172.21.15.60:0/2065680422 >> 172.21.15.60:0/2065680422 conn(0x7f73c0076c70 msgr2=0x7f73c01082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:47:08.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:08.694+0000 7f73c4c00700 1 -- 172.21.15.60:0/2065680422 shutdown_connections 2024-05-03T14:47:08.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:08.694+0000 7f73c4c00700 1 -- 172.21.15.60:0/2065680422 wait complete. 2024-05-03T14:47:08.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:08.695+0000 7f73c4c00700 1 Processor -- start 2024-05-03T14:47:08.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:08.695+0000 7f73c4c00700 1 -- start start 2024-05-03T14:47:08.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:08.695+0000 7f73c4c00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73c0109ad0 0x7f73c01a03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:08.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:08.696+0000 7f73c4c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f73c010a490 0x7f73c01a0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:08.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:08.696+0000 7f73c4c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f73c010fb60 0x7f73c01a4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:08.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:08.696+0000 7f73c4c00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f73c0119f90 con 0x7f73c0109ad0 2024-05-03T14:47:08.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:08.696+0000 7f73c4c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f73c0119e10 con 0x7f73c010fb60 2024-05-03T14:47:08.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:08.696+0000 7f73c4c00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f73c011a110 con 0x7f73c010a490 2024-05-03T14:47:08.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:08.696+0000 7f73bde00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73c0109ad0 0x7f73c01a03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:08.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:08.696+0000 7f73bde00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73c0109ad0 0x7f73c01a03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49080/0 (socket says 172.21.15.60:49080) 2024-05-03T14:47:08.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:08.696+0000 7f73bde00700 1 -- 172.21.15.60:0/3281451539 learned_addr learned my addr 172.21.15.60:0/3281451539 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:47:08.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:08.696+0000 7f73bd400700 1 --2- 172.21.15.60:0/3281451539 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f73c010a490 0x7f73c01a0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:08.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:08.696+0000 7f73be800700 1 --2- 172.21.15.60:0/3281451539 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f73c010fb60 0x7f73c01a4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:08.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:08.696+0000 7f73bd400700 1 -- 172.21.15.60:0/3281451539 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f73c010fb60 msgr2=0x7f73c01a4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:08.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:08.696+0000 7f73bd400700 1 --2- 172.21.15.60:0/3281451539 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f73c010fb60 0x7f73c01a4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:08.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:08.696+0000 7f73bd400700 1 -- 172.21.15.60:0/3281451539 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73c0109ad0 msgr2=0x7f73c01a03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:08.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:08.696+0000 7f73bd400700 1 --2- 172.21.15.60:0/3281451539 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73c0109ad0 0x7f73c01a03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:08.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:08.696+0000 7f73bd400700 1 -- 172.21.15.60:0/3281451539 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73c01a5380 con 0x7f73c010a490 2024-05-03T14:47:08.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:08.696+0000 7f73bde00700 1 --2- 172.21.15.60:0/3281451539 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73c0109ad0 0x7f73c01a03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:47:08.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:08.697+0000 7f73bd400700 1 --2- 172.21.15.60:0/3281451539 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f73c010a490 0x7f73c01a0900 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f73ac0098e0 tx=0x7f73ac005590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:08.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:08.697+0000 7f73b6a00700 1 -- 172.21.15.60:0/3281451539 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73ac027070 con 0x7f73c010a490 2024-05-03T14:47:08.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:08.697+0000 7f73c4c00700 1 -- 172.21.15.60:0/3281451539 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73c01a5610 con 0x7f73c010a490 2024-05-03T14:47:08.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:08.697+0000 7f73b6a00700 1 -- 172.21.15.60:0/3281451539 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f73ac00fe00 con 0x7f73c010a490 2024-05-03T14:47:08.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:08.697+0000 7f73b6a00700 1 -- 172.21.15.60:0/3281451539 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73ac0173f0 con 0x7f73c010a490 2024-05-03T14:47:08.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:08.698+0000 7f73c4c00700 1 -- 172.21.15.60:0/3281451539 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73c01acef0 con 0x7f73c010a490 2024-05-03T14:47:08.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:08.698+0000 7f73c4c00700 1 -- 172.21.15.60:0/3281451539 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f73c0063c30 con 0x7f73c010a490 2024-05-03T14:47:08.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:08.702+0000 7f73b6a00700 1 -- 172.21.15.60:0/3281451539 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f73ac02a070 con 0x7f73c010a490 2024-05-03T14:47:08.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:08.703+0000 7f73b6a00700 1 --2- 172.21.15.60:0/3281451539 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f73a40b2910 0x7f73a40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:08.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:08.704+0000 7f73b6a00700 1 -- 172.21.15.60:0/3281451539 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f73ac0e3610 con 0x7f73c010a490 2024-05-03T14:47:08.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:08.704+0000 7f73b6a00700 1 --2- 172.21.15.60:0/3281451539 >> [v2:172.21.15.149:6800/1359662077,v1:172.21.15.149:6801/1359662077] conn(0x7f73a40bdaf0 0x7f73a40bff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:08.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:08.704+0000 7f73b6a00700 1 -- 172.21.15.60:0/3281451539 --> [v2:172.21.15.149:6800/1359662077,v1:172.21.15.149:6801/1359662077] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f73a40c0660 con 0x7f73a40bdaf0 2024-05-03T14:47:08.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:08.704+0000 7f73b6a00700 1 -- 172.21.15.60:0/3281451539 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f73ac0e39c0 con 0x7f73c010a490 2024-05-03T14:47:08.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:08.704+0000 7f73bde00700 1 --2- 172.21.15.60:0/3281451539 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f73a40b2910 0x7f73a40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:08.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:08.704+0000 7f73be800700 1 --2- 172.21.15.60:0/3281451539 >> [v2:172.21.15.149:6800/1359662077,v1:172.21.15.149:6801/1359662077] conn(0x7f73a40bdaf0 0x7f73a40bff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:08.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:08.705+0000 7f73bde00700 1 --2- 172.21.15.60:0/3281451539 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f73a40b2910 0x7f73a40b4dd0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f73b0009c10 tx=0x7f73b000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:08.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:08.705+0000 7f73be800700 1 --2- 172.21.15.60:0/3281451539 >> [v2:172.21.15.149:6800/1359662077,v1:172.21.15.149:6801/1359662077] conn(0x7f73a40bdaf0 0x7f73a40bff90 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:08.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:08.706+0000 7f73b6a00700 1 -- 172.21.15.60:0/3281451539 <== osd.2 v2:172.21.15.149:6800/1359662077 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f73a40c0660 con 0x7f73a40bdaf0 2024-05-03T14:47:08.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:02.066+0000 7fd783600700 1 -- 172.21.15.60:0/304821138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd77c0756d0 msgr2=0x7fd77c075ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:08.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:02.066+0000 7fd783600700 1 --2- 172.21.15.60:0/304821138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd77c0756d0 0x7fd77c075ab0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd77000b0a0 tx=0x7fd77000b3b0 comp rx=0 tx=0).stop 2024-05-03T14:47:08.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:02.066+0000 7fd783600700 1 -- 172.21.15.60:0/304821138 shutdown_connections 2024-05-03T14:47:08.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:02.066+0000 7fd783600700 1 --2- 172.21.15.60:0/304821138 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd77c10fea0 0x7fd77c112290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:08.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:02.066+0000 7fd783600700 1 --2- 172.21.15.60:0/304821138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd77c0756d0 0x7fd77c075ab0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:08.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:02.066+0000 7fd783600700 1 --2- 172.21.15.60:0/304821138 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd77c075ff0 0x7fd77c10f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:08.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:02.066+0000 7fd783600700 1 -- 172.21.15.60:0/304821138 >> 172.21.15.60:0/304821138 conn(0x7fd77c07a650 msgr2=0x7fd77c07aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:47:08.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:02.066+0000 7fd783600700 1 -- 172.21.15.60:0/304821138 shutdown_connections 2024-05-03T14:47:08.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:02.066+0000 7fd783600700 1 -- 172.21.15.60:0/304821138 wait complete. 2024-05-03T14:47:08.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:02.067+0000 7fd783600700 1 Processor -- start 2024-05-03T14:47:08.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:02.067+0000 7fd783600700 1 -- start start 2024-05-03T14:47:08.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:02.068+0000 7fd783600700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd77c0756d0 0x7fd77c19bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:08.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:02.068+0000 7fd783600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd77c075ff0 0x7fd77c19c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:08.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:02.068+0000 7fd783600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd77c10fea0 0x7fd77c1a0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:08.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:02.068+0000 7fd783600700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd77c115b50 con 0x7fd77c0756d0 2024-05-03T14:47:08.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:02.068+0000 7fd783600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd77c1159d0 con 0x7fd77c075ff0 2024-05-03T14:47:08.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:02.068+0000 7fd783600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd77c115cd0 con 0x7fd77c10fea0 2024-05-03T14:47:08.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:02.068+0000 7fd780c00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd77c0756d0 0x7fd77c19bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:08.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:02.068+0000 7fd781600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd77c10fea0 0x7fd77c1a0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:08.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:02.068+0000 7fd780c00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd77c0756d0 0x7fd77c19bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49010/0 (socket says 172.21.15.60:49010) 2024-05-03T14:47:08.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:02.068+0000 7fd780c00700 1 -- 172.21.15.60:0/1362134853 learned_addr learned my addr 172.21.15.60:0/1362134853 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:47:08.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:02.068+0000 7fd77be00700 1 --2- 172.21.15.60:0/1362134853 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd77c075ff0 0x7fd77c19c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:08.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:02.068+0000 7fd780c00700 1 -- 172.21.15.60:0/1362134853 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd77c10fea0 msgr2=0x7fd77c1a0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:08.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:02.068+0000 7fd780c00700 1 --2- 172.21.15.60:0/1362134853 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd77c10fea0 0x7fd77c1a0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:08.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:02.068+0000 7fd780c00700 1 -- 172.21.15.60:0/1362134853 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd77c075ff0 msgr2=0x7fd77c19c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:08.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:02.068+0000 7fd780c00700 1 --2- 172.21.15.60:0/1362134853 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd77c075ff0 0x7fd77c19c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:08.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:02.068+0000 7fd780c00700 1 -- 172.21.15.60:0/1362134853 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd77c1a0f40 con 0x7fd77c0756d0 2024-05-03T14:47:08.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:02.068+0000 7fd781600700 1 --2- 172.21.15.60:0/1362134853 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd77c10fea0 0x7fd77c1a0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:47:08.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:02.069+0000 7fd780c00700 1 --2- 172.21.15.60:0/1362134853 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd77c0756d0 0x7fd77c19bf80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd7700062a0 tx=0x7fd770004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:08.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:02.069+0000 7fd779600700 1 -- 172.21.15.60:0/1362134853 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd770021070 con 0x7fd77c0756d0 2024-05-03T14:47:08.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:02.069+0000 7fd783600700 1 -- 172.21.15.60:0/1362134853 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd77c1a11d0 con 0x7fd77c0756d0 2024-05-03T14:47:08.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:02.069+0000 7fd779600700 1 -- 172.21.15.60:0/1362134853 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7fd770007b80 con 0x7fd77c0756d0 2024-05-03T14:47:08.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:02.069+0000 7fd779600700 1 -- 172.21.15.60:0/1362134853 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd770005280 con 0x7fd77c0756d0 2024-05-03T14:47:08.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:02.069+0000 7fd783600700 1 -- 172.21.15.60:0/1362134853 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd77c1a8ab0 con 0x7fd77c0756d0 2024-05-03T14:47:08.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:02.070+0000 7fd783600700 1 -- 172.21.15.60:0/1362134853 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd77c101070 con 0x7fd77c0756d0 2024-05-03T14:47:08.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:02.073+0000 7fd779600700 1 -- 172.21.15.60:0/1362134853 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fd770024070 con 0x7fd77c0756d0 2024-05-03T14:47:08.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:02.074+0000 7fd779600700 1 --2- 172.21.15.60:0/1362134853 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fd7640b2910 0x7fd7640b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:08.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:02.074+0000 7fd779600700 1 -- 172.21.15.60:0/1362134853 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7fd77000f020 con 0x7fd77c0756d0 2024-05-03T14:47:08.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:02.074+0000 7fd77be00700 1 --2- 172.21.15.60:0/1362134853 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fd7640b2910 0x7fd7640b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:08.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:02.075+0000 7fd779600700 1 --2- 172.21.15.60:0/1362134853 >> [v2:172.21.15.90:6808/2201391883,v1:172.21.15.90:6809/2201391883] conn(0x7fd7640bdaf0 0x7fd7640bff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:08.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:02.075+0000 7fd779600700 1 -- 172.21.15.60:0/1362134853 --> [v2:172.21.15.90:6808/2201391883,v1:172.21.15.90:6809/2201391883] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd770027e50 con 0x7fd7640bdaf0 2024-05-03T14:47:08.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:02.075+0000 7fd779600700 1 -- 172.21.15.60:0/1362134853 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fd7700e41f0 con 0x7fd77c0756d0 2024-05-03T14:47:08.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:02.075+0000 7fd781600700 1 --2- 172.21.15.60:0/1362134853 >> [v2:172.21.15.90:6808/2201391883,v1:172.21.15.90:6809/2201391883] conn(0x7fd7640bdaf0 0x7fd7640bff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:08.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:02.075+0000 7fd77be00700 1 --2- 172.21.15.60:0/1362134853 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fd7640b2910 0x7fd7640b4dd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd77c1119f0 tx=0x7fd768009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:08.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:02.076+0000 7fd781600700 1 --2- 172.21.15.60:0/1362134853 >> [v2:172.21.15.90:6808/2201391883,v1:172.21.15.90:6809/2201391883] conn(0x7fd7640bdaf0 0x7fd7640bff90 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:08.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:02.077+0000 7fd779600700 1 -- 172.21.15.60:0/1362134853 <== osd.4 v2:172.21.15.90:6808/2201391883 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd770027e50 con 0x7fd7640bdaf0 2024-05-03T14:47:08.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:02.095+0000 7fd783600700 1 -- 172.21.15.60:0/1362134853 --> [v2:172.21.15.90:6808/2201391883,v1:172.21.15.90:6809/2201391883] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd77c076df0 con 0x7fd7640bdaf0 2024-05-03T14:47:08.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:02.096+0000 7fd779600700 1 -- 172.21.15.60:0/1362134853 <== osd.4 v2:172.21.15.90:6808/2201391883 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd77c076df0 con 0x7fd7640bdaf0 2024-05-03T14:47:08.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:02.096+0000 7fd783600700 1 -- 172.21.15.60:0/1362134853 >> [v2:172.21.15.90:6808/2201391883,v1:172.21.15.90:6809/2201391883] conn(0x7fd7640bdaf0 msgr2=0x7fd7640bff90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:08.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:02.097+0000 7fd783600700 1 --2- 172.21.15.60:0/1362134853 >> [v2:172.21.15.90:6808/2201391883,v1:172.21.15.90:6809/2201391883] conn(0x7fd7640bdaf0 0x7fd7640bff90 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:08.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:02.097+0000 7fd783600700 1 -- 172.21.15.60:0/1362134853 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fd7640b2910 msgr2=0x7fd7640b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:08.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:02.097+0000 7fd783600700 1 --2- 172.21.15.60:0/1362134853 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fd7640b2910 0x7fd7640b4dd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd77c1119f0 tx=0x7fd768009260 comp rx=0 tx=0).stop 2024-05-03T14:47:08.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:02.097+0000 7fd783600700 1 -- 172.21.15.60:0/1362134853 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd77c0756d0 msgr2=0x7fd77c19bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:08.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:02.097+0000 7fd783600700 1 --2- 172.21.15.60:0/1362134853 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd77c0756d0 0x7fd77c19bf80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd7700062a0 tx=0x7fd770004d00 comp rx=0 tx=0).stop 2024-05-03T14:47:08.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:02.097+0000 7fd781600700 1 -- 172.21.15.60:0/1362134853 reap_dead start 2024-05-03T14:47:08.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:02.097+0000 7fd783600700 1 -- 172.21.15.60:0/1362134853 shutdown_connections 2024-05-03T14:47:08.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:02.097+0000 7fd783600700 1 -- 172.21.15.60:0/1362134853 >> 172.21.15.60:0/1362134853 conn(0x7fd77c07a650 msgr2=0x7fd77c100610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:47:08.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:02.097+0000 7fd783600700 1 -- 172.21.15.60:0/1362134853 shutdown_connections 2024-05-03T14:47:08.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:02.097+0000 7fd783600700 1 -- 172.21.15.60:0/1362134853 wait complete. 2024-05-03T14:47:08.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:08.724+0000 7f73c4c00700 1 -- 172.21.15.60:0/3281451539 --> [v2:172.21.15.149:6800/1359662077,v1:172.21.15.149:6801/1359662077] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f73c0064030 con 0x7f73a40bdaf0 2024-05-03T14:47:08.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:08.725+0000 7f73b6a00700 1 -- 172.21.15.60:0/3281451539 <== osd.2 v2:172.21.15.149:6800/1359662077 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f73c0064030 con 0x7f73a40bdaf0 2024-05-03T14:47:08.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:08.725+0000 7f73c4c00700 1 -- 172.21.15.60:0/3281451539 >> [v2:172.21.15.149:6800/1359662077,v1:172.21.15.149:6801/1359662077] conn(0x7f73a40bdaf0 msgr2=0x7f73a40bff90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:08.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:08.725+0000 7f73c4c00700 1 --2- 172.21.15.60:0/3281451539 >> [v2:172.21.15.149:6800/1359662077,v1:172.21.15.149:6801/1359662077] conn(0x7f73a40bdaf0 0x7f73a40bff90 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:08.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:08.726+0000 7f73c4c00700 1 -- 172.21.15.60:0/3281451539 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f73a40b2910 msgr2=0x7f73a40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:08.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:08.726+0000 7f73c4c00700 1 --2- 172.21.15.60:0/3281451539 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f73a40b2910 0x7f73a40b4dd0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f73b0009c10 tx=0x7f73b000b040 comp rx=0 tx=0).stop 2024-05-03T14:47:08.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:08.726+0000 7f73c4c00700 1 -- 172.21.15.60:0/3281451539 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f73c010a490 msgr2=0x7f73c01a0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:08.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:08.726+0000 7f73c4c00700 1 --2- 172.21.15.60:0/3281451539 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f73c010a490 0x7f73c01a0900 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f73ac0098e0 tx=0x7f73ac005590 comp rx=0 tx=0).stop 2024-05-03T14:47:08.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:08.726+0000 7f73be800700 1 -- 172.21.15.60:0/3281451539 reap_dead start 2024-05-03T14:47:08.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:08.726+0000 7f73c4c00700 1 -- 172.21.15.60:0/3281451539 shutdown_connections 2024-05-03T14:47:08.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:08.726+0000 7f73c4c00700 1 -- 172.21.15.60:0/3281451539 >> 172.21.15.60:0/3281451539 conn(0x7f73c0076c70 msgr2=0x7f73c0077230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:47:08.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:08.726+0000 7f73c4c00700 1 -- 172.21.15.60:0/3281451539 shutdown_connections 2024-05-03T14:47:08.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:08.726+0000 7f73c4c00700 1 -- 172.21.15.60:0/3281451539 wait complete. 2024-05-03T14:47:09.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:47:09 smithi060 ceph-mon[28873]: pgmap v411: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:47:09.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:47:09 smithi090 ceph-mon[33568]: pgmap v411: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:47:09.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:47:09 smithi149 ceph-mon[33332]: pgmap v411: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:47:10.971 INFO:teuthology.orchestra.run.smithi060.stdout:154618822739 2024-05-03T14:47:10.972 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-03T14:47:11.398 INFO:teuthology.orchestra.run.smithi060.stdout:133143986264 2024-05-03T14:47:11.398 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-03T14:47:11.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:47:11 smithi060 ceph-mon[28873]: pgmap v412: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:47:11.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:47:11 smithi090 ceph-mon[33568]: pgmap v412: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:47:11.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:47:11 smithi149 ceph-mon[33332]: pgmap v412: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:47:12.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:12.026+0000 7f261d400700 1 -- 172.21.15.60:0/385912430 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26181055d0 msgr2=0x7f2618105a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:12.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:12.026+0000 7f261d400700 1 --2- 172.21.15.60:0/385912430 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26181055d0 0x7f2618105a50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2604009990 tx=0x7f2604009ca0 comp rx=0 tx=0).stop 2024-05-03T14:47:12.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:12.026+0000 7f261d400700 1 -- 172.21.15.60:0/385912430 shutdown_connections 2024-05-03T14:47:12.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:12.026+0000 7f261d400700 1 --2- 172.21.15.60:0/385912430 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2618105f90 0x7f2618112450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:12.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:12.026+0000 7f261d400700 1 --2- 172.21.15.60:0/385912430 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f261810b680 0x7f261810ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:12.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:12.026+0000 7f261d400700 1 --2- 172.21.15.60:0/385912430 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26181055d0 0x7f2618105a50 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:12.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:12.026+0000 7f261d400700 1 -- 172.21.15.60:0/385912430 >> 172.21.15.60:0/385912430 conn(0x7f26181010e0 msgr2=0x7f2618103500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:47:12.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:12.026+0000 7f261d400700 1 -- 172.21.15.60:0/385912430 shutdown_connections 2024-05-03T14:47:12.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:12.026+0000 7f261d400700 1 -- 172.21.15.60:0/385912430 wait complete. 2024-05-03T14:47:12.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:12.027+0000 7f261d400700 1 Processor -- start 2024-05-03T14:47:12.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:12.027+0000 7f261d400700 1 -- start start 2024-05-03T14:47:12.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:12.028+0000 7f261d400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26181055d0 0x7f261819bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:12.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:12.028+0000 7f261d400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2618105f90 0x7f261819c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:12.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:12.028+0000 7f261d400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f261810b680 0x7f26181a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:12.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:12.028+0000 7f261d400700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f2618115a40 con 0x7f2618105f90 2024-05-03T14:47:12.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:12.028+0000 7f261d400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f26181158c0 con 0x7f261810b680 2024-05-03T14:47:12.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:12.028+0000 7f261d400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2618115bc0 con 0x7f26181055d0 2024-05-03T14:47:12.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:12.028+0000 7f2617400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2618105f90 0x7f261819c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:12.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:12.028+0000 7f2617e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26181055d0 0x7f261819bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:12.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:12.028+0000 7f2617e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26181055d0 0x7f261819bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.60:60462/0 (socket says 172.21.15.60:60462) 2024-05-03T14:47:12.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:12.028+0000 7f2617e00700 1 -- 172.21.15.60:0/37610414 learned_addr learned my addr 172.21.15.60:0/37610414 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:47:12.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:12.029+0000 7f261ca00700 1 --2- 172.21.15.60:0/37610414 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f261810b680 0x7f26181a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:12.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:12.029+0000 7f2617e00700 1 -- 172.21.15.60:0/37610414 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f261810b680 msgr2=0x7f26181a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:12.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:12.029+0000 7f2617e00700 1 --2- 172.21.15.60:0/37610414 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f261810b680 0x7f26181a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:12.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:12.029+0000 7f2617e00700 1 -- 172.21.15.60:0/37610414 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2618105f90 msgr2=0x7f261819c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:12.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:12.029+0000 7f2617e00700 1 --2- 172.21.15.60:0/37610414 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2618105f90 0x7f261819c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:12.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:12.029+0000 7f2617e00700 1 -- 172.21.15.60:0/37610414 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26181a0de0 con 0x7f26181055d0 2024-05-03T14:47:12.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:12.029+0000 7f2617e00700 1 --2- 172.21.15.60:0/37610414 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26181055d0 0x7f261819bdd0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f260800eb30 tx=0x7f260800ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:12.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:12.030+0000 7f2614c00700 1 -- 172.21.15.60:0/37610414 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260800cbd0 con 0x7f26181055d0 2024-05-03T14:47:12.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:12.030+0000 7f2614c00700 1 -- 172.21.15.60:0/37610414 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f2608004520 con 0x7f26181055d0 2024-05-03T14:47:12.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:12.030+0000 7f2614c00700 1 -- 172.21.15.60:0/37610414 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2608010430 con 0x7f26181055d0 2024-05-03T14:47:12.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:12.030+0000 7f261d400700 1 -- 172.21.15.60:0/37610414 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26181a10d0 con 0x7f26181055d0 2024-05-03T14:47:12.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:12.030+0000 7f261d400700 1 -- 172.21.15.60:0/37610414 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26181a8950 con 0x7f26181055d0 2024-05-03T14:47:12.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:12.031+0000 7f261d400700 1 -- 172.21.15.60:0/37610414 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2618103040 con 0x7f26181055d0 2024-05-03T14:47:12.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:12.034+0000 7f2614c00700 1 -- 172.21.15.60:0/37610414 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f2608003f70 con 0x7f26181055d0 2024-05-03T14:47:12.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:12.035+0000 7f2614c00700 1 --2- 172.21.15.60:0/37610414 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f25fc0b2990 0x7f25fc0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:12.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:12.035+0000 7f2614c00700 1 -- 172.21.15.60:0/37610414 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f26080e2cb0 con 0x7f26181055d0 2024-05-03T14:47:12.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:12.035+0000 7f2617400700 1 --2- 172.21.15.60:0/37610414 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f25fc0b2990 0x7f25fc0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:12.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:12.036+0000 7f2614c00700 1 --2- 172.21.15.60:0/37610414 >> [v2:172.21.15.149:6824/1063245126,v1:172.21.15.149:6825/1063245126] conn(0x7f25fc0bdb70 0x7f25fc0c0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:12.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:12.036+0000 7f2614c00700 1 -- 172.21.15.60:0/37610414 --> [v2:172.21.15.149:6824/1063245126,v1:172.21.15.149:6825/1063245126] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f25fc0c06e0 con 0x7f25fc0bdb70 2024-05-03T14:47:12.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:12.036+0000 7f2614c00700 1 -- 172.21.15.60:0/37610414 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f26080e3060 con 0x7f26181055d0 2024-05-03T14:47:12.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:12.036+0000 7f261ca00700 1 --2- 172.21.15.60:0/37610414 >> [v2:172.21.15.149:6824/1063245126,v1:172.21.15.149:6825/1063245126] conn(0x7f25fc0bdb70 0x7f25fc0c0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:12.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:12.036+0000 7f2617400700 1 --2- 172.21.15.60:0/37610414 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f25fc0b2990 0x7f25fc0b4e50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f261810abc0 tx=0x7f260400b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:12.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:12.037+0000 7f261ca00700 1 --2- 172.21.15.60:0/37610414 >> [v2:172.21.15.149:6824/1063245126,v1:172.21.15.149:6825/1063245126] conn(0x7f25fc0bdb70 0x7f25fc0c0010 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:12.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:12.038+0000 7f2614c00700 1 -- 172.21.15.60:0/37610414 <== osd.11 v2:172.21.15.149:6824/1063245126 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f25fc0c06e0 con 0x7f25fc0bdb70 2024-05-03T14:47:12.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:12.056+0000 7f261d400700 1 -- 172.21.15.60:0/37610414 --> [v2:172.21.15.149:6824/1063245126,v1:172.21.15.149:6825/1063245126] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2618106cf0 con 0x7f25fc0bdb70 2024-05-03T14:47:12.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:12.056+0000 7f2614c00700 1 -- 172.21.15.60:0/37610414 <== osd.11 v2:172.21.15.149:6824/1063245126 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2618106cf0 con 0x7f25fc0bdb70 2024-05-03T14:47:12.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:12.057+0000 7f261d400700 1 -- 172.21.15.60:0/37610414 >> [v2:172.21.15.149:6824/1063245126,v1:172.21.15.149:6825/1063245126] conn(0x7f25fc0bdb70 msgr2=0x7f25fc0c0010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:12.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:12.057+0000 7f261d400700 1 --2- 172.21.15.60:0/37610414 >> [v2:172.21.15.149:6824/1063245126,v1:172.21.15.149:6825/1063245126] conn(0x7f25fc0bdb70 0x7f25fc0c0010 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:12.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:12.057+0000 7f261d400700 1 -- 172.21.15.60:0/37610414 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f25fc0b2990 msgr2=0x7f25fc0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:12.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:12.057+0000 7f261d400700 1 --2- 172.21.15.60:0/37610414 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f25fc0b2990 0x7f25fc0b4e50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f261810abc0 tx=0x7f260400b260 comp rx=0 tx=0).stop 2024-05-03T14:47:12.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:12.057+0000 7f261d400700 1 -- 172.21.15.60:0/37610414 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26181055d0 msgr2=0x7f261819bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:12.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:12.057+0000 7f261d400700 1 --2- 172.21.15.60:0/37610414 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26181055d0 0x7f261819bdd0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f260800eb30 tx=0x7f260800ee40 comp rx=0 tx=0).stop 2024-05-03T14:47:12.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:12.057+0000 7f261ca00700 1 -- 172.21.15.60:0/37610414 reap_dead start 2024-05-03T14:47:12.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:12.057+0000 7f261d400700 1 -- 172.21.15.60:0/37610414 shutdown_connections 2024-05-03T14:47:12.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:12.057+0000 7f261d400700 1 -- 172.21.15.60:0/37610414 >> 172.21.15.60:0/37610414 conn(0x7f26181010e0 msgr2=0x7f26181025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:47:12.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:12.058+0000 7f261d400700 1 -- 172.21.15.60:0/37610414 shutdown_connections 2024-05-03T14:47:12.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:12.058+0000 7f261d400700 1 -- 172.21.15.60:0/37610414 wait complete. 2024-05-03T14:47:12.631 INFO:teuthology.orchestra.run.smithi060.stdout:107374182494 2024-05-03T14:47:12.632 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-03T14:47:13.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:47:13 smithi060 ceph-mon[28873]: pgmap v413: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T14:47:13.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:47:13 smithi090 ceph-mon[33568]: pgmap v413: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T14:47:13.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:47:13 smithi149 ceph-mon[33332]: pgmap v413: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T14:47:14.047 INFO:teuthology.orchestra.run.smithi060.stdout:77309411430 2024-05-03T14:47:14.047 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-03T14:47:14.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.152+0000 7fe3ef800700 1 -- 172.21.15.60:0/3758656128 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe3e8114350 msgr2=0x7fe3e8116740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:14.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.152+0000 7fe3ef800700 1 --2- 172.21.15.60:0/3758656128 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe3e8114350 0x7fe3e8116740 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe3e0009990 tx=0x7fe3e0009ca0 comp rx=0 tx=0).stop 2024-05-03T14:47:14.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.152+0000 7fe3ef800700 1 -- 172.21.15.60:0/3758656128 shutdown_connections 2024-05-03T14:47:14.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.152+0000 7fe3ef800700 1 --2- 172.21.15.60:0/3758656128 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe3e8114350 0x7fe3e8116740 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:14.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.152+0000 7fe3ef800700 1 --2- 172.21.15.60:0/3758656128 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe3e80779d0 0x7fe3e8077db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:14.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.152+0000 7fe3ef800700 1 --2- 172.21.15.60:0/3758656128 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3e80782f0 0x7fe3e8113e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:14.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.152+0000 7fe3ef800700 1 -- 172.21.15.60:0/3758656128 >> 172.21.15.60:0/3758656128 conn(0x7fe3e81011e0 msgr2=0x7fe3e8103600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:47:14.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.152+0000 7fe3ef800700 1 -- 172.21.15.60:0/3758656128 shutdown_connections 2024-05-03T14:47:14.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.152+0000 7fe3ef800700 1 -- 172.21.15.60:0/3758656128 wait complete. 2024-05-03T14:47:14.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.153+0000 7fe3ef800700 1 Processor -- start 2024-05-03T14:47:14.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.153+0000 7fe3ef800700 1 -- start start 2024-05-03T14:47:14.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.154+0000 7fe3ef800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe3e80779d0 0x7fe3e81a0340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:14.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.154+0000 7fe3ef800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3e80782f0 0x7fe3e81a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:14.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.154+0000 7fe3ef800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe3e8114350 0x7fe3e81a4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:14.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.154+0000 7fe3ef800700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e8119f70 con 0x7fe3e80782f0 2024-05-03T14:47:14.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.154+0000 7fe3ef800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e8119df0 con 0x7fe3e80779d0 2024-05-03T14:47:14.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.154+0000 7fe3ef800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e811a0f0 con 0x7fe3e8114350 2024-05-03T14:47:14.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.154+0000 7fe3e7e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3e80782f0 0x7fe3e81a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:14.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.154+0000 7fe3e7e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3e80782f0 0x7fe3e81a0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47272/0 (socket says 172.21.15.60:47272) 2024-05-03T14:47:14.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.154+0000 7fe3e7e00700 1 -- 172.21.15.60:0/3329699562 learned_addr learned my addr 172.21.15.60:0/3329699562 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:47:14.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.154+0000 7fe3ece00700 1 --2- 172.21.15.60:0/3329699562 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe3e80779d0 0x7fe3e81a0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:14.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.154+0000 7fe3ed800700 1 --2- 172.21.15.60:0/3329699562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe3e8114350 0x7fe3e81a4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:14.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.155+0000 7fe3e7e00700 1 -- 172.21.15.60:0/3329699562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe3e8114350 msgr2=0x7fe3e81a4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:14.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.155+0000 7fe3e7e00700 1 --2- 172.21.15.60:0/3329699562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe3e8114350 0x7fe3e81a4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:14.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.155+0000 7fe3e7e00700 1 -- 172.21.15.60:0/3329699562 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe3e80779d0 msgr2=0x7fe3e81a0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:14.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.155+0000 7fe3e7e00700 1 --2- 172.21.15.60:0/3329699562 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe3e80779d0 0x7fe3e81a0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:14.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.155+0000 7fe3e7e00700 1 -- 172.21.15.60:0/3329699562 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3e81a5300 con 0x7fe3e80782f0 2024-05-03T14:47:14.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.155+0000 7fe3ece00700 1 --2- 172.21.15.60:0/3329699562 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe3e80779d0 0x7fe3e81a0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:47:14.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.155+0000 7fe3e7e00700 1 --2- 172.21.15.60:0/3329699562 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3e80782f0 0x7fe3e81a0880 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe3d400e9a0 tx=0x7fe3d400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:14.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.155+0000 7fe3e5600700 1 -- 172.21.15.60:0/3329699562 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d400ca10 con 0x7fe3e80782f0 2024-05-03T14:47:14.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.156+0000 7fe3ef800700 1 -- 172.21.15.60:0/3329699562 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3e81a55f0 con 0x7fe3e80782f0 2024-05-03T14:47:14.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.156+0000 7fe3e5600700 1 -- 172.21.15.60:0/3329699562 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7fe3d400cbb0 con 0x7fe3e80782f0 2024-05-03T14:47:14.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.156+0000 7fe3ef800700 1 -- 172.21.15.60:0/3329699562 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3e81ace70 con 0x7fe3e80782f0 2024-05-03T14:47:14.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.156+0000 7fe3e5600700 1 -- 172.21.15.60:0/3329699562 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d4018740 con 0x7fe3e80782f0 2024-05-03T14:47:14.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.157+0000 7fe3ef800700 1 -- 172.21.15.60:0/3329699562 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe3e8116760 con 0x7fe3e80782f0 2024-05-03T14:47:14.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.159+0000 7fe3e5600700 1 -- 172.21.15.60:0/3329699562 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fe3d4010430 con 0x7fe3e80782f0 2024-05-03T14:47:14.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.161+0000 7fe3e5600700 1 --2- 172.21.15.60:0/3329699562 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fe3d00b2910 0x7fe3d00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:14.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.161+0000 7fe3e5600700 1 -- 172.21.15.60:0/3329699562 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7fe3d40e2d60 con 0x7fe3e80782f0 2024-05-03T14:47:14.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.161+0000 7fe3ece00700 1 --2- 172.21.15.60:0/3329699562 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fe3d00b2910 0x7fe3d00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:14.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.161+0000 7fe3e5600700 1 --2- 172.21.15.60:0/3329699562 >> [v2:172.21.15.60:6826/2156130999,v1:172.21.15.60:6827/2156130999] conn(0x7fe3d00bdaf0 0x7fe3d00bff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:14.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.161+0000 7fe3e5600700 1 -- 172.21.15.60:0/3329699562 --> [v2:172.21.15.60:6826/2156130999,v1:172.21.15.60:6827/2156130999] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe3d40090b0 con 0x7fe3d00bdaf0 2024-05-03T14:47:14.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.161+0000 7fe3e5600700 1 -- 172.21.15.60:0/3329699562 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fe3d40e3110 con 0x7fe3e80782f0 2024-05-03T14:47:14.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.162+0000 7fe3ed800700 1 --2- 172.21.15.60:0/3329699562 >> [v2:172.21.15.60:6826/2156130999,v1:172.21.15.60:6827/2156130999] conn(0x7fe3d00bdaf0 0x7fe3d00bff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:14.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.162+0000 7fe3ece00700 1 --2- 172.21.15.60:0/3329699562 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fe3d00b2910 0x7fe3d00b4dd0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc009480 tx=0x7fe3dc00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:14.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.163+0000 7fe3ed800700 1 --2- 172.21.15.60:0/3329699562 >> [v2:172.21.15.60:6826/2156130999,v1:172.21.15.60:6827/2156130999] conn(0x7fe3d00bdaf0 0x7fe3d00bff90 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:14.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.164+0000 7fe3e5600700 1 -- 172.21.15.60:0/3329699562 <== osd.9 v2:172.21.15.60:6826/2156130999 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe3d40090b0 con 0x7fe3d00bdaf0 2024-05-03T14:47:14.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.182+0000 7fe3ef800700 1 -- 172.21.15.60:0/3329699562 --> [v2:172.21.15.60:6826/2156130999,v1:172.21.15.60:6827/2156130999] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe3e8079110 con 0x7fe3d00bdaf0 2024-05-03T14:47:14.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.183+0000 7fe3e5600700 1 -- 172.21.15.60:0/3329699562 <== osd.9 v2:172.21.15.60:6826/2156130999 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe3e8079110 con 0x7fe3d00bdaf0 2024-05-03T14:47:14.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.183+0000 7fe3ef800700 1 -- 172.21.15.60:0/3329699562 >> [v2:172.21.15.60:6826/2156130999,v1:172.21.15.60:6827/2156130999] conn(0x7fe3d00bdaf0 msgr2=0x7fe3d00bff90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:14.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.183+0000 7fe3ef800700 1 --2- 172.21.15.60:0/3329699562 >> [v2:172.21.15.60:6826/2156130999,v1:172.21.15.60:6827/2156130999] conn(0x7fe3d00bdaf0 0x7fe3d00bff90 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:14.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.184+0000 7fe3ef800700 1 -- 172.21.15.60:0/3329699562 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fe3d00b2910 msgr2=0x7fe3d00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:14.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.184+0000 7fe3ef800700 1 --2- 172.21.15.60:0/3329699562 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fe3d00b2910 0x7fe3d00b4dd0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc009480 tx=0x7fe3dc00b040 comp rx=0 tx=0).stop 2024-05-03T14:47:14.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.184+0000 7fe3ef800700 1 -- 172.21.15.60:0/3329699562 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3e80782f0 msgr2=0x7fe3e81a0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:14.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.184+0000 7fe3ef800700 1 --2- 172.21.15.60:0/3329699562 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3e80782f0 0x7fe3e81a0880 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe3d400e9a0 tx=0x7fe3d400ecb0 comp rx=0 tx=0).stop 2024-05-03T14:47:14.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.184+0000 7fe3ed800700 1 -- 172.21.15.60:0/3329699562 reap_dead start 2024-05-03T14:47:14.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.184+0000 7fe3ef800700 1 -- 172.21.15.60:0/3329699562 shutdown_connections 2024-05-03T14:47:14.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.184+0000 7fe3ef800700 1 -- 172.21.15.60:0/3329699562 >> 172.21.15.60:0/3329699562 conn(0x7fe3e81011e0 msgr2=0x7fe3e81035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:47:14.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.184+0000 7fe3ef800700 1 -- 172.21.15.60:0/3329699562 shutdown_connections 2024-05-03T14:47:14.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.184+0000 7fe3ef800700 1 -- 172.21.15.60:0/3329699562 wait complete. 2024-05-03T14:47:14.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:14.587+0000 7f531b800700 1 -- 172.21.15.60:0/1268114631 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5314075ff0 msgr2=0x7f5314114130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:14.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:14.587+0000 7f531b800700 1 --2- 172.21.15.60:0/1268114631 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5314075ff0 0x7f5314114130 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5300009990 tx=0x7f5300009ca0 comp rx=0 tx=0).stop 2024-05-03T14:47:14.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:14.587+0000 7f531b800700 1 -- 172.21.15.60:0/1268114631 shutdown_connections 2024-05-03T14:47:14.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:14.587+0000 7f531b800700 1 --2- 172.21.15.60:0/1268114631 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5314114670 0x7f5314116a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:14.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:14.587+0000 7f531b800700 1 --2- 172.21.15.60:0/1268114631 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53140756d0 0x7f5314075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:14.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:14.587+0000 7f531b800700 1 --2- 172.21.15.60:0/1268114631 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5314075ff0 0x7f5314114130 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:14.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:14.587+0000 7f531b800700 1 -- 172.21.15.60:0/1268114631 >> 172.21.15.60:0/1268114631 conn(0x7f531407a600 msgr2=0x7f531407aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:47:14.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:14.587+0000 7f531b800700 1 -- 172.21.15.60:0/1268114631 shutdown_connections 2024-05-03T14:47:14.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:14.587+0000 7f531b800700 1 -- 172.21.15.60:0/1268114631 wait complete. 2024-05-03T14:47:14.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:14.588+0000 7f531b800700 1 Processor -- start 2024-05-03T14:47:14.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:14.588+0000 7f531b800700 1 -- start start 2024-05-03T14:47:14.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:14.589+0000 7f531b800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53140756d0 0x7f53141a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:14.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:14.589+0000 7f531b800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5314075ff0 0x7f53141a0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:14.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:14.589+0000 7f531b800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5314114670 0x7f53141a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:14.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:14.589+0000 7f531b800700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5314119f50 con 0x7f53140756d0 2024-05-03T14:47:14.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:14.589+0000 7f531b800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5314119dd0 con 0x7f5314075ff0 2024-05-03T14:47:14.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:14.589+0000 7f531b800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f531411a0d0 con 0x7f5314114670 2024-05-03T14:47:14.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:14.589+0000 7f5318e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53140756d0 0x7f53141a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:14.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:14.589+0000 7f5318e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53140756d0 0x7f53141a03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47296/0 (socket says 172.21.15.60:47296) 2024-05-03T14:47:14.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:14.589+0000 7f5318e00700 1 -- 172.21.15.60:0/1104067478 learned_addr learned my addr 172.21.15.60:0/1104067478 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:47:14.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:14.589+0000 7f530be00700 1 --2- 172.21.15.60:0/1104067478 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5314075ff0 0x7f53141a0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:14.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:14.589+0000 7f5319800700 1 --2- 172.21.15.60:0/1104067478 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5314114670 0x7f53141a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:14.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:14.590+0000 7f530be00700 1 -- 172.21.15.60:0/1104067478 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5314114670 msgr2=0x7f53141a4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:14.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:14.590+0000 7f530be00700 1 --2- 172.21.15.60:0/1104067478 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5314114670 0x7f53141a4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:14.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:14.590+0000 7f530be00700 1 -- 172.21.15.60:0/1104067478 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53140756d0 msgr2=0x7f53141a03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:14.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:14.590+0000 7f530be00700 1 --2- 172.21.15.60:0/1104067478 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53140756d0 0x7f53141a03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:14.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:14.590+0000 7f530be00700 1 -- 172.21.15.60:0/1104067478 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53141a53f0 con 0x7f5314075ff0 2024-05-03T14:47:14.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:14.590+0000 7f5319800700 1 --2- 172.21.15.60:0/1104067478 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5314114670 0x7f53141a4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:47:14.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:14.590+0000 7f5318e00700 1 --2- 172.21.15.60:0/1104067478 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53140756d0 0x7f53141a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:47:14.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:14.590+0000 7f530be00700 1 --2- 172.21.15.60:0/1104067478 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5314075ff0 0x7f53141a0920 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f530000b2e0 tx=0x7f53000047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:14.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:14.590+0000 7f5312000700 1 -- 172.21.15.60:0/1104067478 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5300026070 con 0x7f5314075ff0 2024-05-03T14:47:14.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:14.590+0000 7f531b800700 1 -- 172.21.15.60:0/1104067478 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53141a5680 con 0x7f5314075ff0 2024-05-03T14:47:14.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:14.590+0000 7f5312000700 1 -- 172.21.15.60:0/1104067478 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f530000fb10 con 0x7f5314075ff0 2024-05-03T14:47:14.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:14.590+0000 7f5312000700 1 -- 172.21.15.60:0/1104067478 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5300017650 con 0x7f5314075ff0 2024-05-03T14:47:14.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:14.590+0000 7f531b800700 1 -- 172.21.15.60:0/1104067478 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53141acf60 con 0x7f5314075ff0 2024-05-03T14:47:14.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:14.591+0000 7f531b800700 1 -- 172.21.15.60:0/1104067478 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5314113ec0 con 0x7f5314075ff0 2024-05-03T14:47:14.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:14.594+0000 7f5312000700 1 -- 172.21.15.60:0/1104067478 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f5300029070 con 0x7f5314075ff0 2024-05-03T14:47:14.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:14.595+0000 7f5312000700 1 --2- 172.21.15.60:0/1104067478 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f52f80b2910 0x7f52f80b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:14.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:14.595+0000 7f5312000700 1 -- 172.21.15.60:0/1104067478 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f53000e3ff0 con 0x7f5314075ff0 2024-05-03T14:47:14.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:14.595+0000 7f5318e00700 1 --2- 172.21.15.60:0/1104067478 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f52f80b2910 0x7f52f80b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:14.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:14.596+0000 7f5312000700 1 --2- 172.21.15.60:0/1104067478 >> [v2:172.21.15.90:6824/2492593590,v1:172.21.15.90:6825/2492593590] conn(0x7f52f80bdae0 0x7f52f80bff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:14.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:14.596+0000 7f5312000700 1 -- 172.21.15.60:0/1104067478 --> [v2:172.21.15.90:6824/2492593590,v1:172.21.15.90:6825/2492593590] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5300021990 con 0x7f52f80bdae0 2024-05-03T14:47:14.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:14.596+0000 7f5312000700 1 -- 172.21.15.60:0/1104067478 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f53000e4400 con 0x7f5314075ff0 2024-05-03T14:47:14.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:14.596+0000 7f5319800700 1 --2- 172.21.15.60:0/1104067478 >> [v2:172.21.15.90:6824/2492593590,v1:172.21.15.90:6825/2492593590] conn(0x7f52f80bdae0 0x7f52f80bff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:14.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:14.596+0000 7f5318e00700 1 --2- 172.21.15.60:0/1104067478 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f52f80b2910 0x7f52f80b4dd0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5304009c10 tx=0x7f530400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:14.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:14.597+0000 7f5319800700 1 --2- 172.21.15.60:0/1104067478 >> [v2:172.21.15.90:6824/2492593590,v1:172.21.15.90:6825/2492593590] conn(0x7f52f80bdae0 0x7f52f80bff80 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:14.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:14.598+0000 7f5312000700 1 -- 172.21.15.60:0/1104067478 <== osd.10 v2:172.21.15.90:6824/2492593590 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5300021990 con 0x7f52f80bdae0 2024-05-03T14:47:14.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:14.616+0000 7f531b800700 1 -- 172.21.15.60:0/1104067478 --> [v2:172.21.15.90:6824/2492593590,v1:172.21.15.90:6825/2492593590] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5314076dc0 con 0x7f52f80bdae0 2024-05-03T14:47:14.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:14.617+0000 7f5312000700 1 -- 172.21.15.60:0/1104067478 <== osd.10 v2:172.21.15.90:6824/2492593590 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5314076dc0 con 0x7f52f80bdae0 2024-05-03T14:47:14.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:14.617+0000 7f531b800700 1 -- 172.21.15.60:0/1104067478 >> [v2:172.21.15.90:6824/2492593590,v1:172.21.15.90:6825/2492593590] conn(0x7f52f80bdae0 msgr2=0x7f52f80bff80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:14.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:14.617+0000 7f531b800700 1 --2- 172.21.15.60:0/1104067478 >> [v2:172.21.15.90:6824/2492593590,v1:172.21.15.90:6825/2492593590] conn(0x7f52f80bdae0 0x7f52f80bff80 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:14.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:14.618+0000 7f531b800700 1 -- 172.21.15.60:0/1104067478 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f52f80b2910 msgr2=0x7f52f80b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:14.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:14.618+0000 7f531b800700 1 --2- 172.21.15.60:0/1104067478 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f52f80b2910 0x7f52f80b4dd0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5304009c10 tx=0x7f530400b040 comp rx=0 tx=0).stop 2024-05-03T14:47:14.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:14.618+0000 7f531b800700 1 -- 172.21.15.60:0/1104067478 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5314075ff0 msgr2=0x7f53141a0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:14.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:14.618+0000 7f531b800700 1 --2- 172.21.15.60:0/1104067478 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5314075ff0 0x7f53141a0920 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f530000b2e0 tx=0x7f53000047b0 comp rx=0 tx=0).stop 2024-05-03T14:47:14.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:14.618+0000 7f5319800700 1 -- 172.21.15.60:0/1104067478 reap_dead start 2024-05-03T14:47:14.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:14.618+0000 7f531b800700 1 -- 172.21.15.60:0/1104067478 shutdown_connections 2024-05-03T14:47:14.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:14.618+0000 7f531b800700 1 -- 172.21.15.60:0/1104067478 >> 172.21.15.60:0/1104067478 conn(0x7f531407a600 msgr2=0x7f53141134a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:47:14.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:14.618+0000 7f531b800700 1 -- 172.21.15.60:0/1104067478 shutdown_connections 2024-05-03T14:47:14.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:14.618+0000 7f531b800700 1 -- 172.21.15.60:0/1104067478 wait complete. 2024-05-03T14:47:15.734 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:47:15 smithi060 ceph-mon[28873]: pgmap v414: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:47:15.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:15.683+0000 7fe740e00700 1 -- 172.21.15.60:0/1954157915 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe73c101f30 msgr2=0x7fe73c102310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:15.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:15.683+0000 7fe740e00700 1 --2- 172.21.15.60:0/1954157915 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe73c101f30 0x7fe73c102310 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe72c009990 tx=0x7fe72c009ca0 comp rx=0 tx=0).stop 2024-05-03T14:47:15.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:15.683+0000 7fe740e00700 1 -- 172.21.15.60:0/1954157915 shutdown_connections 2024-05-03T14:47:15.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:15.683+0000 7fe740e00700 1 --2- 172.21.15.60:0/1954157915 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe73c1087a0 0x7fe73c110350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:15.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:15.683+0000 7fe740e00700 1 --2- 172.21.15.60:0/1954157915 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe73c101f30 0x7fe73c102310 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:15.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:15.683+0000 7fe740e00700 1 --2- 172.21.15.60:0/1954157915 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe73c1028e0 0x7fe73c1080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:15.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:15.683+0000 7fe740e00700 1 -- 172.21.15.60:0/1954157915 >> 172.21.15.60:0/1954157915 conn(0x7fe73c077650 msgr2=0x7fe73c077a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:47:15.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:15.683+0000 7fe740e00700 1 -- 172.21.15.60:0/1954157915 shutdown_connections 2024-05-03T14:47:15.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:15.683+0000 7fe740e00700 1 -- 172.21.15.60:0/1954157915 wait complete. 2024-05-03T14:47:15.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:15.684+0000 7fe740e00700 1 Processor -- start 2024-05-03T14:47:15.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:15.685+0000 7fe740e00700 1 -- start start 2024-05-03T14:47:15.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:15.685+0000 7fe740e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe73c101f30 0x7fe73c19bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:15.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:15.685+0000 7fe740e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe73c1028e0 0x7fe73c19c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:15.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:15.685+0000 7fe740e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe73c1087a0 0x7fe73c1a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:15.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:15.685+0000 7fe740e00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe73c1138d0 con 0x7fe73c1087a0 2024-05-03T14:47:15.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:15.685+0000 7fe740e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe73c113750 con 0x7fe73c101f30 2024-05-03T14:47:15.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:15.685+0000 7fe740e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe73c113a50 con 0x7fe73c1028e0 2024-05-03T14:47:15.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:15.685+0000 7fe73b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe73c101f30 0x7fe73c19bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:15.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:15.685+0000 7fe73be00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe73c1087a0 0x7fe73c1a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:15.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:15.685+0000 7fe73aa00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe73c1028e0 0x7fe73c19c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:15.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:15.685+0000 7fe73b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe73c101f30 0x7fe73c19bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.60:41406/0 (socket says 172.21.15.60:41406) 2024-05-03T14:47:15.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:15.685+0000 7fe73b400700 1 -- 172.21.15.60:0/3017375260 learned_addr learned my addr 172.21.15.60:0/3017375260 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:47:15.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:15.686+0000 7fe73be00700 1 -- 172.21.15.60:0/3017375260 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe73c1028e0 msgr2=0x7fe73c19c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:15.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:15.686+0000 7fe73be00700 1 --2- 172.21.15.60:0/3017375260 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe73c1028e0 0x7fe73c19c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:15.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:15.686+0000 7fe73be00700 1 -- 172.21.15.60:0/3017375260 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe73c101f30 msgr2=0x7fe73c19bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:15.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:15.686+0000 7fe73be00700 1 --2- 172.21.15.60:0/3017375260 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe73c101f30 0x7fe73c19bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:15.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:15.686+0000 7fe73be00700 1 -- 172.21.15.60:0/3017375260 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe73c1a0ed0 con 0x7fe73c1087a0 2024-05-03T14:47:15.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:15.686+0000 7fe73be00700 1 --2- 172.21.15.60:0/3017375260 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe73c1087a0 0x7fe73c1a0790 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe73400efb0 tx=0x7fe73400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:15.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:15.686+0000 7fe733e00700 1 -- 172.21.15.60:0/3017375260 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe734003a00 con 0x7fe73c1087a0 2024-05-03T14:47:15.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:15.687+0000 7fe740e00700 1 -- 172.21.15.60:0/3017375260 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe73c1a11c0 con 0x7fe73c1087a0 2024-05-03T14:47:15.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:15.687+0000 7fe733e00700 1 -- 172.21.15.60:0/3017375260 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7fe734003ba0 con 0x7fe73c1087a0 2024-05-03T14:47:15.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:15.687+0000 7fe733e00700 1 -- 172.21.15.60:0/3017375260 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe734007500 con 0x7fe73c1087a0 2024-05-03T14:47:15.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:15.687+0000 7fe740e00700 1 -- 172.21.15.60:0/3017375260 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe73c1a8a40 con 0x7fe73c1087a0 2024-05-03T14:47:15.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:15.688+0000 7fe740e00700 1 -- 172.21.15.60:0/3017375260 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe73c1040c0 con 0x7fe73c1087a0 2024-05-03T14:47:15.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:15.690+0000 7fe733e00700 1 -- 172.21.15.60:0/3017375260 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fe734012050 con 0x7fe73c1087a0 2024-05-03T14:47:15.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:15.692+0000 7fe733e00700 1 --2- 172.21.15.60:0/3017375260 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fe7200b2990 0x7fe7200b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:15.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:15.692+0000 7fe733e00700 1 -- 172.21.15.60:0/3017375260 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7fe7340e2500 con 0x7fe73c1087a0 2024-05-03T14:47:15.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:15.692+0000 7fe73b400700 1 --2- 172.21.15.60:0/3017375260 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fe7200b2990 0x7fe7200b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:15.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:15.693+0000 7fe733e00700 1 --2- 172.21.15.60:0/3017375260 >> [v2:172.21.15.90:6816/1686298661,v1:172.21.15.90:6817/1686298661] conn(0x7fe7200bdb70 0x7fe7200c0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:15.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:15.693+0000 7fe733e00700 1 -- 172.21.15.60:0/3017375260 --> [v2:172.21.15.90:6816/1686298661,v1:172.21.15.90:6817/1686298661] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe73401fe50 con 0x7fe7200bdb70 2024-05-03T14:47:15.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:15.693+0000 7fe733e00700 1 -- 172.21.15.60:0/3017375260 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fe7340e28b0 con 0x7fe73c1087a0 2024-05-03T14:47:15.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:15.693+0000 7fe73aa00700 1 --2- 172.21.15.60:0/3017375260 >> [v2:172.21.15.90:6816/1686298661,v1:172.21.15.90:6817/1686298661] conn(0x7fe7200bdb70 0x7fe7200c0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:15.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:15.693+0000 7fe73b400700 1 --2- 172.21.15.60:0/3017375260 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fe7200b2990 0x7fe7200b4e50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe72c009750 tx=0x7fe72c00b280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:15.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:15.694+0000 7fe73aa00700 1 --2- 172.21.15.60:0/3017375260 >> [v2:172.21.15.90:6816/1686298661,v1:172.21.15.90:6817/1686298661] conn(0x7fe7200bdb70 0x7fe7200c0010 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:15.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:15.694+0000 7fe733e00700 1 -- 172.21.15.60:0/3017375260 <== osd.7 v2:172.21.15.90:6816/1686298661 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe73401fe50 con 0x7fe7200bdb70 2024-05-03T14:47:15.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:15.713+0000 7fe740e00700 1 -- 172.21.15.60:0/3017375260 --> [v2:172.21.15.90:6816/1686298661,v1:172.21.15.90:6817/1686298661] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe73c06ad10 con 0x7fe7200bdb70 2024-05-03T14:47:15.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:15.713+0000 7fe733e00700 1 -- 172.21.15.60:0/3017375260 <== osd.7 v2:172.21.15.90:6816/1686298661 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe73c06ad10 con 0x7fe7200bdb70 2024-05-03T14:47:15.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:15.714+0000 7fe740e00700 1 -- 172.21.15.60:0/3017375260 >> [v2:172.21.15.90:6816/1686298661,v1:172.21.15.90:6817/1686298661] conn(0x7fe7200bdb70 msgr2=0x7fe7200c0010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:15.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:15.714+0000 7fe740e00700 1 --2- 172.21.15.60:0/3017375260 >> [v2:172.21.15.90:6816/1686298661,v1:172.21.15.90:6817/1686298661] conn(0x7fe7200bdb70 0x7fe7200c0010 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:15.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:15.714+0000 7fe740e00700 1 -- 172.21.15.60:0/3017375260 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fe7200b2990 msgr2=0x7fe7200b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:15.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:15.714+0000 7fe740e00700 1 --2- 172.21.15.60:0/3017375260 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fe7200b2990 0x7fe7200b4e50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe72c009750 tx=0x7fe72c00b280 comp rx=0 tx=0).stop 2024-05-03T14:47:15.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:15.714+0000 7fe740e00700 1 -- 172.21.15.60:0/3017375260 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe73c1087a0 msgr2=0x7fe73c1a0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:15.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:15.714+0000 7fe740e00700 1 --2- 172.21.15.60:0/3017375260 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe73c1087a0 0x7fe73c1a0790 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe73400efb0 tx=0x7fe73400c3a0 comp rx=0 tx=0).stop 2024-05-03T14:47:15.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:15.714+0000 7fe73be00700 1 -- 172.21.15.60:0/3017375260 reap_dead start 2024-05-03T14:47:15.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:15.714+0000 7fe740e00700 1 -- 172.21.15.60:0/3017375260 shutdown_connections 2024-05-03T14:47:15.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:15.714+0000 7fe740e00700 1 -- 172.21.15.60:0/3017375260 >> 172.21.15.60:0/3017375260 conn(0x7fe73c077650 msgr2=0x7fe73c0ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:47:15.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:15.714+0000 7fe740e00700 1 -- 172.21.15.60:0/3017375260 shutdown_connections 2024-05-03T14:47:15.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:15.715+0000 7fe740e00700 1 -- 172.21.15.60:0/3017375260 wait complete. 2024-05-03T14:47:15.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:47:15 smithi090 ceph-mon[33568]: pgmap v414: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:47:15.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:47:15 smithi149 ceph-mon[33332]: pgmap v414: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:47:16.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.661+0000 7f3432800700 1 -- 172.21.15.60:0/3472427860 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f342c109510 msgr2=0x7f342c1098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:16.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.661+0000 7f3432800700 1 --2- 172.21.15.60:0/3472427860 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f342c109510 0x7f342c1098f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f341c009990 tx=0x7f341c009ca0 comp rx=0 tx=0).stop 2024-05-03T14:47:16.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.661+0000 7f3432800700 1 -- 172.21.15.60:0/3472427860 shutdown_connections 2024-05-03T14:47:16.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.661+0000 7f3432800700 1 --2- 172.21.15.60:0/3472427860 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f342c06bb40 0x7f342c110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:16.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.661+0000 7f3432800700 1 --2- 172.21.15.60:0/3472427860 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f342c109510 0x7f342c1098f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:16.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.661+0000 7f3432800700 1 --2- 172.21.15.60:0/3472427860 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f342c06b180 0x7f342c06b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:16.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.661+0000 7f3432800700 1 -- 172.21.15.60:0/3472427860 >> 172.21.15.60:0/3472427860 conn(0x7f342c079350 msgr2=0x7f342c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:47:16.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.661+0000 7f3432800700 1 -- 172.21.15.60:0/3472427860 shutdown_connections 2024-05-03T14:47:16.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.661+0000 7f3432800700 1 -- 172.21.15.60:0/3472427860 wait complete. 2024-05-03T14:47:16.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.662+0000 7f3432800700 1 Processor -- start 2024-05-03T14:47:16.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.663+0000 7f3432800700 1 -- start start 2024-05-03T14:47:16.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.663+0000 7f3432800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f342c06b180 0x7f342c19bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:16.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.663+0000 7f3432800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f342c06bb40 0x7f342c19c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:16.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.663+0000 7f3432800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f342c109510 0x7f342c1a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:16.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.663+0000 7f3432800700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f342c113900 con 0x7f342c06bb40 2024-05-03T14:47:16.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.663+0000 7f3432800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f342c113780 con 0x7f342c109510 2024-05-03T14:47:16.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.663+0000 7f3432800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f342c113a80 con 0x7f342c06b180 2024-05-03T14:47:16.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.664+0000 7f342b400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f342c06b180 0x7f342c19bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:16.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.664+0000 7f3422a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f342c06bb40 0x7f342c19c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:16.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.664+0000 7f342b400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f342c06b180 0x7f342c19bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.60:60506/0 (socket says 172.21.15.60:60506) 2024-05-03T14:47:16.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.664+0000 7f342be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f342c109510 0x7f342c1a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:16.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.664+0000 7f342b400700 1 -- 172.21.15.60:0/4004250797 learned_addr learned my addr 172.21.15.60:0/4004250797 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:47:16.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.664+0000 7f342b400700 1 -- 172.21.15.60:0/4004250797 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f342c109510 msgr2=0x7f342c1a08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:16.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.664+0000 7f342b400700 1 --2- 172.21.15.60:0/4004250797 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f342c109510 0x7f342c1a08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:16.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.664+0000 7f342b400700 1 -- 172.21.15.60:0/4004250797 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f342c06bb40 msgr2=0x7f342c19c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:16.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.664+0000 7f342b400700 1 --2- 172.21.15.60:0/4004250797 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f342c06bb40 0x7f342c19c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:16.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.664+0000 7f342b400700 1 -- 172.21.15.60:0/4004250797 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f342c1a0ff0 con 0x7f342c06b180 2024-05-03T14:47:16.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.664+0000 7f342b400700 1 --2- 172.21.15.60:0/4004250797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f342c06b180 0x7f342c19bfe0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f341c000c00 tx=0x7f341c005590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:16.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.665+0000 7f3428c00700 1 -- 172.21.15.60:0/4004250797 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341c027070 con 0x7f342c06b180 2024-05-03T14:47:16.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.665+0000 7f3432800700 1 -- 172.21.15.60:0/4004250797 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f342c1a1280 con 0x7f342c06b180 2024-05-03T14:47:16.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.665+0000 7f3428c00700 1 -- 172.21.15.60:0/4004250797 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f341c00fdc0 con 0x7f342c06b180 2024-05-03T14:47:16.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.665+0000 7f3428c00700 1 -- 172.21.15.60:0/4004250797 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341c0173f0 con 0x7f342c06b180 2024-05-03T14:47:16.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.665+0000 7f3432800700 1 -- 172.21.15.60:0/4004250797 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f342c1a8b60 con 0x7f342c06b180 2024-05-03T14:47:16.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.666+0000 7f3432800700 1 -- 172.21.15.60:0/4004250797 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f342c10f060 con 0x7f342c06b180 2024-05-03T14:47:16.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.669+0000 7f3428c00700 1 -- 172.21.15.60:0/4004250797 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f341c02a070 con 0x7f342c06b180 2024-05-03T14:47:16.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.670+0000 7f3428c00700 1 --2- 172.21.15.60:0/4004250797 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f34100b2910 0x7f34100b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:16.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.670+0000 7f3428c00700 1 -- 172.21.15.60:0/4004250797 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f341c0e3600 con 0x7f342c06b180 2024-05-03T14:47:16.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.671+0000 7f3422a00700 1 --2- 172.21.15.60:0/4004250797 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f34100b2910 0x7f34100b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:16.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.671+0000 7f3428c00700 1 --2- 172.21.15.60:0/4004250797 >> [v2:172.21.15.149:6816/2608348248,v1:172.21.15.149:6817/2608348248] conn(0x7f34100bdaf0 0x7f34100bff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:16.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.671+0000 7f3428c00700 1 -- 172.21.15.60:0/4004250797 --> [v2:172.21.15.149:6816/2608348248,v1:172.21.15.149:6817/2608348248] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f341c01ae50 con 0x7f34100bdaf0 2024-05-03T14:47:16.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.671+0000 7f3428c00700 1 -- 172.21.15.60:0/4004250797 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f341c0e39b0 con 0x7f342c06b180 2024-05-03T14:47:16.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.671+0000 7f342be00700 1 --2- 172.21.15.60:0/4004250797 >> [v2:172.21.15.149:6816/2608348248,v1:172.21.15.149:6817/2608348248] conn(0x7f34100bdaf0 0x7f34100bff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:16.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.671+0000 7f3422a00700 1 --2- 172.21.15.60:0/4004250797 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f34100b2910 0x7f34100b4dd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f342c1043e0 tx=0x7f3414009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:16.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.672+0000 7f342be00700 1 --2- 172.21.15.60:0/4004250797 >> [v2:172.21.15.149:6816/2608348248,v1:172.21.15.149:6817/2608348248] conn(0x7f34100bdaf0 0x7f34100bff90 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:16.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.673+0000 7f3428c00700 1 -- 172.21.15.60:0/4004250797 <== osd.8 v2:172.21.15.149:6816/2608348248 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f341c01ae50 con 0x7f34100bdaf0 2024-05-03T14:47:16.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.692+0000 7f3432800700 1 -- 172.21.15.60:0/4004250797 --> [v2:172.21.15.149:6816/2608348248,v1:172.21.15.149:6817/2608348248] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f342c104bd0 con 0x7f34100bdaf0 2024-05-03T14:47:16.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.692+0000 7f3428c00700 1 -- 172.21.15.60:0/4004250797 <== osd.8 v2:172.21.15.149:6816/2608348248 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f342c104bd0 con 0x7f34100bdaf0 2024-05-03T14:47:16.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.693+0000 7f3432800700 1 -- 172.21.15.60:0/4004250797 >> [v2:172.21.15.149:6816/2608348248,v1:172.21.15.149:6817/2608348248] conn(0x7f34100bdaf0 msgr2=0x7f34100bff90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:16.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.693+0000 7f3432800700 1 --2- 172.21.15.60:0/4004250797 >> [v2:172.21.15.149:6816/2608348248,v1:172.21.15.149:6817/2608348248] conn(0x7f34100bdaf0 0x7f34100bff90 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:16.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.693+0000 7f3432800700 1 -- 172.21.15.60:0/4004250797 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f34100b2910 msgr2=0x7f34100b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:16.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.693+0000 7f3432800700 1 --2- 172.21.15.60:0/4004250797 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f34100b2910 0x7f34100b4dd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f342c1043e0 tx=0x7f3414009260 comp rx=0 tx=0).stop 2024-05-03T14:47:16.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.693+0000 7f3432800700 1 -- 172.21.15.60:0/4004250797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f342c06b180 msgr2=0x7f342c19bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:16.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.693+0000 7f3432800700 1 --2- 172.21.15.60:0/4004250797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f342c06b180 0x7f342c19bfe0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f341c000c00 tx=0x7f341c005590 comp rx=0 tx=0).stop 2024-05-03T14:47:16.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.693+0000 7f342be00700 1 -- 172.21.15.60:0/4004250797 reap_dead start 2024-05-03T14:47:16.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.693+0000 7f3432800700 1 -- 172.21.15.60:0/4004250797 shutdown_connections 2024-05-03T14:47:16.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.693+0000 7f3432800700 1 -- 172.21.15.60:0/4004250797 >> 172.21.15.60:0/4004250797 conn(0x7f342c079350 msgr2=0x7f342c10e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:47:16.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.694+0000 7f3432800700 1 -- 172.21.15.60:0/4004250797 shutdown_connections 2024-05-03T14:47:16.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:13.694+0000 7f3432800700 1 -- 172.21.15.60:0/4004250797 wait complete. 2024-05-03T14:47:17.417 INFO:teuthology.orchestra.run.smithi060.stdout:227633266756 2024-05-03T14:47:17.418 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-05-03T14:47:17.614 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:47:17 smithi060 ceph-mon[28873]: pgmap v415: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:47:17.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:47:17 smithi090 ceph-mon[33568]: pgmap v415: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:47:17.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:47:17 smithi149 ceph-mon[33332]: pgmap v415: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:47:18.091 INFO:teuthology.orchestra.run.smithi060.stdout:253403070526 2024-05-03T14:47:18.091 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-05-03T14:47:18.894 INFO:teuthology.orchestra.run.smithi060.stdout:300647710770 2024-05-03T14:47:18.895 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-05-03T14:47:19.218 INFO:teuthology.orchestra.run.smithi060.stdout:274877906999 2024-05-03T14:47:19.219 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-05-03T14:47:19.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:47:19 smithi060 ceph-mon[28873]: pgmap v416: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:47:19.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:47:19 smithi090 ceph-mon[33568]: pgmap v416: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:47:19.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:47:19 smithi149 ceph-mon[33332]: pgmap v416: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:47:20.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:19.497+0000 7fa0bda00700 1 -- 172.21.15.60:0/2134435020 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa0b8102230 msgr2=0x7fa0b8109280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:20.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:19.497+0000 7fa0bda00700 1 --2- 172.21.15.60:0/2134435020 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa0b8102230 0x7fa0b8109280 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4009920 tx=0x7fa0a4009c30 comp rx=0 tx=0).stop 2024-05-03T14:47:20.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:19.498+0000 7fa0bda00700 1 -- 172.21.15.60:0/2134435020 shutdown_connections 2024-05-03T14:47:20.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:19.498+0000 7fa0bda00700 1 --2- 172.21.15.60:0/2134435020 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0b8109960 0x7fa0b8110390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:20.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:19.498+0000 7fa0bda00700 1 --2- 172.21.15.60:0/2134435020 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0b8101880 0x7fa0b8101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:20.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:19.498+0000 7fa0bda00700 1 --2- 172.21.15.60:0/2134435020 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa0b8102230 0x7fa0b8109280 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:20.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:19.498+0000 7fa0bda00700 1 -- 172.21.15.60:0/2134435020 >> 172.21.15.60:0/2134435020 conn(0x7fa0b80785a0 msgr2=0x7fa0b80ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:47:20.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:19.498+0000 7fa0bda00700 1 -- 172.21.15.60:0/2134435020 shutdown_connections 2024-05-03T14:47:20.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:19.498+0000 7fa0bda00700 1 -- 172.21.15.60:0/2134435020 wait complete. 2024-05-03T14:47:20.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:19.499+0000 7fa0bda00700 1 Processor -- start 2024-05-03T14:47:20.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:19.499+0000 7fa0bda00700 1 -- start start 2024-05-03T14:47:20.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:19.499+0000 7fa0bda00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa0b8101880 0x7fa0b8199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:20.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:19.500+0000 7fa0bda00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0b8102230 0x7fa0b819a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:20.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:19.500+0000 7fa0bda00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0b8109960 0x7fa0b819e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:20.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:19.500+0000 7fa0bda00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b8113910 con 0x7fa0b8102230 2024-05-03T14:47:20.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:19.500+0000 7fa0bda00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b8113790 con 0x7fa0b8109960 2024-05-03T14:47:20.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:19.500+0000 7fa0bda00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b8113a90 con 0x7fa0b8101880 2024-05-03T14:47:20.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:19.500+0000 7fa0b6000700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0b8102230 0x7fa0b819a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:20.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:19.500+0000 7fa0b6000700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0b8102230 0x7fa0b819a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47334/0 (socket says 172.21.15.60:47334) 2024-05-03T14:47:20.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:19.500+0000 7fa0b6000700 1 -- 172.21.15.60:0/886884298 learned_addr learned my addr 172.21.15.60:0/886884298 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:47:20.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:19.500+0000 7fa0b6a00700 1 --2- 172.21.15.60:0/886884298 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa0b8101880 0x7fa0b8199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:20.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:19.500+0000 7fa0b7400700 1 --2- 172.21.15.60:0/886884298 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0b8109960 0x7fa0b819e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:20.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:19.501+0000 7fa0b6000700 1 -- 172.21.15.60:0/886884298 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa0b8101880 msgr2=0x7fa0b8199e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:20.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:19.501+0000 7fa0b6000700 1 --2- 172.21.15.60:0/886884298 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa0b8101880 0x7fa0b8199e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:20.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:19.501+0000 7fa0b6000700 1 -- 172.21.15.60:0/886884298 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0b8109960 msgr2=0x7fa0b819e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:20.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:19.501+0000 7fa0b6000700 1 --2- 172.21.15.60:0/886884298 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0b8109960 0x7fa0b819e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:20.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:19.501+0000 7fa0b6000700 1 -- 172.21.15.60:0/886884298 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0b819ee50 con 0x7fa0b8102230 2024-05-03T14:47:20.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:19.501+0000 7fa0b6a00700 1 --2- 172.21.15.60:0/886884298 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa0b8101880 0x7fa0b8199e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:47:20.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:19.501+0000 7fa0b6000700 1 --2- 172.21.15.60:0/886884298 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0b8102230 0x7fa0b819a380 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4009840 tx=0x7fa0a4010f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:20.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:19.501+0000 7fa0af400700 1 -- 172.21.15.60:0/886884298 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a400b5e0 con 0x7fa0b8102230 2024-05-03T14:47:20.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:19.501+0000 7fa0bda00700 1 -- 172.21.15.60:0/886884298 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0b819f0e0 con 0x7fa0b8102230 2024-05-03T14:47:20.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:19.501+0000 7fa0af400700 1 -- 172.21.15.60:0/886884298 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7fa0a400f460 con 0x7fa0b8102230 2024-05-03T14:47:20.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:19.501+0000 7fa0af400700 1 -- 172.21.15.60:0/886884298 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a4003ae0 con 0x7fa0b8102230 2024-05-03T14:47:20.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:19.502+0000 7fa0bda00700 1 -- 172.21.15.60:0/886884298 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0b81a69c0 con 0x7fa0b8102230 2024-05-03T14:47:20.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:19.503+0000 7fa0bda00700 1 -- 172.21.15.60:0/886884298 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa0b8063c30 con 0x7fa0b8102230 2024-05-03T14:47:20.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:19.505+0000 7fa0af400700 1 -- 172.21.15.60:0/886884298 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fa0a400f610 con 0x7fa0b8102230 2024-05-03T14:47:20.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:19.507+0000 7fa0af400700 1 --2- 172.21.15.60:0/886884298 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fa09c0b2a10 0x7fa09c0b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:20.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:19.507+0000 7fa0af400700 1 -- 172.21.15.60:0/886884298 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7fa0a40e3aa0 con 0x7fa0b8102230 2024-05-03T14:47:20.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:19.507+0000 7fa0af400700 1 --2- 172.21.15.60:0/886884298 >> [v2:172.21.15.90:6800/720063242,v1:172.21.15.90:6801/720063242] conn(0x7fa09c0bdac0 0x7fa09c0bff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:20.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:19.507+0000 7fa0b6a00700 1 --2- 172.21.15.60:0/886884298 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fa09c0b2a10 0x7fa09c0b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:20.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:19.507+0000 7fa0af400700 1 -- 172.21.15.60:0/886884298 --> [v2:172.21.15.90:6800/720063242,v1:172.21.15.90:6801/720063242] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa0a4010ba0 con 0x7fa09c0bdac0 2024-05-03T14:47:20.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:19.507+0000 7fa0af400700 1 -- 172.21.15.60:0/886884298 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fa0a40e3e90 con 0x7fa0b8102230 2024-05-03T14:47:20.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:19.508+0000 7fa0b7400700 1 --2- 172.21.15.60:0/886884298 >> [v2:172.21.15.90:6800/720063242,v1:172.21.15.90:6801/720063242] conn(0x7fa09c0bdac0 0x7fa09c0bff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:20.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:19.508+0000 7fa0b6a00700 1 --2- 172.21.15.60:0/886884298 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fa09c0b2a10 0x7fa09c0b4ed0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8011490 tx=0x7fa0a8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:20.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:19.509+0000 7fa0b7400700 1 --2- 172.21.15.60:0/886884298 >> [v2:172.21.15.90:6800/720063242,v1:172.21.15.90:6801/720063242] conn(0x7fa09c0bdac0 0x7fa09c0bff60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:20.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:19.509+0000 7fa0af400700 1 -- 172.21.15.60:0/886884298 <== osd.1 v2:172.21.15.90:6800/720063242 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa0a4010ba0 con 0x7fa09c0bdac0 2024-05-03T14:47:20.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:19.528+0000 7fa0bda00700 1 -- 172.21.15.60:0/886884298 --> [v2:172.21.15.90:6800/720063242,v1:172.21.15.90:6801/720063242] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa0b80640b0 con 0x7fa09c0bdac0 2024-05-03T14:47:20.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:19.529+0000 7fa0af400700 1 -- 172.21.15.60:0/886884298 <== osd.1 v2:172.21.15.90:6800/720063242 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa0b80640b0 con 0x7fa09c0bdac0 2024-05-03T14:47:20.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:19.529+0000 7fa0bda00700 1 -- 172.21.15.60:0/886884298 >> [v2:172.21.15.90:6800/720063242,v1:172.21.15.90:6801/720063242] conn(0x7fa09c0bdac0 msgr2=0x7fa09c0bff60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:20.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:19.529+0000 7fa0bda00700 1 --2- 172.21.15.60:0/886884298 >> [v2:172.21.15.90:6800/720063242,v1:172.21.15.90:6801/720063242] conn(0x7fa09c0bdac0 0x7fa09c0bff60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:20.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:19.529+0000 7fa0bda00700 1 -- 172.21.15.60:0/886884298 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fa09c0b2a10 msgr2=0x7fa09c0b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:20.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:19.529+0000 7fa0bda00700 1 --2- 172.21.15.60:0/886884298 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fa09c0b2a10 0x7fa09c0b4ed0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8011490 tx=0x7fa0a8009260 comp rx=0 tx=0).stop 2024-05-03T14:47:20.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:19.529+0000 7fa0bda00700 1 -- 172.21.15.60:0/886884298 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0b8102230 msgr2=0x7fa0b819a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:20.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:19.529+0000 7fa0bda00700 1 --2- 172.21.15.60:0/886884298 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0b8102230 0x7fa0b819a380 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4009840 tx=0x7fa0a4010f80 comp rx=0 tx=0).stop 2024-05-03T14:47:20.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:19.530+0000 7fa0b7400700 1 -- 172.21.15.60:0/886884298 reap_dead start 2024-05-03T14:47:20.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:19.530+0000 7fa0bda00700 1 -- 172.21.15.60:0/886884298 shutdown_connections 2024-05-03T14:47:20.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:19.530+0000 7fa0bda00700 1 -- 172.21.15.60:0/886884298 >> 172.21.15.60:0/886884298 conn(0x7fa0b80785a0 msgr2=0x7fa0b8100220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:47:20.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:19.530+0000 7fa0bda00700 1 -- 172.21.15.60:0/886884298 shutdown_connections 2024-05-03T14:47:20.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:19.530+0000 7fa0bda00700 1 -- 172.21.15.60:0/886884298 wait complete. 2024-05-03T14:47:20.229 INFO:teuthology.orchestra.run.smithi060.stdout:201863462985 2024-05-03T14:47:20.229 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-05-03T14:47:21.683 INFO:teuthology.orchestra.run.smithi060.stdout:55834574955 2024-05-03T14:47:21.683 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-03T14:47:21.754 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:47:21 smithi060 ceph-mon[28873]: pgmap v417: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:47:21.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.151+0000 7ff8ec000700 1 -- 172.21.15.60:0/539527503 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff8e410a490 msgr2=0x7ff8e41169a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:21.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.151+0000 7ff8ec000700 1 --2- 172.21.15.60:0/539527503 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff8e410a490 0x7ff8e41169a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc00b0a0 tx=0x7ff8dc00b3b0 comp rx=0 tx=0).stop 2024-05-03T14:47:21.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.151+0000 7ff8ec000700 1 -- 172.21.15.60:0/539527503 shutdown_connections 2024-05-03T14:47:21.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.151+0000 7ff8ec000700 1 --2- 172.21.15.60:0/539527503 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff8e410a490 0x7ff8e41169a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:21.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.151+0000 7ff8ec000700 1 --2- 172.21.15.60:0/539527503 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff8e410fb60 0x7ff8e410ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:21.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.151+0000 7ff8ec000700 1 --2- 172.21.15.60:0/539527503 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8e4109ad0 0x7ff8e4109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:21.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.151+0000 7ff8ec000700 1 -- 172.21.15.60:0/539527503 >> 172.21.15.60:0/539527503 conn(0x7ff8e4076c70 msgr2=0x7ff8e41082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:47:21.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.151+0000 7ff8ec000700 1 -- 172.21.15.60:0/539527503 shutdown_connections 2024-05-03T14:47:21.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.151+0000 7ff8ec000700 1 -- 172.21.15.60:0/539527503 wait complete. 2024-05-03T14:47:21.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.153+0000 7ff8ec000700 1 Processor -- start 2024-05-03T14:47:21.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.153+0000 7ff8ec000700 1 -- start start 2024-05-03T14:47:21.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.154+0000 7ff8ec000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff8e4109ad0 0x7ff8e41a0340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:21.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.154+0000 7ff8ec000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8e410a490 0x7ff8e41a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:21.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.154+0000 7ff8ec000700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff8e410fb60 0x7ff8e41a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:21.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.154+0000 7ff8ec000700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e4119f90 con 0x7ff8e410fb60 2024-05-03T14:47:21.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.154+0000 7ff8ec000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e4119e10 con 0x7ff8e410a490 2024-05-03T14:47:21.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.154+0000 7ff8ec000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e411a110 con 0x7ff8e4109ad0 2024-05-03T14:47:21.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.154+0000 7ff8e9600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff8e4109ad0 0x7ff8e41a0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:21.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.154+0000 7ff8e8c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8e410a490 0x7ff8e41a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:21.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.154+0000 7ff8e9600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff8e4109ad0 0x7ff8e41a0340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.60:60590/0 (socket says 172.21.15.60:60590) 2024-05-03T14:47:21.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.154+0000 7ff8e9600700 1 -- 172.21.15.60:0/485730990 learned_addr learned my addr 172.21.15.60:0/485730990 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:47:21.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.154+0000 7ff8ea000700 1 --2- 172.21.15.60:0/485730990 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff8e410fb60 0x7ff8e41a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:21.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.154+0000 7ff8e9600700 1 -- 172.21.15.60:0/485730990 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8e410a490 msgr2=0x7ff8e41a0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:21.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.154+0000 7ff8e9600700 1 --2- 172.21.15.60:0/485730990 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8e410a490 0x7ff8e41a0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:21.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.154+0000 7ff8e9600700 1 -- 172.21.15.60:0/485730990 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff8e410fb60 msgr2=0x7ff8e41a4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:21.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.154+0000 7ff8e9600700 1 --2- 172.21.15.60:0/485730990 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff8e410fb60 0x7ff8e41a4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:21.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.154+0000 7ff8e9600700 1 -- 172.21.15.60:0/485730990 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8e41a5350 con 0x7ff8e4109ad0 2024-05-03T14:47:21.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.155+0000 7ff8e9600700 1 --2- 172.21.15.60:0/485730990 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff8e4109ad0 0x7ff8e41a0340 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff8d800e9a0 tx=0x7ff8d800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:21.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.155+0000 7ff8e2000700 1 -- 172.21.15.60:0/485730990 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d800ca10 con 0x7ff8e4109ad0 2024-05-03T14:47:21.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.155+0000 7ff8e2000700 1 -- 172.21.15.60:0/485730990 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7ff8d800cbb0 con 0x7ff8e4109ad0 2024-05-03T14:47:21.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.155+0000 7ff8e2000700 1 -- 172.21.15.60:0/485730990 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d80187a0 con 0x7ff8e4109ad0 2024-05-03T14:47:21.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.155+0000 7ff8ec000700 1 -- 172.21.15.60:0/485730990 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8e41a5640 con 0x7ff8e4109ad0 2024-05-03T14:47:21.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.155+0000 7ff8ec000700 1 -- 172.21.15.60:0/485730990 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8e41acec0 con 0x7ff8e4109ad0 2024-05-03T14:47:21.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.156+0000 7ff8ec000700 1 -- 172.21.15.60:0/485730990 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff8e4077af0 con 0x7ff8e4109ad0 2024-05-03T14:47:21.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.159+0000 7ff8e2000700 1 -- 172.21.15.60:0/485730990 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7ff8d8010430 con 0x7ff8e4109ad0 2024-05-03T14:47:21.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.160+0000 7ff8e2000700 1 --2- 172.21.15.60:0/485730990 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7ff8cc0b2910 0x7ff8cc0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:21.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.161+0000 7ff8e2000700 1 -- 172.21.15.60:0/485730990 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7ff8d80e2d90 con 0x7ff8e4109ad0 2024-05-03T14:47:21.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.161+0000 7ff8e8c00700 1 --2- 172.21.15.60:0/485730990 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7ff8cc0b2910 0x7ff8cc0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:21.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.161+0000 7ff8e2000700 1 --2- 172.21.15.60:0/485730990 >> [v2:172.21.15.60:6818/3738521002,v1:172.21.15.60:6819/3738521002] conn(0x7ff8cc0bdaf0 0x7ff8cc0bff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:21.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.161+0000 7ff8e2000700 1 -- 172.21.15.60:0/485730990 --> [v2:172.21.15.60:6818/3738521002,v1:172.21.15.60:6819/3738521002] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff8d8016ec0 con 0x7ff8cc0bdaf0 2024-05-03T14:47:21.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.161+0000 7ff8e2000700 1 -- 172.21.15.60:0/485730990 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7ff8d80e3140 con 0x7ff8e4109ad0 2024-05-03T14:47:21.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.161+0000 7ff8ea000700 1 --2- 172.21.15.60:0/485730990 >> [v2:172.21.15.60:6818/3738521002,v1:172.21.15.60:6819/3738521002] conn(0x7ff8cc0bdaf0 0x7ff8cc0bff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:21.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.162+0000 7ff8e8c00700 1 --2- 172.21.15.60:0/485730990 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7ff8cc0b2910 0x7ff8cc0b4dd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4078a00 tx=0x7ff8d0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:21.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.162+0000 7ff8ea000700 1 --2- 172.21.15.60:0/485730990 >> [v2:172.21.15.60:6818/3738521002,v1:172.21.15.60:6819/3738521002] conn(0x7ff8cc0bdaf0 0x7ff8cc0bff90 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:21.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.163+0000 7ff8e2000700 1 -- 172.21.15.60:0/485730990 <== osd.6 v2:172.21.15.60:6818/3738521002 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff8d8016ec0 con 0x7ff8cc0bdaf0 2024-05-03T14:47:21.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.182+0000 7ff8ec000700 1 -- 172.21.15.60:0/485730990 --> [v2:172.21.15.60:6818/3738521002,v1:172.21.15.60:6819/3738521002] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff8e410b1d0 con 0x7ff8cc0bdaf0 2024-05-03T14:47:21.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.183+0000 7ff8e2000700 1 -- 172.21.15.60:0/485730990 <== osd.6 v2:172.21.15.60:6818/3738521002 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff8e410b1d0 con 0x7ff8cc0bdaf0 2024-05-03T14:47:21.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.184+0000 7ff8ec000700 1 -- 172.21.15.60:0/485730990 >> [v2:172.21.15.60:6818/3738521002,v1:172.21.15.60:6819/3738521002] conn(0x7ff8cc0bdaf0 msgr2=0x7ff8cc0bff90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:21.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.184+0000 7ff8ec000700 1 --2- 172.21.15.60:0/485730990 >> [v2:172.21.15.60:6818/3738521002,v1:172.21.15.60:6819/3738521002] conn(0x7ff8cc0bdaf0 0x7ff8cc0bff90 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:21.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.184+0000 7ff8ec000700 1 -- 172.21.15.60:0/485730990 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7ff8cc0b2910 msgr2=0x7ff8cc0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:21.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.184+0000 7ff8ec000700 1 --2- 172.21.15.60:0/485730990 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7ff8cc0b2910 0x7ff8cc0b4dd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4078a00 tx=0x7ff8d0009260 comp rx=0 tx=0).stop 2024-05-03T14:47:21.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.184+0000 7ff8ec000700 1 -- 172.21.15.60:0/485730990 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff8e4109ad0 msgr2=0x7ff8e41a0340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:21.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.184+0000 7ff8ec000700 1 --2- 172.21.15.60:0/485730990 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff8e4109ad0 0x7ff8e41a0340 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff8d800e9a0 tx=0x7ff8d800ecb0 comp rx=0 tx=0).stop 2024-05-03T14:47:21.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.184+0000 7ff8ea000700 1 -- 172.21.15.60:0/485730990 reap_dead start 2024-05-03T14:47:21.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.184+0000 7ff8ec000700 1 -- 172.21.15.60:0/485730990 shutdown_connections 2024-05-03T14:47:21.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.184+0000 7ff8ec000700 1 -- 172.21.15.60:0/485730990 >> 172.21.15.60:0/485730990 conn(0x7ff8e4076c70 msgr2=0x7ff8e4077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:47:21.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.184+0000 7ff8ec000700 1 -- 172.21.15.60:0/485730990 shutdown_connections 2024-05-03T14:47:21.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.184+0000 7ff8ec000700 1 -- 172.21.15.60:0/485730990 wait complete. 2024-05-03T14:47:21.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:47:21 smithi090 ceph-mon[33568]: pgmap v417: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:47:21.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:47:21 smithi149 ceph-mon[33332]: pgmap v417: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:47:22.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.345+0000 7ff84ee00700 1 -- 172.21.15.60:0/2820608835 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff848103eb0 msgr2=0x7ff848110390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:22.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.345+0000 7ff84ee00700 1 --2- 172.21.15.60:0/2820608835 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff848103eb0 0x7ff848110390 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff840009f90 tx=0x7ff84000b330 comp rx=0 tx=0).stop 2024-05-03T14:47:22.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.345+0000 7ff84ee00700 1 -- 172.21.15.60:0/2820608835 shutdown_connections 2024-05-03T14:47:22.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.345+0000 7ff84ee00700 1 --2- 172.21.15.60:0/2820608835 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff848103eb0 0x7ff848110390 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:22.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.345+0000 7ff84ee00700 1 --2- 172.21.15.60:0/2820608835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff848109600 0x7ff8481099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:22.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.345+0000 7ff84ee00700 1 --2- 172.21.15.60:0/2820608835 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff8481034f0 0x7ff848103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:22.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.345+0000 7ff84ee00700 1 -- 172.21.15.60:0/2820608835 >> 172.21.15.60:0/2820608835 conn(0x7ff84807a650 msgr2=0x7ff84807aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:47:22.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.345+0000 7ff84ee00700 1 -- 172.21.15.60:0/2820608835 shutdown_connections 2024-05-03T14:47:22.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.345+0000 7ff84ee00700 1 -- 172.21.15.60:0/2820608835 wait complete. 2024-05-03T14:47:22.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.346+0000 7ff84ee00700 1 Processor -- start 2024-05-03T14:47:22.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.346+0000 7ff84ee00700 1 -- start start 2024-05-03T14:47:22.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.347+0000 7ff84ee00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff8481034f0 0x7ff84819bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:22.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.347+0000 7ff84ee00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff848103eb0 0x7ff84819c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:22.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.347+0000 7ff84ee00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff848109600 0x7ff8481a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:22.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.347+0000 7ff84ee00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff848113900 con 0x7ff848109600 2024-05-03T14:47:22.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.347+0000 7ff84ee00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff848113780 con 0x7ff848103eb0 2024-05-03T14:47:22.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.347+0000 7ff84ee00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff848113a80 con 0x7ff8481034f0 2024-05-03T14:47:22.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.347+0000 7ff84ce00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff848109600 0x7ff8481a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:22.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.347+0000 7ff84ce00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff848109600 0x7ff8481a07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47366/0 (socket says 172.21.15.60:47366) 2024-05-03T14:47:22.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.347+0000 7ff84ce00700 1 -- 172.21.15.60:0/85200533 learned_addr learned my addr 172.21.15.60:0/85200533 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:47:22.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.347+0000 7ff847e00700 1 --2- 172.21.15.60:0/85200533 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff8481034f0 0x7ff84819bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:22.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.347+0000 7ff83f400700 1 --2- 172.21.15.60:0/85200533 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff848103eb0 0x7ff84819c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:22.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.348+0000 7ff84ce00700 1 -- 172.21.15.60:0/85200533 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff8481034f0 msgr2=0x7ff84819bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:22.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.348+0000 7ff84ce00700 1 --2- 172.21.15.60:0/85200533 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff8481034f0 0x7ff84819bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:22.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.348+0000 7ff84ce00700 1 -- 172.21.15.60:0/85200533 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff848103eb0 msgr2=0x7ff84819c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:22.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.348+0000 7ff84ce00700 1 --2- 172.21.15.60:0/85200533 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff848103eb0 0x7ff84819c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:22.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.348+0000 7ff84ce00700 1 -- 172.21.15.60:0/85200533 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8481a0f20 con 0x7ff848109600 2024-05-03T14:47:22.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.348+0000 7ff84ce00700 1 --2- 172.21.15.60:0/85200533 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff848109600 0x7ff8481a07e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff84000d010 tx=0x7ff84000bca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:22.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.348+0000 7ff845600700 1 -- 172.21.15.60:0/85200533 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff840021070 con 0x7ff848109600 2024-05-03T14:47:22.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.348+0000 7ff845600700 1 -- 172.21.15.60:0/85200533 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7ff84001d450 con 0x7ff848109600 2024-05-03T14:47:22.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.348+0000 7ff845600700 1 -- 172.21.15.60:0/85200533 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8400052a0 con 0x7ff848109600 2024-05-03T14:47:22.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.348+0000 7ff84ee00700 1 -- 172.21.15.60:0/85200533 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8481a11b0 con 0x7ff848109600 2024-05-03T14:47:22.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.349+0000 7ff84ee00700 1 -- 172.21.15.60:0/85200533 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8481a8a90 con 0x7ff848109600 2024-05-03T14:47:22.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.350+0000 7ff84ee00700 1 -- 172.21.15.60:0/85200533 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff848102700 con 0x7ff848109600 2024-05-03T14:47:22.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.353+0000 7ff845600700 1 -- 172.21.15.60:0/85200533 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7ff840024070 con 0x7ff848109600 2024-05-03T14:47:22.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.354+0000 7ff845600700 1 --2- 172.21.15.60:0/85200533 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7ff8280b2910 0x7ff8280b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:22.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.354+0000 7ff845600700 1 -- 172.21.15.60:0/85200533 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7ff8400e3900 con 0x7ff848109600 2024-05-03T14:47:22.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.354+0000 7ff847e00700 1 --2- 172.21.15.60:0/85200533 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7ff8280b2910 0x7ff8280b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:22.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.355+0000 7ff845600700 1 --2- 172.21.15.60:0/85200533 >> [v2:172.21.15.60:6802/2952930162,v1:172.21.15.60:6803/2952930162] conn(0x7ff8280bdaf0 0x7ff8280bff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:22.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.355+0000 7ff845600700 1 -- 172.21.15.60:0/85200533 --> [v2:172.21.15.60:6802/2952930162,v1:172.21.15.60:6803/2952930162] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff8280c0660 con 0x7ff8280bdaf0 2024-05-03T14:47:22.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.355+0000 7ff845600700 1 -- 172.21.15.60:0/85200533 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7ff8400e3cb0 con 0x7ff848109600 2024-05-03T14:47:22.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.355+0000 7ff83f400700 1 --2- 172.21.15.60:0/85200533 >> [v2:172.21.15.60:6802/2952930162,v1:172.21.15.60:6803/2952930162] conn(0x7ff8280bdaf0 0x7ff8280bff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:22.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.355+0000 7ff847e00700 1 --2- 172.21.15.60:0/85200533 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7ff8280b2910 0x7ff8280b4dd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff838005fd0 tx=0x7ff838009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:22.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.356+0000 7ff83f400700 1 --2- 172.21.15.60:0/85200533 >> [v2:172.21.15.60:6802/2952930162,v1:172.21.15.60:6803/2952930162] conn(0x7ff8280bdaf0 0x7ff8280bff90 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:22.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.357+0000 7ff845600700 1 -- 172.21.15.60:0/85200533 <== osd.0 v2:172.21.15.60:6802/2952930162 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff8280c0660 con 0x7ff8280bdaf0 2024-05-03T14:47:22.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.375+0000 7ff84ee00700 1 -- 172.21.15.60:0/85200533 --> [v2:172.21.15.60:6802/2952930162,v1:172.21.15.60:6803/2952930162] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff848104c10 con 0x7ff8280bdaf0 2024-05-03T14:47:22.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.376+0000 7ff845600700 1 -- 172.21.15.60:0/85200533 <== osd.0 v2:172.21.15.60:6802/2952930162 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff848104c10 con 0x7ff8280bdaf0 2024-05-03T14:47:22.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.376+0000 7ff84ee00700 1 -- 172.21.15.60:0/85200533 >> [v2:172.21.15.60:6802/2952930162,v1:172.21.15.60:6803/2952930162] conn(0x7ff8280bdaf0 msgr2=0x7ff8280bff90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:22.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.376+0000 7ff84ee00700 1 --2- 172.21.15.60:0/85200533 >> [v2:172.21.15.60:6802/2952930162,v1:172.21.15.60:6803/2952930162] conn(0x7ff8280bdaf0 0x7ff8280bff90 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:22.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.376+0000 7ff84ee00700 1 -- 172.21.15.60:0/85200533 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7ff8280b2910 msgr2=0x7ff8280b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:22.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.376+0000 7ff84ee00700 1 --2- 172.21.15.60:0/85200533 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7ff8280b2910 0x7ff8280b4dd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff838005fd0 tx=0x7ff838009310 comp rx=0 tx=0).stop 2024-05-03T14:47:22.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.376+0000 7ff84ee00700 1 -- 172.21.15.60:0/85200533 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff848109600 msgr2=0x7ff8481a07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:22.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.376+0000 7ff84ee00700 1 --2- 172.21.15.60:0/85200533 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff848109600 0x7ff8481a07e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff84000d010 tx=0x7ff84000bca0 comp rx=0 tx=0).stop 2024-05-03T14:47:22.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.377+0000 7ff84ce00700 1 -- 172.21.15.60:0/85200533 reap_dead start 2024-05-03T14:47:22.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.377+0000 7ff84ee00700 1 -- 172.21.15.60:0/85200533 shutdown_connections 2024-05-03T14:47:22.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.377+0000 7ff84ee00700 1 -- 172.21.15.60:0/85200533 >> 172.21.15.60:0/85200533 conn(0x7ff84807a650 msgr2=0x7ff848101d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:47:22.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.377+0000 7ff84ee00700 1 -- 172.21.15.60:0/85200533 shutdown_connections 2024-05-03T14:47:22.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:21.377+0000 7ff84ee00700 1 -- 172.21.15.60:0/85200533 wait complete. 2024-05-03T14:47:22.943 INFO:teuthology.orchestra.run.smithi060.stdout:180388626512 2024-05-03T14:47:22.944 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-05-03T14:47:23.259 INFO:teuthology.orchestra.run.smithi060.stdout:34359738483 2024-05-03T14:47:23.259 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-03T14:47:23.360 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:47:23.386 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:47:23.429 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:47:23.458 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:47:23.459 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:47:23.501 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:47:23 smithi060 ceph-mon[28873]: pgmap v418: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T14:47:23.516 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:47:23.653 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:47:23.869 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:47:23.873 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:47:23.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:47:23 smithi090 ceph-mon[33568]: pgmap v418: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T14:47:23.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:47:23 smithi149 ceph-mon[33332]: pgmap v418: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T14:47:25.312 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:47:25.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:47:25 smithi060 ceph-mon[28873]: pgmap v419: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-05-03T14:47:25.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:47:25 smithi090 ceph-mon[33568]: pgmap v419: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-05-03T14:47:25.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:47:25 smithi149 ceph-mon[33332]: pgmap v419: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-05-03T14:47:27.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:47:27 smithi060 ceph-mon[28873]: pgmap v420: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:47:27.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:47:27 smithi090 ceph-mon[33568]: pgmap v420: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:47:27.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:47:27 smithi149 ceph-mon[33332]: pgmap v420: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:47:29.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:47:29 smithi060 ceph-mon[28873]: pgmap v421: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:47:29.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:47:29 smithi090 ceph-mon[33568]: pgmap v421: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:47:29.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:47:29 smithi149 ceph-mon[33332]: pgmap v421: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:47:31.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:47:31 smithi060 ceph-mon[28873]: pgmap v422: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:47:31.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.712+0000 7f5127800700 1 -- 172.21.15.60:0/2853396399 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5120104100 msgr2=0x7f5120108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:31.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.712+0000 7f5127800700 1 --2- 172.21.15.60:0/2853396399 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5120104100 0x7f5120108120 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5108009920 tx=0x7f5108009c30 comp rx=0 tx=0).stop 2024-05-03T14:47:31.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.713+0000 7f5127800700 1 -- 172.21.15.60:0/2853396399 shutdown_connections 2024-05-03T14:47:31.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.713+0000 7f5127800700 1 --2- 172.21.15.60:0/2853396399 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5120108850 0x7f51201103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:31.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.713+0000 7f5127800700 1 --2- 172.21.15.60:0/2853396399 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5120103750 0x7f5120103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:31.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.713+0000 7f5127800700 1 --2- 172.21.15.60:0/2853396399 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5120104100 0x7f5120108120 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:31.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.713+0000 7f5127800700 1 -- 172.21.15.60:0/2853396399 >> 172.21.15.60:0/2853396399 conn(0x7f512007a600 msgr2=0x7f512007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:47:31.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.713+0000 7f5127800700 1 -- 172.21.15.60:0/2853396399 shutdown_connections 2024-05-03T14:47:31.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.713+0000 7f5127800700 1 -- 172.21.15.60:0/2853396399 wait complete. 2024-05-03T14:47:31.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.714+0000 7f5127800700 1 Processor -- start 2024-05-03T14:47:31.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.714+0000 7f5127800700 1 -- start start 2024-05-03T14:47:31.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.715+0000 7f5127800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5120103750 0x7f5120199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:31.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.715+0000 7f5127800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5120104100 0x7f512019a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:31.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.715+0000 7f5127800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5120108850 0x7f512019e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:31.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.715+0000 7f5127800700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f51201138f0 con 0x7f5120103750 2024-05-03T14:47:31.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.715+0000 7f5127800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5120113770 con 0x7f5120104100 2024-05-03T14:47:31.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.715+0000 7f5127800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5120113a70 con 0x7f5120108850 2024-05-03T14:47:31.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.715+0000 7f5117e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5120104100 0x7f512019a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:31.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.715+0000 7f5117e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5120104100 0x7f512019a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.60:60868/0 (socket says 172.21.15.60:60868) 2024-05-03T14:47:31.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.715+0000 7f5117e00700 1 -- 172.21.15.60:0/3947221149 learned_addr learned my addr 172.21.15.60:0/3947221149 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:47:31.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.715+0000 7f5124e00700 1 --2- 172.21.15.60:0/3947221149 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5120103750 0x7f5120199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:31.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.716+0000 7f5125800700 1 --2- 172.21.15.60:0/3947221149 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5120108850 0x7f512019e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:31.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.716+0000 7f5117e00700 1 -- 172.21.15.60:0/3947221149 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5120108850 msgr2=0x7f512019e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:31.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.716+0000 7f5117e00700 1 --2- 172.21.15.60:0/3947221149 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5120108850 0x7f512019e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:31.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.716+0000 7f5117e00700 1 -- 172.21.15.60:0/3947221149 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5120103750 msgr2=0x7f5120199dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:31.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.716+0000 7f5117e00700 1 --2- 172.21.15.60:0/3947221149 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5120103750 0x7f5120199dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:31.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.716+0000 7f5117e00700 1 -- 172.21.15.60:0/3947221149 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f512019ed80 con 0x7f5120104100 2024-05-03T14:47:31.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.716+0000 7f5124e00700 1 --2- 172.21.15.60:0/3947221149 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5120103750 0x7f5120199dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T14:47:31.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.716+0000 7f5117e00700 1 --2- 172.21.15.60:0/3947221149 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5120104100 0x7f512019a300 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5108009230 tx=0x7f510800b560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:31.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.717+0000 7f511e000700 1 -- 172.21.15.60:0/3947221149 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f510800b780 con 0x7f5120104100 2024-05-03T14:47:31.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.717+0000 7f511e000700 1 -- 172.21.15.60:0/3947221149 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f510800f960 con 0x7f5120104100 2024-05-03T14:47:31.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.717+0000 7f5127800700 1 -- 172.21.15.60:0/3947221149 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f512019f010 con 0x7f5120104100 2024-05-03T14:47:31.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.717+0000 7f511e000700 1 -- 172.21.15.60:0/3947221149 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5108003c00 con 0x7f5120104100 2024-05-03T14:47:31.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.717+0000 7f5127800700 1 -- 172.21.15.60:0/3947221149 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51201a68f0 con 0x7f5120104100 2024-05-03T14:47:31.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.718+0000 7f5127800700 1 -- 172.21.15.60:0/3947221149 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5120104be0 con 0x7f5120104100 2024-05-03T14:47:31.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.721+0000 7f511e000700 1 -- 172.21.15.60:0/3947221149 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f510800f460 con 0x7f5120104100 2024-05-03T14:47:31.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.722+0000 7f511e000700 1 --2- 172.21.15.60:0/3947221149 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f51000b2990 0x7f51000b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:31.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.722+0000 7f511e000700 1 -- 172.21.15.60:0/3947221149 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f51080e3a10 con 0x7f5120104100 2024-05-03T14:47:31.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.722+0000 7f5124e00700 1 --2- 172.21.15.60:0/3947221149 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f51000b2990 0x7f51000b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:31.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.723+0000 7f5124e00700 1 --2- 172.21.15.60:0/3947221149 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f51000b2990 0x7f51000b4e50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5110009cb0 tx=0x7f5110009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:31.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.724+0000 7f511e000700 1 -- 172.21.15.60:0/3947221149 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f510805fd70 con 0x7f5120104100 2024-05-03T14:47:31.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.619+0000 7f7e9b800700 1 -- 172.21.15.60:0/3999488691 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e9406b180 msgr2=0x7f7e9406b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:31.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.619+0000 7f7e9b800700 1 --2- 172.21.15.60:0/3999488691 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e9406b180 0x7f7e9406b600 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7e840098f0 tx=0x7f7e84009c00 comp rx=0 tx=0).stop 2024-05-03T14:47:31.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.620+0000 7f7e9b800700 1 -- 172.21.15.60:0/3999488691 shutdown_connections 2024-05-03T14:47:31.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.620+0000 7f7e9b800700 1 --2- 172.21.15.60:0/3999488691 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e9406bb40 0x7f7e94110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:31.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.620+0000 7f7e9b800700 1 --2- 172.21.15.60:0/3999488691 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e94109510 0x7f7e941098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:31.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.620+0000 7f7e9b800700 1 --2- 172.21.15.60:0/3999488691 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e9406b180 0x7f7e9406b600 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:31.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.620+0000 7f7e9b800700 1 -- 172.21.15.60:0/3999488691 >> 172.21.15.60:0/3999488691 conn(0x7f7e94079350 msgr2=0x7f7e94079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:47:31.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.620+0000 7f7e9b800700 1 -- 172.21.15.60:0/3999488691 shutdown_connections 2024-05-03T14:47:31.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.620+0000 7f7e9b800700 1 -- 172.21.15.60:0/3999488691 wait complete. 2024-05-03T14:47:31.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.621+0000 7f7e9b800700 1 Processor -- start 2024-05-03T14:47:31.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.621+0000 7f7e9b800700 1 -- start start 2024-05-03T14:47:31.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.622+0000 7f7e9b800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e9406b180 0x7f7e9419bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:31.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.622+0000 7f7e9b800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e9406bb40 0x7f7e9419c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:31.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.622+0000 7f7e9b800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e94109510 0x7f7e941a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:31.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.622+0000 7f7e9b800700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f7e94113900 con 0x7f7e9406bb40 2024-05-03T14:47:31.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.622+0000 7f7e9b800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7e94113780 con 0x7f7e9406b180 2024-05-03T14:47:31.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.622+0000 7f7e9b800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7e94113a80 con 0x7f7e94109510 2024-05-03T14:47:31.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.622+0000 7f7e98e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e9406b180 0x7f7e9419bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:31.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.622+0000 7f7e93e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e9406bb40 0x7f7e9419c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:31.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.622+0000 7f7e98e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e9406b180 0x7f7e9419bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.60:60838/0 (socket says 172.21.15.60:60838) 2024-05-03T14:47:31.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.622+0000 7f7e98e00700 1 -- 172.21.15.60:0/4152383220 learned_addr learned my addr 172.21.15.60:0/4152383220 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:47:31.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.622+0000 7f7e99800700 1 --2- 172.21.15.60:0/4152383220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e94109510 0x7f7e941a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:31.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.623+0000 7f7e93e00700 1 -- 172.21.15.60:0/4152383220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e94109510 msgr2=0x7f7e941a08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:31.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.623+0000 7f7e93e00700 1 --2- 172.21.15.60:0/4152383220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e94109510 0x7f7e941a08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:31.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.623+0000 7f7e93e00700 1 -- 172.21.15.60:0/4152383220 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e9406b180 msgr2=0x7f7e9419bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:31.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.623+0000 7f7e93e00700 1 --2- 172.21.15.60:0/4152383220 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e9406b180 0x7f7e9419bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:31.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.623+0000 7f7e93e00700 1 -- 172.21.15.60:0/4152383220 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e941a0ff0 con 0x7f7e9406bb40 2024-05-03T14:47:31.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.623+0000 7f7e99800700 1 --2- 172.21.15.60:0/4152383220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e94109510 0x7f7e941a08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:47:31.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.623+0000 7f7e93e00700 1 --2- 172.21.15.60:0/4152383220 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e9406bb40 0x7f7e9419c520 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7e8400c010 tx=0x7f7e84010f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:31.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.624+0000 7f7e91600700 1 -- 172.21.15.60:0/4152383220 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e84027070 con 0x7f7e9406bb40 2024-05-03T14:47:31.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.624+0000 7f7e9b800700 1 -- 172.21.15.60:0/4152383220 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e941a1280 con 0x7f7e9406bb40 2024-05-03T14:47:31.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.624+0000 7f7e91600700 1 -- 172.21.15.60:0/4152383220 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f7e8400bb60 con 0x7f7e9406bb40 2024-05-03T14:47:31.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.624+0000 7f7e91600700 1 -- 172.21.15.60:0/4152383220 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e84004a20 con 0x7f7e9406bb40 2024-05-03T14:47:31.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.624+0000 7f7e9b800700 1 -- 172.21.15.60:0/4152383220 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e941a8b60 con 0x7f7e9406bb40 2024-05-03T14:47:31.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.625+0000 7f7e9b800700 1 -- 172.21.15.60:0/4152383220 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e9406a190 con 0x7f7e9406bb40 2024-05-03T14:47:31.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.628+0000 7f7e91600700 1 -- 172.21.15.60:0/4152383220 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f7e8402a070 con 0x7f7e9406bb40 2024-05-03T14:47:31.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.631+0000 7f7e91600700 1 --2- 172.21.15.60:0/4152383220 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f7e7c0b2910 0x7f7e7c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:31.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.631+0000 7f7e91600700 1 -- 172.21.15.60:0/4152383220 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f7e840e3260 con 0x7f7e9406bb40 2024-05-03T14:47:31.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.631+0000 7f7e98e00700 1 --2- 172.21.15.60:0/4152383220 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f7e7c0b2910 0x7f7e7c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:31.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.632+0000 7f7e98e00700 1 --2- 172.21.15.60:0/4152383220 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f7e7c0b2910 0x7f7e7c0b4dd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7e88009750 tx=0x7f7e88009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:31.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.634+0000 7f7e91600700 1 -- 172.21.15.60:0/4152383220 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f7e8405fd70 con 0x7f7e9406bb40 2024-05-03T14:47:31.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:47:31 smithi090 ceph-mon[33568]: pgmap v422: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:47:31.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.885+0000 7f7e9b800700 1 -- 172.21.15.60:0/4152383220 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f7e941a8d30 con 0x7f7e9406bb40 2024-05-03T14:47:31.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.886+0000 7f7e91600700 1 -- 172.21.15.60:0/4152383220 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f7e84013030 con 0x7f7e9406bb40 2024-05-03T14:47:31.888 INFO:teuthology.orchestra.run.smithi060.stdout:253403070529 2024-05-03T14:47:31.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.889+0000 7f7e9b800700 1 -- 172.21.15.60:0/4152383220 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f7e7c0b2910 msgr2=0x7f7e7c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:31.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.889+0000 7f7e9b800700 1 --2- 172.21.15.60:0/4152383220 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f7e7c0b2910 0x7f7e7c0b4dd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7e88009750 tx=0x7f7e88009310 comp rx=0 tx=0).stop 2024-05-03T14:47:31.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.889+0000 7f7e9b800700 1 -- 172.21.15.60:0/4152383220 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e9406bb40 msgr2=0x7f7e9419c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:31.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.889+0000 7f7e9b800700 1 --2- 172.21.15.60:0/4152383220 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e9406bb40 0x7f7e9419c520 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7e8400c010 tx=0x7f7e84010f80 comp rx=0 tx=0).stop 2024-05-03T14:47:31.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.889+0000 7f7e9b800700 1 -- 172.21.15.60:0/4152383220 shutdown_connections 2024-05-03T14:47:31.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.889+0000 7f7e9b800700 1 --2- 172.21.15.60:0/4152383220 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f7e7c0b2910 0x7f7e7c0b4dd0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:31.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.889+0000 7f7e9b800700 1 --2- 172.21.15.60:0/4152383220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e94109510 0x7f7e941a08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:31.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.889+0000 7f7e9b800700 1 --2- 172.21.15.60:0/4152383220 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e9406b180 0x7f7e9419bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:31.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.889+0000 7f7e9b800700 1 --2- 172.21.15.60:0/4152383220 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e9406bb40 0x7f7e9419c520 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:31.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.889+0000 7f7e9b800700 1 -- 172.21.15.60:0/4152383220 >> 172.21.15.60:0/4152383220 conn(0x7f7e94079350 msgr2=0x7f7e9410e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:47:31.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.889+0000 7f7e9b800700 1 -- 172.21.15.60:0/4152383220 shutdown_connections 2024-05-03T14:47:31.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.889+0000 7f7e9b800700 1 -- 172.21.15.60:0/4152383220 wait complete. 2024-05-03T14:47:31.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:47:31 smithi149 ceph-mon[33332]: pgmap v422: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:47:31.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.984+0000 7f5127800700 1 -- 172.21.15.60:0/3947221149 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f512002e510 con 0x7f5120104100 2024-05-03T14:47:31.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.985+0000 7f511e000700 1 -- 172.21.15.60:0/3947221149 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f5108013030 con 0x7f5120104100 2024-05-03T14:47:31.987 INFO:teuthology.orchestra.run.smithi060.stdout:274877907003 2024-05-03T14:47:31.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.987+0000 7f5127800700 1 -- 172.21.15.60:0/3947221149 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f51000b2990 msgr2=0x7f51000b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:31.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.987+0000 7f5127800700 1 --2- 172.21.15.60:0/3947221149 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f51000b2990 0x7f51000b4e50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5110009cb0 tx=0x7f5110009260 comp rx=0 tx=0).stop 2024-05-03T14:47:31.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.987+0000 7f5127800700 1 -- 172.21.15.60:0/3947221149 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5120104100 msgr2=0x7f512019a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:31.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.987+0000 7f5127800700 1 --2- 172.21.15.60:0/3947221149 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5120104100 0x7f512019a300 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5108009230 tx=0x7f510800b560 comp rx=0 tx=0).stop 2024-05-03T14:47:31.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.988+0000 7f5127800700 1 -- 172.21.15.60:0/3947221149 shutdown_connections 2024-05-03T14:47:31.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.988+0000 7f5127800700 1 --2- 172.21.15.60:0/3947221149 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f51000b2990 0x7f51000b4e50 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:31.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.988+0000 7f5127800700 1 --2- 172.21.15.60:0/3947221149 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5120108850 0x7f512019e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:31.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.988+0000 7f5127800700 1 --2- 172.21.15.60:0/3947221149 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5120103750 0x7f5120199dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:31.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.988+0000 7f5127800700 1 --2- 172.21.15.60:0/3947221149 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5120104100 0x7f512019a300 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:31.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.988+0000 7f5127800700 1 -- 172.21.15.60:0/3947221149 >> 172.21.15.60:0/3947221149 conn(0x7f512007a600 msgr2=0x7f51201015c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:47:31.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.988+0000 7f5127800700 1 -- 172.21.15.60:0/3947221149 shutdown_connections 2024-05-03T14:47:31.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:31.988+0000 7f5127800700 1 -- 172.21.15.60:0/3947221149 wait complete. 2024-05-03T14:47:32.797 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:47:32 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/4152383220' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-03T14:47:32.798 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:47:32 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/3947221149' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-03T14:47:32.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:47:32 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/4152383220' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-03T14:47:32.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:47:32 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/3947221149' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-03T14:47:32.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:47:32 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/4152383220' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-03T14:47:32.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:47:32 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/3947221149' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-03T14:47:33.574 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:47:33.799 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:47:33 smithi060 ceph-mon[28873]: pgmap v423: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T14:47:33.799 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:47:33 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/1789096868' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-03T14:47:33.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:47:33 smithi090 ceph-mon[33568]: pgmap v423: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T14:47:33.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:47:33 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/1789096868' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-03T14:47:33.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:47:33 smithi149 ceph-mon[33332]: pgmap v423: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T14:47:33.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:47:33 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/1789096868' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-03T14:47:33.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:32.401+0000 7f1666200700 1 -- 172.21.15.60:0/1584365913 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f16600756d0 msgr2=0x7f1660075ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:33.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:32.401+0000 7f1666200700 1 --2- 172.21.15.60:0/1584365913 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f16600756d0 0x7f1660075ab0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1650009f90 tx=0x7f165000b330 comp rx=0 tx=0).stop 2024-05-03T14:47:33.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:32.402+0000 7f1666200700 1 -- 172.21.15.60:0/1584365913 shutdown_connections 2024-05-03T14:47:33.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:32.402+0000 7f1666200700 1 --2- 172.21.15.60:0/1584365913 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1660114670 0x7f1660116a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:33.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:32.402+0000 7f1666200700 1 --2- 172.21.15.60:0/1584365913 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f16600756d0 0x7f1660075ab0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:33.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:32.402+0000 7f1666200700 1 --2- 172.21.15.60:0/1584365913 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1660075ff0 0x7f1660114130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:33.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:32.402+0000 7f1666200700 1 -- 172.21.15.60:0/1584365913 >> 172.21.15.60:0/1584365913 conn(0x7f166007a600 msgr2=0x7f166007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:47:33.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:32.402+0000 7f1666200700 1 -- 172.21.15.60:0/1584365913 shutdown_connections 2024-05-03T14:47:33.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:32.402+0000 7f1666200700 1 -- 172.21.15.60:0/1584365913 wait complete. 2024-05-03T14:47:33.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:32.403+0000 7f1666200700 1 Processor -- start 2024-05-03T14:47:33.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:32.403+0000 7f1666200700 1 -- start start 2024-05-03T14:47:33.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:32.403+0000 7f1666200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f16600756d0 0x7f16601a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:33.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:32.403+0000 7f1666200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1660075ff0 0x7f16601a0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:33.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:32.403+0000 7f1666200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1660114670 0x7f16601a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:33.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:32.403+0000 7f1666200700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f1660119ef0 con 0x7f1660075ff0 2024-05-03T14:47:33.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:32.403+0000 7f1666200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1660119d70 con 0x7f16600756d0 2024-05-03T14:47:33.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:32.403+0000 7f1666200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f166011a070 con 0x7f1660114670 2024-05-03T14:47:33.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:32.404+0000 7f165ea00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1660075ff0 0x7f16601a0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:33.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:32.404+0000 7f165ea00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1660075ff0 0x7f16601a0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37654/0 (socket says 172.21.15.60:37654) 2024-05-03T14:47:33.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:32.404+0000 7f165ea00700 1 -- 172.21.15.60:0/1789096868 learned_addr learned my addr 172.21.15.60:0/1789096868 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:47:33.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:32.404+0000 7f165f400700 1 --2- 172.21.15.60:0/1789096868 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f16600756d0 0x7f16601a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:33.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:32.404+0000 7f165fe00700 1 --2- 172.21.15.60:0/1789096868 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1660114670 0x7f16601a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:33.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:32.404+0000 7f165f400700 1 -- 172.21.15.60:0/1789096868 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1660114670 msgr2=0x7f16601a4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:33.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:32.404+0000 7f165f400700 1 --2- 172.21.15.60:0/1789096868 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1660114670 0x7f16601a4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:33.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:32.404+0000 7f165f400700 1 -- 172.21.15.60:0/1789096868 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1660075ff0 msgr2=0x7f16601a0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:33.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:32.404+0000 7f165f400700 1 --2- 172.21.15.60:0/1789096868 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1660075ff0 0x7f16601a0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:33.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:32.404+0000 7f165f400700 1 -- 172.21.15.60:0/1789096868 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16601a5330 con 0x7f16600756d0 2024-05-03T14:47:33.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:32.404+0000 7f165ea00700 1 --2- 172.21.15.60:0/1789096868 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1660075ff0 0x7f16601a0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:47:33.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:32.404+0000 7f165fe00700 1 --2- 172.21.15.60:0/1789096868 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1660114670 0x7f16601a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:47:33.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:32.405+0000 7f165f400700 1 --2- 172.21.15.60:0/1789096868 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f16600756d0 0x7f16601a0320 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1650000f80 tx=0x7f1650009510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:33.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:32.405+0000 7f1657e00700 1 -- 172.21.15.60:0/1789096868 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1650021070 con 0x7f16600756d0 2024-05-03T14:47:33.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:32.405+0000 7f1657e00700 1 -- 172.21.15.60:0/1789096868 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f165001d450 con 0x7f16600756d0 2024-05-03T14:47:33.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:32.405+0000 7f1666200700 1 -- 172.21.15.60:0/1789096868 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16601a55c0 con 0x7f16600756d0 2024-05-03T14:47:33.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:32.405+0000 7f1657e00700 1 -- 172.21.15.60:0/1789096868 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16500052a0 con 0x7f16600756d0 2024-05-03T14:47:33.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:32.405+0000 7f1666200700 1 -- 172.21.15.60:0/1789096868 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16601acea0 con 0x7f16600756d0 2024-05-03T14:47:33.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:32.406+0000 7f1666200700 1 -- 172.21.15.60:0/1789096868 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f166006a190 con 0x7f16600756d0 2024-05-03T14:47:33.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:32.409+0000 7f1657e00700 1 -- 172.21.15.60:0/1789096868 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f1650024070 con 0x7f16600756d0 2024-05-03T14:47:33.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:32.410+0000 7f1657e00700 1 --2- 172.21.15.60:0/1789096868 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f16440b2990 0x7f16440b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:33.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:32.410+0000 7f1657e00700 1 -- 172.21.15.60:0/1789096868 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f165000f020 con 0x7f16600756d0 2024-05-03T14:47:33.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:32.411+0000 7f165ea00700 1 --2- 172.21.15.60:0/1789096868 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f16440b2990 0x7f16440b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:33.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:32.411+0000 7f165ea00700 1 --2- 172.21.15.60:0/1789096868 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f16440b2990 0x7f16440b4e50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1660116550 tx=0x7f164c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:33.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:32.412+0000 7f1657e00700 1 -- 172.21.15.60:0/1789096868 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f165002b020 con 0x7f16600756d0 2024-05-03T14:47:33.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:32.661+0000 7f1666200700 1 -- 172.21.15.60:0/1789096868 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f16601ad070 con 0x7f16600756d0 2024-05-03T14:47:33.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:32.661+0000 7f1657e00700 1 -- 172.21.15.60:0/1789096868 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f16500ab650 con 0x7f16600756d0 2024-05-03T14:47:33.998 INFO:teuthology.orchestra.run.smithi060.stdout:227633266760 2024-05-03T14:47:33.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:32.664+0000 7f1666200700 1 -- 172.21.15.60:0/1789096868 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f16440b2990 msgr2=0x7f16440b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:33.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:32.664+0000 7f1666200700 1 --2- 172.21.15.60:0/1789096868 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f16440b2990 0x7f16440b4e50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1660116550 tx=0x7f164c009260 comp rx=0 tx=0).stop 2024-05-03T14:47:33.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:32.664+0000 7f1666200700 1 -- 172.21.15.60:0/1789096868 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f16600756d0 msgr2=0x7f16601a0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:33.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:32.664+0000 7f1666200700 1 --2- 172.21.15.60:0/1789096868 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f16600756d0 0x7f16601a0320 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1650000f80 tx=0x7f1650009510 comp rx=0 tx=0).stop 2024-05-03T14:47:33.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:32.664+0000 7f1666200700 1 -- 172.21.15.60:0/1789096868 shutdown_connections 2024-05-03T14:47:33.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:32.664+0000 7f1666200700 1 --2- 172.21.15.60:0/1789096868 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f16440b2990 0x7f16440b4e50 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:33.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:32.664+0000 7f1666200700 1 --2- 172.21.15.60:0/1789096868 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1660114670 0x7f16601a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:33.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:32.664+0000 7f1666200700 1 --2- 172.21.15.60:0/1789096868 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f16600756d0 0x7f16601a0320 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:33.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:32.664+0000 7f1666200700 1 --2- 172.21.15.60:0/1789096868 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1660075ff0 0x7f16601a0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:33.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:32.664+0000 7f1666200700 1 -- 172.21.15.60:0/1789096868 >> 172.21.15.60:0/1789096868 conn(0x7f166007a600 msgr2=0x7f1660113430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:47:33.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:32.665+0000 7f1666200700 1 -- 172.21.15.60:0/1789096868 shutdown_connections 2024-05-03T14:47:33.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:32.665+0000 7f1666200700 1 -- 172.21.15.60:0/1789096868 wait complete. 2024-05-03T14:47:35.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:47:35 smithi060 ceph-mon[28873]: pgmap v424: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-03T14:47:35.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:47:35 smithi090 ceph-mon[33568]: pgmap v424: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-03T14:47:35.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:47:35 smithi149 ceph-mon[33332]: pgmap v424: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-03T14:47:36.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:47:36 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/1439542830' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-03T14:47:36.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:35.881+0000 7f3aeb400700 1 -- 172.21.15.60:0/3651191011 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ae41056c0 msgr2=0x7f3ae4105b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:36.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:35.881+0000 7f3aeb400700 1 --2- 172.21.15.60:0/3651191011 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ae41056c0 0x7f3ae4105b40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0009990 tx=0x7f3ad0009ca0 comp rx=0 tx=0).stop 2024-05-03T14:47:36.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:35.881+0000 7f3aeb400700 1 -- 172.21.15.60:0/3651191011 shutdown_connections 2024-05-03T14:47:36.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:35.881+0000 7f3aeb400700 1 --2- 172.21.15.60:0/3651191011 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ae4106080 0x7f3ae4112590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:36.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:35.881+0000 7f3aeb400700 1 --2- 172.21.15.60:0/3651191011 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ae410b6c0 0x7f3ae410baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:36.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:35.881+0000 7f3aeb400700 1 --2- 172.21.15.60:0/3651191011 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ae41056c0 0x7f3ae4105b40 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:36.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:35.881+0000 7f3aeb400700 1 -- 172.21.15.60:0/3651191011 >> 172.21.15.60:0/3651191011 conn(0x7f3ae41011b0 msgr2=0x7f3ae41035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:47:36.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:35.881+0000 7f3aeb400700 1 -- 172.21.15.60:0/3651191011 shutdown_connections 2024-05-03T14:47:36.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:35.881+0000 7f3aeb400700 1 -- 172.21.15.60:0/3651191011 wait complete. 2024-05-03T14:47:36.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:35.882+0000 7f3aeb400700 1 Processor -- start 2024-05-03T14:47:36.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:35.883+0000 7f3aeb400700 1 -- start start 2024-05-03T14:47:36.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:35.883+0000 7f3aeb400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ae41056c0 0x7f3ae419bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:36.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:35.883+0000 7f3aeb400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ae4106080 0x7f3ae419c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:36.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:35.883+0000 7f3aeb400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ae410b6c0 0x7f3ae41a07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:36.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:35.883+0000 7f3aeb400700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae4115b70 con 0x7f3ae4106080 2024-05-03T14:47:36.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:35.883+0000 7f3aeb400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae41159f0 con 0x7f3ae410b6c0 2024-05-03T14:47:36.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:35.883+0000 7f3aeb400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae4115cf0 con 0x7f3ae41056c0 2024-05-03T14:47:36.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:35.883+0000 7f3ae3e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ae4106080 0x7f3ae419c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:36.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:35.883+0000 7f3ae3e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ae4106080 0x7f3ae419c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37662/0 (socket says 172.21.15.60:37662) 2024-05-03T14:47:36.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:35.883+0000 7f3ae3e00700 1 -- 172.21.15.60:0/1439542830 learned_addr learned my addr 172.21.15.60:0/1439542830 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:47:36.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:35.883+0000 7f3ae9400700 1 --2- 172.21.15.60:0/1439542830 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ae410b6c0 0x7f3ae41a07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:36.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:35.883+0000 7f3ae8a00700 1 --2- 172.21.15.60:0/1439542830 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ae41056c0 0x7f3ae419bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:36.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:35.884+0000 7f3ae9400700 1 -- 172.21.15.60:0/1439542830 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ae41056c0 msgr2=0x7f3ae419bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:36.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:35.884+0000 7f3ae9400700 1 --2- 172.21.15.60:0/1439542830 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ae41056c0 0x7f3ae419bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:36.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:35.884+0000 7f3ae9400700 1 -- 172.21.15.60:0/1439542830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ae4106080 msgr2=0x7f3ae419c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:36.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:35.884+0000 7f3ae9400700 1 --2- 172.21.15.60:0/1439542830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ae4106080 0x7f3ae419c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:36.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:35.884+0000 7f3ae9400700 1 -- 172.21.15.60:0/1439542830 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ae41a0f10 con 0x7f3ae410b6c0 2024-05-03T14:47:36.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:35.884+0000 7f3ae3e00700 1 --2- 172.21.15.60:0/1439542830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ae4106080 0x7f3ae419c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:47:36.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:35.884+0000 7f3ae9400700 1 --2- 172.21.15.60:0/1439542830 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ae410b6c0 0x7f3ae41a07d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3adc00efc0 tx=0x7f3adc00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:36.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:35.884+0000 7f3ae1600700 1 -- 172.21.15.60:0/1439542830 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3adc0092e0 con 0x7f3ae410b6c0 2024-05-03T14:47:36.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:35.885+0000 7f3ae1600700 1 -- 172.21.15.60:0/1439542830 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f3adc004bb0 con 0x7f3ae410b6c0 2024-05-03T14:47:36.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:35.885+0000 7f3aeb400700 1 -- 172.21.15.60:0/1439542830 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ae41a11a0 con 0x7f3ae410b6c0 2024-05-03T14:47:36.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:35.885+0000 7f3ae1600700 1 -- 172.21.15.60:0/1439542830 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3adc004280 con 0x7f3ae410b6c0 2024-05-03T14:47:36.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:35.885+0000 7f3aeb400700 1 -- 172.21.15.60:0/1439542830 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ae41a8a80 con 0x7f3ae410b6c0 2024-05-03T14:47:36.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:35.886+0000 7f3aeb400700 1 -- 172.21.15.60:0/1439542830 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ae406a190 con 0x7f3ae410b6c0 2024-05-03T14:47:36.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:35.888+0000 7f3ae1600700 1 -- 172.21.15.60:0/1439542830 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f3adc01b030 con 0x7f3ae410b6c0 2024-05-03T14:47:36.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:35.890+0000 7f3ae1600700 1 --2- 172.21.15.60:0/1439542830 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f3acc0b2910 0x7f3acc0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:36.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:35.890+0000 7f3ae1600700 1 -- 172.21.15.60:0/1439542830 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f3adc0e23c0 con 0x7f3ae410b6c0 2024-05-03T14:47:36.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:35.890+0000 7f3ae8a00700 1 --2- 172.21.15.60:0/1439542830 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f3acc0b2910 0x7f3acc0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:36.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:35.891+0000 7f3ae8a00700 1 --2- 172.21.15.60:0/1439542830 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f3acc0b2910 0x7f3acc0b4dd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8009be0 tx=0x7f3ad8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:36.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:35.892+0000 7f3ae1600700 1 -- 172.21.15.60:0/1439542830 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f3adc022020 con 0x7f3ae410b6c0 2024-05-03T14:47:36.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:36.133+0000 7f3aeb400700 1 -- 172.21.15.60:0/1439542830 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f3ae402e510 con 0x7f3ae410b6c0 2024-05-03T14:47:36.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:36.133+0000 7f3ae1600700 1 -- 172.21.15.60:0/1439542830 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f3adc010040 con 0x7f3ae410b6c0 2024-05-03T14:47:36.845 INFO:teuthology.orchestra.run.smithi060.stdout:300647710774 2024-05-03T14:47:36.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:36.136+0000 7f3aeb400700 1 -- 172.21.15.60:0/1439542830 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f3acc0b2910 msgr2=0x7f3acc0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:36.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:36.136+0000 7f3aeb400700 1 --2- 172.21.15.60:0/1439542830 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f3acc0b2910 0x7f3acc0b4dd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8009be0 tx=0x7f3ad8009260 comp rx=0 tx=0).stop 2024-05-03T14:47:36.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:36.136+0000 7f3aeb400700 1 -- 172.21.15.60:0/1439542830 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ae410b6c0 msgr2=0x7f3ae41a07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:36.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:36.136+0000 7f3aeb400700 1 --2- 172.21.15.60:0/1439542830 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ae410b6c0 0x7f3ae41a07d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3adc00efc0 tx=0x7f3adc00c3a0 comp rx=0 tx=0).stop 2024-05-03T14:47:36.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:36.136+0000 7f3aeb400700 1 -- 172.21.15.60:0/1439542830 shutdown_connections 2024-05-03T14:47:36.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:36.136+0000 7f3aeb400700 1 --2- 172.21.15.60:0/1439542830 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f3acc0b2910 0x7f3acc0b4dd0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:36.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:36.136+0000 7f3aeb400700 1 --2- 172.21.15.60:0/1439542830 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ae410b6c0 0x7f3ae41a07d0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:36.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:36.136+0000 7f3aeb400700 1 --2- 172.21.15.60:0/1439542830 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ae41056c0 0x7f3ae419bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:36.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:36.136+0000 7f3aeb400700 1 --2- 172.21.15.60:0/1439542830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ae4106080 0x7f3ae419c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:36.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:36.137+0000 7f3aeb400700 1 -- 172.21.15.60:0/1439542830 >> 172.21.15.60:0/1439542830 conn(0x7f3ae41011b0 msgr2=0x7f3ae4101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:47:36.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:36.137+0000 7f3aeb400700 1 -- 172.21.15.60:0/1439542830 shutdown_connections 2024-05-03T14:47:36.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:36.137+0000 7f3aeb400700 1 -- 172.21.15.60:0/1439542830 wait complete. 2024-05-03T14:47:36.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:47:36 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/1439542830' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-03T14:47:36.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:47:36 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/1439542830' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-03T14:47:37.804 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:47:37 smithi060 ceph-mon[28873]: pgmap v425: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-03T14:47:37.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:47:37 smithi090 ceph-mon[33568]: pgmap v425: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-03T14:47:37.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:47:37 smithi149 ceph-mon[33332]: pgmap v425: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-03T14:47:38.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:47:38 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:47:38.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:47:38 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:47:38.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:47:38 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:47:39.023 INFO:tasks.cephadm.ceph_manager.ceph:need seq 300647710770 got 300647710774 for osd.11 2024-05-03T14:47:39.023 DEBUG:teuthology.parallel:result is None 2024-05-03T14:47:39.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:38.299+0000 7fdff4a00700 1 -- 172.21.15.60:0/498039844 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdff0075ff0 msgr2=0x7fdff0114130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:39.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:38.299+0000 7fdff4a00700 1 --2- 172.21.15.60:0/498039844 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdff0075ff0 0x7fdff0114130 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc009990 tx=0x7fdfdc009ca0 comp rx=0 tx=0).stop 2024-05-03T14:47:39.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:38.299+0000 7fdff4a00700 1 -- 172.21.15.60:0/498039844 shutdown_connections 2024-05-03T14:47:39.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:38.299+0000 7fdff4a00700 1 --2- 172.21.15.60:0/498039844 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdff0114670 0x7fdff0116a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:39.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:38.299+0000 7fdff4a00700 1 --2- 172.21.15.60:0/498039844 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdff00756d0 0x7fdff0075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:39.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:38.299+0000 7fdff4a00700 1 --2- 172.21.15.60:0/498039844 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdff0075ff0 0x7fdff0114130 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:39.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:38.299+0000 7fdff4a00700 1 -- 172.21.15.60:0/498039844 >> 172.21.15.60:0/498039844 conn(0x7fdff007a600 msgr2=0x7fdff007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:47:39.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:38.299+0000 7fdff4a00700 1 -- 172.21.15.60:0/498039844 shutdown_connections 2024-05-03T14:47:39.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:38.299+0000 7fdff4a00700 1 -- 172.21.15.60:0/498039844 wait complete. 2024-05-03T14:47:39.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:38.300+0000 7fdff4a00700 1 Processor -- start 2024-05-03T14:47:39.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:38.300+0000 7fdff4a00700 1 -- start start 2024-05-03T14:47:39.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:38.301+0000 7fdff4a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdff00756d0 0x7fdff01a0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:39.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:38.301+0000 7fdff4a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdff0075ff0 0x7fdff01a08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:39.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:38.301+0000 7fdff4a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdff0114670 0x7fdff01a4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:39.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:38.301+0000 7fdff4a00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fdff0119f40 con 0x7fdff00756d0 2024-05-03T14:47:39.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:38.301+0000 7fdff4a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdff0119dc0 con 0x7fdff0114670 2024-05-03T14:47:39.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:38.301+0000 7fdff4a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fdff011a0c0 con 0x7fdff0075ff0 2024-05-03T14:47:39.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:38.301+0000 7fdfede00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdff00756d0 0x7fdff01a0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:39.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:38.301+0000 7fdfede00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdff00756d0 0x7fdff01a0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37684/0 (socket says 172.21.15.60:37684) 2024-05-03T14:47:39.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:38.301+0000 7fdfede00700 1 -- 172.21.15.60:0/2653282341 learned_addr learned my addr 172.21.15.60:0/2653282341 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:47:39.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:38.301+0000 7fdfed400700 1 --2- 172.21.15.60:0/2653282341 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdff0075ff0 0x7fdff01a08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:39.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:38.301+0000 7fdfee800700 1 --2- 172.21.15.60:0/2653282341 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdff0114670 0x7fdff01a4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:39.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:38.302+0000 7fdfede00700 1 -- 172.21.15.60:0/2653282341 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdff0075ff0 msgr2=0x7fdff01a08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:39.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:38.302+0000 7fdfede00700 1 --2- 172.21.15.60:0/2653282341 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdff0075ff0 0x7fdff01a08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:39.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:38.302+0000 7fdfede00700 1 -- 172.21.15.60:0/2653282341 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdff0114670 msgr2=0x7fdff01a4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:39.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:38.302+0000 7fdfede00700 1 --2- 172.21.15.60:0/2653282341 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdff0114670 0x7fdff01a4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:39.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:38.302+0000 7fdfede00700 1 -- 172.21.15.60:0/2653282341 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdff01a53a0 con 0x7fdff00756d0 2024-05-03T14:47:39.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:38.302+0000 7fdfede00700 1 --2- 172.21.15.60:0/2653282341 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdff00756d0 0x7fdff01a0390 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fdfe000d950 tx=0x7fdfe000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:39.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:38.302+0000 7fdfe6a00700 1 -- 172.21.15.60:0/2653282341 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe0016070 con 0x7fdff00756d0 2024-05-03T14:47:39.274 INFO:teuthology.orchestra.run.smithi060.stdout:154618822746 2024-05-03T14:47:39.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:38.303+0000 7fdff4a00700 1 -- 172.21.15.60:0/2653282341 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdff01a5690 con 0x7fdff00756d0 2024-05-03T14:47:39.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:38.303+0000 7fdfe6a00700 1 -- 172.21.15.60:0/2653282341 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7fdfe000fd90 con 0x7fdff00756d0 2024-05-03T14:47:39.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:38.303+0000 7fdfe6a00700 1 -- 172.21.15.60:0/2653282341 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe000e610 con 0x7fdff00756d0 2024-05-03T14:47:39.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:38.303+0000 7fdff4a00700 1 -- 172.21.15.60:0/2653282341 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdff01acf10 con 0x7fdff00756d0 2024-05-03T14:47:39.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:38.304+0000 7fdff4a00700 1 -- 172.21.15.60:0/2653282341 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdff006a190 con 0x7fdff00756d0 2024-05-03T14:47:39.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:38.306+0000 7fdfe6a00700 1 -- 172.21.15.60:0/2653282341 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fdfe0019070 con 0x7fdff00756d0 2024-05-03T14:47:39.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:38.308+0000 7fdfe6a00700 1 --2- 172.21.15.60:0/2653282341 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fdfd40b2910 0x7fdfd40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:39.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:38.308+0000 7fdfe6a00700 1 -- 172.21.15.60:0/2653282341 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7fdfe00e2240 con 0x7fdff00756d0 2024-05-03T14:47:39.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:38.308+0000 7fdfed400700 1 --2- 172.21.15.60:0/2653282341 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fdfd40b2910 0x7fdfd40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:39.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:38.309+0000 7fdfed400700 1 --2- 172.21.15.60:0/2653282341 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fdfd40b2910 0x7fdfd40b4dd0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdff0110f00 tx=0x7fdfdc00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:39.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:38.312+0000 7fdfe6a00700 1 -- 172.21.15.60:0/2653282341 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fdfe00a9a20 con 0x7fdff00756d0 2024-05-03T14:47:39.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:38.588+0000 7fdff4a00700 1 -- 172.21.15.60:0/2653282341 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fdff01ad1b0 con 0x7fdff00756d0 2024-05-03T14:47:39.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:38.588+0000 7fdfe6a00700 1 -- 172.21.15.60:0/2653282341 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fdfe00a9320 con 0x7fdff00756d0 2024-05-03T14:47:39.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:38.591+0000 7fdff4a00700 1 -- 172.21.15.60:0/2653282341 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fdfd40b2910 msgr2=0x7fdfd40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:39.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:38.591+0000 7fdff4a00700 1 --2- 172.21.15.60:0/2653282341 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fdfd40b2910 0x7fdfd40b4dd0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdff0110f00 tx=0x7fdfdc00b540 comp rx=0 tx=0).stop 2024-05-03T14:47:39.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:38.591+0000 7fdff4a00700 1 -- 172.21.15.60:0/2653282341 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdff00756d0 msgr2=0x7fdff01a0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:39.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:38.591+0000 7fdff4a00700 1 --2- 172.21.15.60:0/2653282341 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdff00756d0 0x7fdff01a0390 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fdfe000d950 tx=0x7fdfe000dc60 comp rx=0 tx=0).stop 2024-05-03T14:47:39.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:38.592+0000 7fdff4a00700 1 -- 172.21.15.60:0/2653282341 shutdown_connections 2024-05-03T14:47:39.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:38.592+0000 7fdff4a00700 1 --2- 172.21.15.60:0/2653282341 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fdfd40b2910 0x7fdfd40b4dd0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:39.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:38.592+0000 7fdff4a00700 1 --2- 172.21.15.60:0/2653282341 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdff0114670 0x7fdff01a4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:39.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:38.592+0000 7fdff4a00700 1 --2- 172.21.15.60:0/2653282341 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdff00756d0 0x7fdff01a0390 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:39.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:38.592+0000 7fdff4a00700 1 --2- 172.21.15.60:0/2653282341 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdff0075ff0 0x7fdff01a08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:39.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:38.592+0000 7fdff4a00700 1 -- 172.21.15.60:0/2653282341 >> 172.21.15.60:0/2653282341 conn(0x7fdff007a600 msgr2=0x7fdff01133c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:47:39.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:38.592+0000 7fdff4a00700 1 -- 172.21.15.60:0/2653282341 shutdown_connections 2024-05-03T14:47:39.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:38.593+0000 7fdff4a00700 1 -- 172.21.15.60:0/2653282341 wait complete. 2024-05-03T14:47:39.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:47:39 smithi060 ceph-mon[28873]: pgmap v426: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-03T14:47:39.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:47:39 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/2653282341' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-03T14:47:39.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:47:39 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:47:39.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:47:39 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:47:39.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:47:39 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:47:39.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:47:39 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:47:39.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:47:39 smithi090 ceph-mon[33568]: pgmap v426: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-03T14:47:39.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:47:39 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/2653282341' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-03T14:47:39.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:47:39 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:47:39.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:47:39 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:47:39.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:47:39 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:47:39.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:47:39 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:47:39.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:47:39 smithi149 ceph-mon[33332]: pgmap v426: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-03T14:47:39.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:47:39 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/2653282341' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-03T14:47:39.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:47:39 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:47:39.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:47:39 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:47:39.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:47:39 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:47:39.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:47:39 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:47:40.898 INFO:tasks.cephadm.ceph_manager.ceph:need seq 274877906999 got 274877907003 for osd.10 2024-05-03T14:47:40.898 DEBUG:teuthology.parallel:result is None 2024-05-03T14:47:41.238 INFO:tasks.cephadm.ceph_manager.ceph:need seq 227633266756 got 227633266760 for osd.8 2024-05-03T14:47:41.238 DEBUG:teuthology.parallel:result is None 2024-05-03T14:47:41.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:41.450+0000 7f377e200700 1 -- 172.21.15.60:0/3952364548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f37780782f0 msgr2=0x7f3778113e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:41.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:41.450+0000 7f377e200700 1 --2- 172.21.15.60:0/3952364548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f37780782f0 0x7f3778113e10 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3764009990 tx=0x7f3764009ca0 comp rx=0 tx=0).stop 2024-05-03T14:47:41.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:41.450+0000 7f377e200700 1 -- 172.21.15.60:0/3952364548 shutdown_connections 2024-05-03T14:47:41.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:41.450+0000 7f377e200700 1 --2- 172.21.15.60:0/3952364548 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3778114350 0x7f3778116740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:41.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:41.450+0000 7f377e200700 1 --2- 172.21.15.60:0/3952364548 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f37780779d0 0x7f3778077db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:41.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:41.450+0000 7f377e200700 1 --2- 172.21.15.60:0/3952364548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f37780782f0 0x7f3778113e10 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:41.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:41.450+0000 7f377e200700 1 -- 172.21.15.60:0/3952364548 >> 172.21.15.60:0/3952364548 conn(0x7f37781011a0 msgr2=0x7f37781035c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:47:41.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:41.450+0000 7f377e200700 1 -- 172.21.15.60:0/3952364548 shutdown_connections 2024-05-03T14:47:41.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:41.450+0000 7f377e200700 1 -- 172.21.15.60:0/3952364548 wait complete. 2024-05-03T14:47:41.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:41.451+0000 7f377e200700 1 Processor -- start 2024-05-03T14:47:41.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:41.452+0000 7f377e200700 1 -- start start 2024-05-03T14:47:41.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:41.452+0000 7f377e200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f37780779d0 0x7f37781a0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:41.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:41.452+0000 7f377e200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37780782f0 0x7f37781a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:41.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:41.452+0000 7f377e200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3778114350 0x7f37781a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:41.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:41.452+0000 7f377e200700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f3778119f50 con 0x7f37780779d0 2024-05-03T14:47:41.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:41.452+0000 7f377e200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3778119dd0 con 0x7f37780782f0 2024-05-03T14:47:41.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:41.452+0000 7f377e200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f377811a0d0 con 0x7f3778114350 2024-05-03T14:47:41.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:41.452+0000 7f3777400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f37780779d0 0x7f37781a0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:41.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:41.452+0000 7f3777400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f37780779d0 0x7f37781a0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37750/0 (socket says 172.21.15.60:37750) 2024-05-03T14:47:41.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:41.452+0000 7f3777e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3778114350 0x7f37781a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:41.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:41.452+0000 7f3777400700 1 -- 172.21.15.60:0/1580259211 learned_addr learned my addr 172.21.15.60:0/1580259211 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:47:41.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:41.452+0000 7f3776a00700 1 --2- 172.21.15.60:0/1580259211 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37780782f0 0x7f37781a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:41.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:41.453+0000 7f3777e00700 1 -- 172.21.15.60:0/1580259211 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37780782f0 msgr2=0x7f37781a08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:41.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:41.453+0000 7f3777e00700 1 --2- 172.21.15.60:0/1580259211 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37780782f0 0x7f37781a08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:41.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:41.453+0000 7f3777e00700 1 -- 172.21.15.60:0/1580259211 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f37780779d0 msgr2=0x7f37781a0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:41.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:41.453+0000 7f3777e00700 1 --2- 172.21.15.60:0/1580259211 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f37780779d0 0x7f37781a0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:41.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:41.453+0000 7f3777e00700 1 -- 172.21.15.60:0/1580259211 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37781a5370 con 0x7f3778114350 2024-05-03T14:47:41.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:41.453+0000 7f3777400700 1 --2- 172.21.15.60:0/1580259211 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f37780779d0 0x7f37781a0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:47:41.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:41.453+0000 7f3777e00700 1 --2- 172.21.15.60:0/1580259211 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3778114350 0x7f37781a4c30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f377807bcc0 tx=0x7f377000ebb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:41.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:41.453+0000 7f376fe00700 1 -- 172.21.15.60:0/1580259211 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3770004220 con 0x7f3778114350 2024-05-03T14:47:41.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:41.453+0000 7f376fe00700 1 -- 172.21.15.60:0/1580259211 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f37700043c0 con 0x7f3778114350 2024-05-03T14:47:41.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:41.453+0000 7f377e200700 1 -- 172.21.15.60:0/1580259211 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37781a5660 con 0x7f3778114350 2024-05-03T14:47:41.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:41.454+0000 7f376fe00700 1 -- 172.21.15.60:0/1580259211 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3770003e40 con 0x7f3778114350 2024-05-03T14:47:41.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:41.454+0000 7f377e200700 1 -- 172.21.15.60:0/1580259211 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37781acee0 con 0x7f3778114350 2024-05-03T14:47:41.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:41.455+0000 7f377e200700 1 -- 172.21.15.60:0/1580259211 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f377806a190 con 0x7f3778114350 2024-05-03T14:47:41.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:41.457+0000 7f376fe00700 1 -- 172.21.15.60:0/1580259211 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f377000c760 con 0x7f3778114350 2024-05-03T14:47:41.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:41.459+0000 7f376fe00700 1 --2- 172.21.15.60:0/1580259211 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f375c0b2910 0x7f375c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:41.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:41.459+0000 7f376fe00700 1 -- 172.21.15.60:0/1580259211 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f37700e27e0 con 0x7f3778114350 2024-05-03T14:47:41.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:41.459+0000 7f3777400700 1 --2- 172.21.15.60:0/1580259211 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f375c0b2910 0x7f375c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:41.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:41.460+0000 7f3777400700 1 --2- 172.21.15.60:0/1580259211 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f375c0b2910 0x7f375c0b4dd0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3768009480 tx=0x7f376800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:41.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:41.461+0000 7f376fe00700 1 -- 172.21.15.60:0/1580259211 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f37700aa050 con 0x7f3778114350 2024-05-03T14:47:41.692 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:47:41 smithi060 ceph-mon[28873]: pgmap v427: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 11 op/s 2024-05-03T14:47:41.692 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:47:41 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:47:41.692 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:47:41 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:47:41.692 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:47:41 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:47:41.692 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:47:41 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/1355261627' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-03T14:47:41.692 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:47:41 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/4008802067' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-03T14:47:41.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:41.714+0000 7f377e200700 1 -- 172.21.15.60:0/1580259211 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f377802e510 con 0x7f3778114350 2024-05-03T14:47:41.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:41.714+0000 7f376fe00700 1 -- 172.21.15.60:0/1580259211 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f37700a9950 con 0x7f3778114350 2024-05-03T14:47:41.716 INFO:teuthology.orchestra.run.smithi060.stdout:77309411437 2024-05-03T14:47:41.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:41.717+0000 7f377e200700 1 -- 172.21.15.60:0/1580259211 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f375c0b2910 msgr2=0x7f375c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:41.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:41.717+0000 7f377e200700 1 --2- 172.21.15.60:0/1580259211 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f375c0b2910 0x7f375c0b4dd0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3768009480 tx=0x7f376800b040 comp rx=0 tx=0).stop 2024-05-03T14:47:41.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:41.717+0000 7f377e200700 1 -- 172.21.15.60:0/1580259211 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3778114350 msgr2=0x7f37781a4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:41.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:41.717+0000 7f377e200700 1 --2- 172.21.15.60:0/1580259211 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3778114350 0x7f37781a4c30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f377807bcc0 tx=0x7f377000ebb0 comp rx=0 tx=0).stop 2024-05-03T14:47:41.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:41.717+0000 7f377e200700 1 -- 172.21.15.60:0/1580259211 shutdown_connections 2024-05-03T14:47:41.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:41.718+0000 7f377e200700 1 --2- 172.21.15.60:0/1580259211 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f375c0b2910 0x7f375c0b4dd0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:41.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:41.718+0000 7f377e200700 1 --2- 172.21.15.60:0/1580259211 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3778114350 0x7f37781a4c30 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:41.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:41.718+0000 7f377e200700 1 --2- 172.21.15.60:0/1580259211 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f37780779d0 0x7f37781a0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:41.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:41.718+0000 7f377e200700 1 --2- 172.21.15.60:0/1580259211 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37780782f0 0x7f37781a08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:41.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:41.718+0000 7f377e200700 1 -- 172.21.15.60:0/1580259211 >> 172.21.15.60:0/1580259211 conn(0x7f37781011a0 msgr2=0x7f3778103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:47:41.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:41.718+0000 7f377e200700 1 -- 172.21.15.60:0/1580259211 shutdown_connections 2024-05-03T14:47:41.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:41.718+0000 7f377e200700 1 -- 172.21.15.60:0/1580259211 wait complete. 2024-05-03T14:47:41.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.569+0000 7f9c67200700 1 -- 172.21.15.60:0/1859720331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c6006bad0 msgr2=0x7f9c6010fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:41.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.569+0000 7f9c67200700 1 --2- 172.21.15.60:0/1859720331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c6006bad0 0x7f9c6010fa10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c009990 tx=0x7f9c4c009ca0 comp rx=0 tx=0).stop 2024-05-03T14:47:41.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.569+0000 7f9c67200700 1 -- 172.21.15.60:0/1859720331 shutdown_connections 2024-05-03T14:47:41.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.569+0000 7f9c67200700 1 --2- 172.21.15.60:0/1859720331 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9c6010ff50 0x7f9c60112340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:41.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.569+0000 7f9c67200700 1 --2- 172.21.15.60:0/1859720331 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c6006b1b0 0x7f9c6006b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:41.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.569+0000 7f9c67200700 1 --2- 172.21.15.60:0/1859720331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c6006bad0 0x7f9c6010fa10 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:41.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.569+0000 7f9c67200700 1 -- 172.21.15.60:0/1859720331 >> 172.21.15.60:0/1859720331 conn(0x7f9c60079350 msgr2=0x7f9c60079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:47:41.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.569+0000 7f9c67200700 1 -- 172.21.15.60:0/1859720331 shutdown_connections 2024-05-03T14:47:41.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.569+0000 7f9c67200700 1 -- 172.21.15.60:0/1859720331 wait complete. 2024-05-03T14:47:41.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.571+0000 7f9c67200700 1 Processor -- start 2024-05-03T14:47:41.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.571+0000 7f9c67200700 1 -- start start 2024-05-03T14:47:41.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.572+0000 7f9c67200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9c6006b1b0 0x7f9c60104680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:41.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.572+0000 7f9c67200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c6006bad0 0x7f9c60104bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:41.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.572+0000 7f9c67200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c6010ff50 0x7f9c60105100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:41.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.572+0000 7f9c67200700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f9c60115b40 con 0x7f9c6006bad0 2024-05-03T14:47:41.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.572+0000 7f9c67200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9c601159c0 con 0x7f9c6006b1b0 2024-05-03T14:47:41.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.572+0000 7f9c67200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9c60115cc0 con 0x7f9c6010ff50 2024-05-03T14:47:41.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.572+0000 7f9c5f400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c6006bad0 0x7f9c60104bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:41.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.572+0000 7f9c5f400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c6006bad0 0x7f9c60104bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37698/0 (socket says 172.21.15.60:37698) 2024-05-03T14:47:41.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.572+0000 7f9c5f400700 1 -- 172.21.15.60:0/1355261627 learned_addr learned my addr 172.21.15.60:0/1355261627 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:47:41.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.572+0000 7f9c5fe00700 1 --2- 172.21.15.60:0/1355261627 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9c6006b1b0 0x7f9c60104680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:41.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.573+0000 7f9c65200700 1 --2- 172.21.15.60:0/1355261627 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c6010ff50 0x7f9c60105100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:41.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.573+0000 7f9c5f400700 1 -- 172.21.15.60:0/1355261627 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c6010ff50 msgr2=0x7f9c60105100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:41.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.573+0000 7f9c5f400700 1 --2- 172.21.15.60:0/1355261627 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c6010ff50 0x7f9c60105100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:41.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.573+0000 7f9c5f400700 1 -- 172.21.15.60:0/1355261627 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9c6006b1b0 msgr2=0x7f9c60104680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:41.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.573+0000 7f9c5f400700 1 --2- 172.21.15.60:0/1355261627 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9c6006b1b0 0x7f9c60104680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:41.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.573+0000 7f9c5f400700 1 -- 172.21.15.60:0/1355261627 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c601ac620 con 0x7f9c6006bad0 2024-05-03T14:47:41.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.574+0000 7f9c5f400700 1 --2- 172.21.15.60:0/1355261627 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c6006bad0 0x7f9c60104bc0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c003b40 tx=0x7f9c4c003b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:41.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.574+0000 7f9c5cc00700 1 -- 172.21.15.60:0/1355261627 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c0050c0 con 0x7f9c6006bad0 2024-05-03T14:47:41.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.574+0000 7f9c67200700 1 -- 172.21.15.60:0/1355261627 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c601ac8b0 con 0x7f9c6006bad0 2024-05-03T14:47:41.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.574+0000 7f9c67200700 1 -- 172.21.15.60:0/1355261627 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c601acd60 con 0x7f9c6006bad0 2024-05-03T14:47:41.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.574+0000 7f9c5cc00700 1 -- 172.21.15.60:0/1355261627 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f9c4c00f460 con 0x7f9c6006bad0 2024-05-03T14:47:41.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.574+0000 7f9c5cc00700 1 -- 172.21.15.60:0/1355261627 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c017650 con 0x7f9c6006bad0 2024-05-03T14:47:41.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.577+0000 7f9c67200700 1 -- 172.21.15.60:0/1355261627 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c40005330 con 0x7f9c6006bad0 2024-05-03T14:47:41.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.579+0000 7f9c5cc00700 1 -- 172.21.15.60:0/1355261627 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f9c4c005260 con 0x7f9c6006bad0 2024-05-03T14:47:41.753 INFO:teuthology.orchestra.run.smithi060.stdout:201863462989 2024-05-03T14:47:41.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.581+0000 7f9c5cc00700 1 --2- 172.21.15.60:0/1355261627 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f9c440b2990 0x7f9c440b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:41.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.582+0000 7f9c5cc00700 1 -- 172.21.15.60:0/1355261627 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f9c4c0e3530 con 0x7f9c6006bad0 2024-05-03T14:47:41.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.582+0000 7f9c5fe00700 1 --2- 172.21.15.60:0/1355261627 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f9c440b2990 0x7f9c440b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:41.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.584+0000 7f9c5fe00700 1 --2- 172.21.15.60:0/1355261627 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f9c440b2990 0x7f9c440b4e50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9c50005950 tx=0x7f9c5000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:41.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.587+0000 7f9c5cc00700 1 -- 172.21.15.60:0/1355261627 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f9c4c013070 con 0x7f9c6006bad0 2024-05-03T14:47:41.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.853+0000 7f9c67200700 1 -- 172.21.15.60:0/1355261627 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f9c400058f0 con 0x7f9c6006bad0 2024-05-03T14:47:41.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.854+0000 7f9c5cc00700 1 -- 172.21.15.60:0/1355261627 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f9c4c0aada0 con 0x7f9c6006bad0 2024-05-03T14:47:41.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.857+0000 7f9c67200700 1 -- 172.21.15.60:0/1355261627 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f9c440b2990 msgr2=0x7f9c440b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:41.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.857+0000 7f9c67200700 1 --2- 172.21.15.60:0/1355261627 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f9c440b2990 0x7f9c440b4e50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9c50005950 tx=0x7f9c5000b410 comp rx=0 tx=0).stop 2024-05-03T14:47:41.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.857+0000 7f9c67200700 1 -- 172.21.15.60:0/1355261627 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c6006bad0 msgr2=0x7f9c60104bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:41.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.857+0000 7f9c67200700 1 --2- 172.21.15.60:0/1355261627 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c6006bad0 0x7f9c60104bc0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c003b40 tx=0x7f9c4c003b70 comp rx=0 tx=0).stop 2024-05-03T14:47:41.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.857+0000 7f9c67200700 1 -- 172.21.15.60:0/1355261627 shutdown_connections 2024-05-03T14:47:41.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.857+0000 7f9c67200700 1 --2- 172.21.15.60:0/1355261627 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f9c440b2990 0x7f9c440b4e50 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:41.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.857+0000 7f9c67200700 1 --2- 172.21.15.60:0/1355261627 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c6010ff50 0x7f9c60105100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:41.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.857+0000 7f9c67200700 1 --2- 172.21.15.60:0/1355261627 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9c6006b1b0 0x7f9c60104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:41.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.857+0000 7f9c67200700 1 --2- 172.21.15.60:0/1355261627 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c6006bad0 0x7f9c60104bc0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:41.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.857+0000 7f9c67200700 1 -- 172.21.15.60:0/1355261627 >> 172.21.15.60:0/1355261627 conn(0x7f9c60079350 msgr2=0x7f9c60111360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:47:41.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.858+0000 7f9c67200700 1 -- 172.21.15.60:0/1355261627 shutdown_connections 2024-05-03T14:47:41.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.858+0000 7f9c67200700 1 -- 172.21.15.60:0/1355261627 wait complete. 2024-05-03T14:47:41.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:47:41 smithi090 ceph-mon[33568]: pgmap v427: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 11 op/s 2024-05-03T14:47:41.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:47:41 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:47:41.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:47:41 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:47:41.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:47:41 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:47:41.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:47:41 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/1355261627' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-03T14:47:41.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:47:41 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/4008802067' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-03T14:47:41.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:47:41 smithi149 ceph-mon[33332]: pgmap v427: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 11 op/s 2024-05-03T14:47:41.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:47:41 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:47:41.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:47:41 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:47:41.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:47:41 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:47:41.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:47:41 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/1355261627' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-03T14:47:41.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:47:41 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/4008802067' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-03T14:47:42.441 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070526 got 253403070529 for osd.9 2024-05-03T14:47:42.441 DEBUG:teuthology.parallel:result is None 2024-05-03T14:47:42.824 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:47:42 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/1580259211' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-03T14:47:42.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:47:42 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/1580259211' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-03T14:47:42.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.690+0000 7f5581a00700 1 -- 172.21.15.60:0/756108928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f557c114670 msgr2=0x7f557c116a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:42.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.690+0000 7f5581a00700 1 --2- 172.21.15.60:0/756108928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f557c114670 0x7f557c116a60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f557400b0a0 tx=0x7f557400b3b0 comp rx=0 tx=0).stop 2024-05-03T14:47:42.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.690+0000 7f5581a00700 1 -- 172.21.15.60:0/756108928 shutdown_connections 2024-05-03T14:47:42.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.690+0000 7f5581a00700 1 --2- 172.21.15.60:0/756108928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f557c114670 0x7f557c116a60 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:42.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.690+0000 7f5581a00700 1 --2- 172.21.15.60:0/756108928 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f557c0756d0 0x7f557c075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:42.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.690+0000 7f5581a00700 1 --2- 172.21.15.60:0/756108928 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f557c075ff0 0x7f557c114130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:42.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.690+0000 7f5581a00700 1 -- 172.21.15.60:0/756108928 >> 172.21.15.60:0/756108928 conn(0x7f557c07a600 msgr2=0x7f557c07aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:47:42.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.691+0000 7f5581a00700 1 -- 172.21.15.60:0/756108928 shutdown_connections 2024-05-03T14:47:42.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.691+0000 7f5581a00700 1 -- 172.21.15.60:0/756108928 wait complete. 2024-05-03T14:47:42.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.692+0000 7f5581a00700 1 Processor -- start 2024-05-03T14:47:42.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.692+0000 7f5581a00700 1 -- start start 2024-05-03T14:47:42.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.692+0000 7f5581a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f557c0756d0 0x7f557c1a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:42.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.692+0000 7f5581a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f557c075ff0 0x7f557c1a0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:42.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.692+0000 7f5581a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f557c114670 0x7f557c1a4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:42.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.693+0000 7f5581a00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f557c119f40 con 0x7f557c075ff0 2024-05-03T14:47:42.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.693+0000 7f5581a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f557c119dc0 con 0x7f557c114670 2024-05-03T14:47:42.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.693+0000 7f5581a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f557c11a0c0 con 0x7f557c0756d0 2024-05-03T14:47:42.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.693+0000 7f557a000700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f557c075ff0 0x7f557c1a0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:42.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.693+0000 7f557aa00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f557c0756d0 0x7f557c1a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:42.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.693+0000 7f557a000700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f557c075ff0 0x7f557c1a0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37728/0 (socket says 172.21.15.60:37728) 2024-05-03T14:47:42.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.693+0000 7f557a000700 1 -- 172.21.15.60:0/4008802067 learned_addr learned my addr 172.21.15.60:0/4008802067 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:47:42.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.693+0000 7f557b400700 1 --2- 172.21.15.60:0/4008802067 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f557c114670 0x7f557c1a4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:42.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.693+0000 7f557aa00700 1 -- 172.21.15.60:0/4008802067 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f557c114670 msgr2=0x7f557c1a4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:42.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.693+0000 7f557aa00700 1 --2- 172.21.15.60:0/4008802067 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f557c114670 0x7f557c1a4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:42.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.693+0000 7f557aa00700 1 -- 172.21.15.60:0/4008802067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f557c075ff0 msgr2=0x7f557c1a0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:42.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.693+0000 7f557aa00700 1 --2- 172.21.15.60:0/4008802067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f557c075ff0 0x7f557c1a0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:42.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.693+0000 7f557aa00700 1 -- 172.21.15.60:0/4008802067 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f557c1a5320 con 0x7f557c0756d0 2024-05-03T14:47:42.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.694+0000 7f557aa00700 1 --2- 172.21.15.60:0/4008802067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f557c0756d0 0x7f557c1a0310 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f556c00e9f0 tx=0x7f556c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:42.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.694+0000 7f5573400700 1 -- 172.21.15.60:0/4008802067 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f556c00c9c0 con 0x7f557c0756d0 2024-05-03T14:47:42.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.694+0000 7f5573400700 1 -- 172.21.15.60:0/4008802067 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f556c00cb60 con 0x7f557c0756d0 2024-05-03T14:47:42.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.694+0000 7f5573400700 1 -- 172.21.15.60:0/4008802067 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f556c018680 con 0x7f557c0756d0 2024-05-03T14:47:42.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.694+0000 7f5581a00700 1 -- 172.21.15.60:0/4008802067 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f557c1a5610 con 0x7f557c0756d0 2024-05-03T14:47:42.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.694+0000 7f5581a00700 1 -- 172.21.15.60:0/4008802067 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f557c1ace90 con 0x7f557c0756d0 2024-05-03T14:47:42.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.696+0000 7f5581a00700 1 -- 172.21.15.60:0/4008802067 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f557c06a190 con 0x7f557c0756d0 2024-05-03T14:47:42.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.699+0000 7f5573400700 1 -- 172.21.15.60:0/4008802067 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f556c00cd10 con 0x7f557c0756d0 2024-05-03T14:47:42.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.701+0000 7f5573400700 1 --2- 172.21.15.60:0/4008802067 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f55600b2990 0x7f55600b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:42.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.701+0000 7f5573400700 1 -- 172.21.15.60:0/4008802067 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f556c0e2d10 con 0x7f557c0756d0 2024-05-03T14:47:42.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.702+0000 7f557a000700 1 --2- 172.21.15.60:0/4008802067 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f55600b2990 0x7f55600b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:42.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.702+0000 7f5573400700 1 -- 172.21.15.60:0/4008802067 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f556c0aa4f0 con 0x7f557c0756d0 2024-05-03T14:47:42.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.703+0000 7f557a000700 1 --2- 172.21.15.60:0/4008802067 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f55600b2990 0x7f55600b4e50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f557c116590 tx=0x7f5568009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:42.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.969+0000 7f5581a00700 1 -- 172.21.15.60:0/4008802067 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f557c1ad0c0 con 0x7f557c0756d0 2024-05-03T14:47:42.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.970+0000 7f5573400700 1 -- 172.21.15.60:0/4008802067 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f556c0a9df0 con 0x7f557c0756d0 2024-05-03T14:47:42.881 INFO:teuthology.orchestra.run.smithi060.stdout:133143986272 2024-05-03T14:47:42.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.973+0000 7f5581a00700 1 -- 172.21.15.60:0/4008802067 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f55600b2990 msgr2=0x7f55600b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:42.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.973+0000 7f5581a00700 1 --2- 172.21.15.60:0/4008802067 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f55600b2990 0x7f55600b4e50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f557c116590 tx=0x7f5568009260 comp rx=0 tx=0).stop 2024-05-03T14:47:42.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.973+0000 7f5581a00700 1 -- 172.21.15.60:0/4008802067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f557c0756d0 msgr2=0x7f557c1a0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:42.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.973+0000 7f5581a00700 1 --2- 172.21.15.60:0/4008802067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f557c0756d0 0x7f557c1a0310 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f556c00e9f0 tx=0x7f556c00ed00 comp rx=0 tx=0).stop 2024-05-03T14:47:42.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.973+0000 7f5581a00700 1 -- 172.21.15.60:0/4008802067 shutdown_connections 2024-05-03T14:47:42.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.973+0000 7f5581a00700 1 --2- 172.21.15.60:0/4008802067 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f55600b2990 0x7f55600b4e50 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:42.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.973+0000 7f5581a00700 1 --2- 172.21.15.60:0/4008802067 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f557c114670 0x7f557c1a4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:42.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.973+0000 7f5581a00700 1 --2- 172.21.15.60:0/4008802067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f557c0756d0 0x7f557c1a0310 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:42.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.973+0000 7f5581a00700 1 --2- 172.21.15.60:0/4008802067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f557c075ff0 0x7f557c1a0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:42.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.973+0000 7f5581a00700 1 -- 172.21.15.60:0/4008802067 >> 172.21.15.60:0/4008802067 conn(0x7f557c07a600 msgr2=0x7f557c113360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:47:42.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.974+0000 7f5581a00700 1 -- 172.21.15.60:0/4008802067 shutdown_connections 2024-05-03T14:47:42.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:40.974+0000 7f5581a00700 1 -- 172.21.15.60:0/4008802067 wait complete. 2024-05-03T14:47:42.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:47:42 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/1580259211' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-03T14:47:43.698 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:47:43 smithi060 ceph-mon[28873]: pgmap v428: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-03T14:47:43.698 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:47:43 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/3031288317' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-03T14:47:43.698 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:47:43 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/3175536300' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-03T14:47:43.771 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986264 got 133143986272 for osd.4 2024-05-03T14:47:43.771 DEBUG:teuthology.parallel:result is None 2024-05-03T14:47:43.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.822+0000 7f5f39800700 1 -- 172.21.15.60:0/2017127057 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f34108850 msgr2=0x7f5f341103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:43.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.822+0000 7f5f39800700 1 --2- 172.21.15.60:0/2017127057 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f34108850 0x7f5f341103d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c009f90 tx=0x7f5f2c00b330 comp rx=0 tx=0).stop 2024-05-03T14:47:43.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.822+0000 7f5f39800700 1 -- 172.21.15.60:0/2017127057 shutdown_connections 2024-05-03T14:47:43.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.822+0000 7f5f39800700 1 --2- 172.21.15.60:0/2017127057 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f34108850 0x7f5f341103d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:43.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.822+0000 7f5f39800700 1 --2- 172.21.15.60:0/2017127057 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f34103750 0x7f5f34103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:43.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.822+0000 7f5f39800700 1 --2- 172.21.15.60:0/2017127057 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f34104100 0x7f5f34108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:43.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.822+0000 7f5f39800700 1 -- 172.21.15.60:0/2017127057 >> 172.21.15.60:0/2017127057 conn(0x7f5f3407a600 msgr2=0x7f5f3407aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:47:43.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.822+0000 7f5f39800700 1 -- 172.21.15.60:0/2017127057 shutdown_connections 2024-05-03T14:47:43.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.822+0000 7f5f39800700 1 -- 172.21.15.60:0/2017127057 wait complete. 2024-05-03T14:47:43.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.823+0000 7f5f39800700 1 Processor -- start 2024-05-03T14:47:43.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.823+0000 7f5f39800700 1 -- start start 2024-05-03T14:47:43.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.824+0000 7f5f39800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f34103750 0x7f5f3419be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:43.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.824+0000 7f5f39800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f34104100 0x7f5f3419c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:43.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.824+0000 7f5f39800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f34108850 0x7f5f341a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:43.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.824+0000 7f5f39800700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5f34113900 con 0x7f5f34108850 2024-05-03T14:47:43.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.824+0000 7f5f39800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5f34113780 con 0x7f5f34104100 2024-05-03T14:47:43.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.824+0000 7f5f39800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5f34113a80 con 0x7f5f34103750 2024-05-03T14:47:43.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.824+0000 7f5f32a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f34103750 0x7f5f3419be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:43.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.824+0000 7f5f32a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f34103750 0x7f5f3419be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.60:38616/0 (socket says 172.21.15.60:38616) 2024-05-03T14:47:43.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.824+0000 7f5f32a00700 1 -- 172.21.15.60:0/3175536300 learned_addr learned my addr 172.21.15.60:0/3175536300 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:47:43.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.824+0000 7f5f32000700 1 --2- 172.21.15.60:0/3175536300 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f34104100 0x7f5f3419c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:43.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.824+0000 7f5f33400700 1 --2- 172.21.15.60:0/3175536300 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f34108850 0x7f5f341a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:43.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.824+0000 7f5f32a00700 1 -- 172.21.15.60:0/3175536300 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f34104100 msgr2=0x7f5f3419c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:43.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.824+0000 7f5f32a00700 1 --2- 172.21.15.60:0/3175536300 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f34104100 0x7f5f3419c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:43.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.824+0000 7f5f32a00700 1 -- 172.21.15.60:0/3175536300 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f34108850 msgr2=0x7f5f341a0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:43.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.824+0000 7f5f32a00700 1 --2- 172.21.15.60:0/3175536300 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f34108850 0x7f5f341a0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:43.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.825+0000 7f5f32a00700 1 -- 172.21.15.60:0/3175536300 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f341a0e80 con 0x7f5f34103750 2024-05-03T14:47:43.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.825+0000 7f5f33400700 1 --2- 172.21.15.60:0/3175536300 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f34108850 0x7f5f341a0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T14:47:43.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.825+0000 7f5f32a00700 1 --2- 172.21.15.60:0/3175536300 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f34103750 0x7f5f3419be70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5f2400ec10 tx=0x7f5f2400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:43.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.825+0000 7f5f2b400700 1 -- 172.21.15.60:0/3175536300 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2400cd40 con 0x7f5f34103750 2024-05-03T14:47:43.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.825+0000 7f5f39800700 1 -- 172.21.15.60:0/3175536300 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f341a1170 con 0x7f5f34103750 2024-05-03T14:47:43.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.825+0000 7f5f2b400700 1 -- 172.21.15.60:0/3175536300 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f5f24004520 con 0x7f5f34103750 2024-05-03T14:47:43.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.825+0000 7f5f2b400700 1 -- 172.21.15.60:0/3175536300 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f24010690 con 0x7f5f34103750 2024-05-03T14:47:43.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.825+0000 7f5f39800700 1 -- 172.21.15.60:0/3175536300 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f341a89f0 con 0x7f5f34103750 2024-05-03T14:47:43.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.826+0000 7f5f39800700 1 -- 172.21.15.60:0/3175536300 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f3406a190 con 0x7f5f34103750 2024-05-03T14:47:43.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.829+0000 7f5f2b400700 1 -- 172.21.15.60:0/3175536300 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f5f24003fe0 con 0x7f5f34103750 2024-05-03T14:47:43.793 INFO:teuthology.orchestra.run.smithi060.stdout:107374182501 2024-05-03T14:47:43.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.830+0000 7f5f2b400700 1 --2- 172.21.15.60:0/3175536300 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f5f180b2990 0x7f5f180b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:43.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.830+0000 7f5f2b400700 1 -- 172.21.15.60:0/3175536300 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f5f240e2af0 con 0x7f5f34103750 2024-05-03T14:47:43.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.830+0000 7f5f32000700 1 --2- 172.21.15.60:0/3175536300 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f5f180b2990 0x7f5f180b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:43.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.831+0000 7f5f32000700 1 --2- 172.21.15.60:0/3175536300 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f5f180b2990 0x7f5f180b4e50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5f34109e60 tx=0x7f5f20009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:43.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.832+0000 7f5f2b400700 1 -- 172.21.15.60:0/3175536300 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f5f240aa360 con 0x7f5f34103750 2024-05-03T14:47:43.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:43.076+0000 7f5f39800700 1 -- 172.21.15.60:0/3175536300 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f5f3419d1d0 con 0x7f5f34103750 2024-05-03T14:47:43.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:43.076+0000 7f5f2b400700 1 -- 172.21.15.60:0/3175536300 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f5f240a9c60 con 0x7f5f34103750 2024-05-03T14:47:43.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:43.080+0000 7f5f39800700 1 -- 172.21.15.60:0/3175536300 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f5f180b2990 msgr2=0x7f5f180b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:43.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:43.080+0000 7f5f39800700 1 --2- 172.21.15.60:0/3175536300 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f5f180b2990 0x7f5f180b4e50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5f34109e60 tx=0x7f5f20009260 comp rx=0 tx=0).stop 2024-05-03T14:47:43.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:43.080+0000 7f5f39800700 1 -- 172.21.15.60:0/3175536300 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f34103750 msgr2=0x7f5f3419be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:43.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:43.080+0000 7f5f39800700 1 --2- 172.21.15.60:0/3175536300 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f34103750 0x7f5f3419be70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5f2400ec10 tx=0x7f5f2400ef20 comp rx=0 tx=0).stop 2024-05-03T14:47:43.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:43.080+0000 7f5f39800700 1 -- 172.21.15.60:0/3175536300 shutdown_connections 2024-05-03T14:47:43.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:43.080+0000 7f5f39800700 1 --2- 172.21.15.60:0/3175536300 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f5f180b2990 0x7f5f180b4e50 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:43.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:43.080+0000 7f5f39800700 1 --2- 172.21.15.60:0/3175536300 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f34108850 0x7f5f341a0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:43.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:43.080+0000 7f5f39800700 1 --2- 172.21.15.60:0/3175536300 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f34103750 0x7f5f3419be70 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:43.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:43.080+0000 7f5f39800700 1 --2- 172.21.15.60:0/3175536300 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f34104100 0x7f5f3419c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:43.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:43.080+0000 7f5f39800700 1 -- 172.21.15.60:0/3175536300 >> 172.21.15.60:0/3175536300 conn(0x7f5f3407a600 msgr2=0x7f5f341016a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:47:43.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:43.080+0000 7f5f39800700 1 -- 172.21.15.60:0/3175536300 shutdown_connections 2024-05-03T14:47:43.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:43.080+0000 7f5f39800700 1 -- 172.21.15.60:0/3175536300 wait complete. 2024-05-03T14:47:43.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:47:43 smithi090 ceph-mon[33568]: pgmap v428: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-03T14:47:43.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:47:43 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/3031288317' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-03T14:47:43.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:47:43 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/3175536300' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-03T14:47:43.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:47:43 smithi149 ceph-mon[33332]: pgmap v428: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-03T14:47:43.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:47:43 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/3031288317' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-03T14:47:43.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:47:43 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/3175536300' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-03T14:47:44.149 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462985 got 201863462989 for osd.7 2024-05-03T14:47:44.149 DEBUG:teuthology.parallel:result is None 2024-05-03T14:47:44.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.453+0000 7f9eae600700 1 -- 172.21.15.60:0/1669914525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ea80756d0 msgr2=0x7f9ea8075ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:44.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.453+0000 7f9eae600700 1 --2- 172.21.15.60:0/1669914525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ea80756d0 0x7f9ea8075ab0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9e980098f0 tx=0x7f9e98009c00 comp rx=0 tx=0).stop 2024-05-03T14:47:44.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.453+0000 7f9eae600700 1 -- 172.21.15.60:0/1669914525 shutdown_connections 2024-05-03T14:47:44.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.453+0000 7f9eae600700 1 --2- 172.21.15.60:0/1669914525 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ea810ff00 0x7f9ea81122f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:44.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.453+0000 7f9eae600700 1 --2- 172.21.15.60:0/1669914525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ea80756d0 0x7f9ea8075ab0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:44.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.453+0000 7f9eae600700 1 --2- 172.21.15.60:0/1669914525 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ea8075ff0 0x7f9ea810f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:44.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.453+0000 7f9eae600700 1 -- 172.21.15.60:0/1669914525 >> 172.21.15.60:0/1669914525 conn(0x7f9ea807a580 msgr2=0x7f9ea807a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:47:44.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.453+0000 7f9eae600700 1 -- 172.21.15.60:0/1669914525 shutdown_connections 2024-05-03T14:47:44.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.453+0000 7f9eae600700 1 -- 172.21.15.60:0/1669914525 wait complete. 2024-05-03T14:47:44.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.455+0000 7f9eae600700 1 Processor -- start 2024-05-03T14:47:44.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.455+0000 7f9eae600700 1 -- start start 2024-05-03T14:47:44.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.455+0000 7f9eae600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ea80756d0 0x7f9ea819bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:44.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.455+0000 7f9eae600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ea8075ff0 0x7f9ea819c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:44.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.455+0000 7f9eae600700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ea810ff00 0x7f9ea81a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:44.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.455+0000 7f9eae600700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea8115b60 con 0x7f9ea810ff00 2024-05-03T14:47:44.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.455+0000 7f9eae600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea81159e0 con 0x7f9ea8075ff0 2024-05-03T14:47:44.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.455+0000 7f9eae600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea8115ce0 con 0x7f9ea80756d0 2024-05-03T14:47:44.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.456+0000 7f9ea7e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ea810ff00 0x7f9ea81a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:44.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.456+0000 7f9ea7e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ea810ff00 0x7f9ea81a0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53334/0 (socket says 172.21.15.60:53334) 2024-05-03T14:47:44.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.456+0000 7f9ea6a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ea8075ff0 0x7f9ea819c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:44.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.456+0000 7f9ea7e00700 1 -- 172.21.15.60:0/3031288317 learned_addr learned my addr 172.21.15.60:0/3031288317 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:47:44.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.456+0000 7f9ea7400700 1 --2- 172.21.15.60:0/3031288317 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ea80756d0 0x7f9ea819bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:44.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.456+0000 7f9ea6a00700 1 -- 172.21.15.60:0/3031288317 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ea80756d0 msgr2=0x7f9ea819bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:44.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.456+0000 7f9ea6a00700 1 --2- 172.21.15.60:0/3031288317 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ea80756d0 0x7f9ea819bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:44.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.456+0000 7f9ea6a00700 1 -- 172.21.15.60:0/3031288317 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ea810ff00 msgr2=0x7f9ea81a0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:44.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.456+0000 7f9ea6a00700 1 --2- 172.21.15.60:0/3031288317 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ea810ff00 0x7f9ea81a0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:44.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.456+0000 7f9ea6a00700 1 -- 172.21.15.60:0/3031288317 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ea81a0fc0 con 0x7f9ea8075ff0 2024-05-03T14:47:44.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.456+0000 7f9ea7e00700 1 --2- 172.21.15.60:0/3031288317 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ea810ff00 0x7f9ea81a0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:47:44.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.456+0000 7f9ea6a00700 1 --2- 172.21.15.60:0/3031288317 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ea8075ff0 0x7f9ea819c4f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9e9400e9c0 tx=0x7f9e9400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:44.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.457+0000 7f9e9fe00700 1 -- 172.21.15.60:0/3031288317 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9400c9c0 con 0x7f9ea8075ff0 2024-05-03T14:47:44.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.457+0000 7f9eae600700 1 -- 172.21.15.60:0/3031288317 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ea81a12b0 con 0x7f9ea8075ff0 2024-05-03T14:47:44.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.457+0000 7f9e9fe00700 1 -- 172.21.15.60:0/3031288317 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f9e9400cb60 con 0x7f9ea8075ff0 2024-05-03T14:47:44.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.457+0000 7f9e9fe00700 1 -- 172.21.15.60:0/3031288317 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e94018680 con 0x7f9ea8075ff0 2024-05-03T14:47:44.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.457+0000 7f9eae600700 1 -- 172.21.15.60:0/3031288317 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ea81a8b30 con 0x7f9ea8075ff0 2024-05-03T14:47:44.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.458+0000 7f9eae600700 1 -- 172.21.15.60:0/3031288317 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ea806a190 con 0x7f9ea8075ff0 2024-05-03T14:47:44.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.461+0000 7f9e9fe00700 1 -- 172.21.15.60:0/3031288317 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f9e9400cd10 con 0x7f9ea8075ff0 2024-05-03T14:47:44.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.463+0000 7f9e9fe00700 1 --2- 172.21.15.60:0/3031288317 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f9e8c0b2910 0x7f9e8c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:44.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.463+0000 7f9e9fe00700 1 -- 172.21.15.60:0/3031288317 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f9e940e29f0 con 0x7f9ea8075ff0 2024-05-03T14:47:44.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.463+0000 7f9ea7400700 1 --2- 172.21.15.60:0/3031288317 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f9e8c0b2910 0x7f9e8c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:44.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.464+0000 7f9e9fe00700 1 -- 172.21.15.60:0/3031288317 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f9e940aa260 con 0x7f9ea8075ff0 2024-05-03T14:47:44.514 INFO:teuthology.orchestra.run.smithi060.stdout:55834574960 2024-05-03T14:47:44.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.464+0000 7f9ea7400700 1 --2- 172.21.15.60:0/3031288317 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f9e8c0b2910 0x7f9e8c0b4dd0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9e980098c0 tx=0x7f9e98019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:44.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.715+0000 7f9eae600700 1 -- 172.21.15.60:0/3031288317 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f9ea81a8d60 con 0x7f9ea8075ff0 2024-05-03T14:47:44.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.715+0000 7f9e9fe00700 1 -- 172.21.15.60:0/3031288317 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9e940b2250 con 0x7f9ea8075ff0 2024-05-03T14:47:44.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.718+0000 7f9eae600700 1 -- 172.21.15.60:0/3031288317 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f9e8c0b2910 msgr2=0x7f9e8c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:44.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.718+0000 7f9eae600700 1 --2- 172.21.15.60:0/3031288317 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f9e8c0b2910 0x7f9e8c0b4dd0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9e980098c0 tx=0x7f9e98019040 comp rx=0 tx=0).stop 2024-05-03T14:47:44.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.718+0000 7f9eae600700 1 -- 172.21.15.60:0/3031288317 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ea8075ff0 msgr2=0x7f9ea819c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:44.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.718+0000 7f9eae600700 1 --2- 172.21.15.60:0/3031288317 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ea8075ff0 0x7f9ea819c4f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9e9400e9c0 tx=0x7f9e9400ecd0 comp rx=0 tx=0).stop 2024-05-03T14:47:44.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.719+0000 7f9eae600700 1 -- 172.21.15.60:0/3031288317 shutdown_connections 2024-05-03T14:47:44.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.719+0000 7f9eae600700 1 --2- 172.21.15.60:0/3031288317 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f9e8c0b2910 0x7f9e8c0b4dd0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:44.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.719+0000 7f9eae600700 1 --2- 172.21.15.60:0/3031288317 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ea810ff00 0x7f9ea81a0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:44.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.719+0000 7f9eae600700 1 --2- 172.21.15.60:0/3031288317 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ea80756d0 0x7f9ea819bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:44.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.719+0000 7f9eae600700 1 --2- 172.21.15.60:0/3031288317 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ea8075ff0 0x7f9ea819c4f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:44.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.719+0000 7f9eae600700 1 -- 172.21.15.60:0/3031288317 >> 172.21.15.60:0/3031288317 conn(0x7f9ea807a580 msgr2=0x7f9ea8101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:47:44.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.719+0000 7f9eae600700 1 -- 172.21.15.60:0/3031288317 shutdown_connections 2024-05-03T14:47:44.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:42.719+0000 7f9eae600700 1 -- 172.21.15.60:0/3031288317 wait complete. 2024-05-03T14:47:45.438 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411430 got 77309411437 for osd.2 2024-05-03T14:47:45.438 DEBUG:teuthology.parallel:result is None 2024-05-03T14:47:45.763 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:47:45 smithi060 ceph-mon[28873]: pgmap v429: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-03T14:47:45.794 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:47:45.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:47:45 smithi090 ceph-mon[33568]: pgmap v429: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-03T14:47:45.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:47:45 smithi149 ceph-mon[33332]: pgmap v429: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-03T14:47:46.231 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182494 got 107374182501 for osd.3 2024-05-03T14:47:46.231 DEBUG:teuthology.parallel:result is None 2024-05-03T14:47:46.665 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574955 got 55834574960 for osd.1 2024-05-03T14:47:46.665 DEBUG:teuthology.parallel:result is None 2024-05-03T14:47:47.732 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:47:47 smithi060 ceph-mon[28873]: pgmap v430: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 11 op/s 2024-05-03T14:47:47.802 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822739 got 154618822746 for osd.5 2024-05-03T14:47:47.803 DEBUG:teuthology.parallel:result is None 2024-05-03T14:47:47.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:47:47 smithi090 ceph-mon[33568]: pgmap v430: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 11 op/s 2024-05-03T14:47:47.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:47:47 smithi149 ceph-mon[33332]: pgmap v430: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 11 op/s 2024-05-03T14:47:48.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:48.394+0000 7f840c200700 1 -- 172.21.15.60:0/4171144441 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8404106080 msgr2=0x7f8404112590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:48.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:48.394+0000 7f840c200700 1 --2- 172.21.15.60:0/4171144441 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8404106080 0x7f8404112590 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f83fc00b0a0 tx=0x7f83fc00b3b0 comp rx=0 tx=0).stop 2024-05-03T14:47:48.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:48.394+0000 7f840c200700 1 -- 172.21.15.60:0/4171144441 shutdown_connections 2024-05-03T14:47:48.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:48.394+0000 7f840c200700 1 --2- 172.21.15.60:0/4171144441 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8404106080 0x7f8404112590 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:48.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:48.394+0000 7f840c200700 1 --2- 172.21.15.60:0/4171144441 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f840410b6c0 0x7f840410baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:48.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:48.394+0000 7f840c200700 1 --2- 172.21.15.60:0/4171144441 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84041056c0 0x7f8404105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:48.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:48.394+0000 7f840c200700 1 -- 172.21.15.60:0/4171144441 >> 172.21.15.60:0/4171144441 conn(0x7f84041011b0 msgr2=0x7f84041035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:47:48.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:48.394+0000 7f840c200700 1 -- 172.21.15.60:0/4171144441 shutdown_connections 2024-05-03T14:47:48.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:48.394+0000 7f840c200700 1 -- 172.21.15.60:0/4171144441 wait complete. 2024-05-03T14:47:48.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:48.395+0000 7f840c200700 1 Processor -- start 2024-05-03T14:47:48.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:48.396+0000 7f840c200700 1 -- start start 2024-05-03T14:47:48.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:48.396+0000 7f840c200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84041056c0 0x7f840419bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:48.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:48.396+0000 7f840c200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8404106080 0x7f840419c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:48.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:48.396+0000 7f840c200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f840410b6c0 0x7f84041a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:48.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:48.396+0000 7f840c200700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f8404115b80 con 0x7f840410b6c0 2024-05-03T14:47:48.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:48.396+0000 7f840c200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8404115a00 con 0x7f8404106080 2024-05-03T14:47:48.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:48.396+0000 7f840c200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8404115d00 con 0x7f84041056c0 2024-05-03T14:47:48.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:48.397+0000 7f8409800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84041056c0 0x7f840419bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:48.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:48.397+0000 7f8409800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84041056c0 0x7f840419bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.60:38624/0 (socket says 172.21.15.60:38624) 2024-05-03T14:47:48.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:48.397+0000 7f8409800700 1 -- 172.21.15.60:0/2044765682 learned_addr learned my addr 172.21.15.60:0/2044765682 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:47:48.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:48.397+0000 7f8408e00700 1 --2- 172.21.15.60:0/2044765682 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8404106080 0x7f840419c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:48.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:48.397+0000 7f840a200700 1 --2- 172.21.15.60:0/2044765682 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f840410b6c0 0x7f84041a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:48.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:48.397+0000 7f8409800700 1 -- 172.21.15.60:0/2044765682 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8404106080 msgr2=0x7f840419c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:48.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:48.397+0000 7f8409800700 1 --2- 172.21.15.60:0/2044765682 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8404106080 0x7f840419c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:48.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:48.397+0000 7f8409800700 1 -- 172.21.15.60:0/2044765682 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f840410b6c0 msgr2=0x7f84041a0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:48.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:48.397+0000 7f8409800700 1 --2- 172.21.15.60:0/2044765682 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f840410b6c0 0x7f84041a0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:48.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:48.397+0000 7f8409800700 1 -- 172.21.15.60:0/2044765682 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84041a0ed0 con 0x7f84041056c0 2024-05-03T14:47:48.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:48.397+0000 7f8409800700 1 --2- 172.21.15.60:0/2044765682 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84041056c0 0x7f840419bf10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f83f800d980 tx=0x7f83f800dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:48.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:48.398+0000 7f8402000700 1 -- 172.21.15.60:0/2044765682 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f8016070 con 0x7f84041056c0 2024-05-03T14:47:48.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:48.398+0000 7f8402000700 1 -- 172.21.15.60:0/2044765682 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f83f8004d30 con 0x7f84041056c0 2024-05-03T14:47:48.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:48.398+0000 7f840c200700 1 -- 172.21.15.60:0/2044765682 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84041a11c0 con 0x7f84041056c0 2024-05-03T14:47:48.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:48.398+0000 7f8402000700 1 -- 172.21.15.60:0/2044765682 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f80056c0 con 0x7f84041056c0 2024-05-03T14:47:48.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:48.398+0000 7f840c200700 1 -- 172.21.15.60:0/2044765682 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84041a8a40 con 0x7f84041056c0 2024-05-03T14:47:48.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:48.399+0000 7f840c200700 1 -- 172.21.15.60:0/2044765682 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f840406a190 con 0x7f84041056c0 2024-05-03T14:47:48.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:48.402+0000 7f8402000700 1 -- 172.21.15.60:0/2044765682 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f83f8004750 con 0x7f84041056c0 2024-05-03T14:47:48.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:48.403+0000 7f8402000700 1 --2- 172.21.15.60:0/2044765682 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f83ec0b2910 0x7f83ec0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:48.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:48.404+0000 7f8402000700 1 -- 172.21.15.60:0/2044765682 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f83f80e2930 con 0x7f84041056c0 2024-05-03T14:47:48.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:48.404+0000 7f8408e00700 1 --2- 172.21.15.60:0/2044765682 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f83ec0b2910 0x7f83ec0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:48.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:48.405+0000 7f8408e00700 1 --2- 172.21.15.60:0/2044765682 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f83ec0b2910 0x7f83ec0b4dd0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f840410aa50 tx=0x7f83f0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:48.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:48.405+0000 7f8402000700 1 -- 172.21.15.60:0/2044765682 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f83f801f020 con 0x7f84041056c0 2024-05-03T14:47:48.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:48.655+0000 7f840c200700 1 -- 172.21.15.60:0/2044765682 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f84041a8c10 con 0x7f84041056c0 2024-05-03T14:47:48.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:48.656+0000 7f8402000700 1 -- 172.21.15.60:0/2044765682 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f83f80aa110 con 0x7f84041056c0 2024-05-03T14:47:48.658 INFO:teuthology.orchestra.run.smithi060.stdout:180388626517 2024-05-03T14:47:48.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:48.659+0000 7f840c200700 1 -- 172.21.15.60:0/2044765682 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f83ec0b2910 msgr2=0x7f83ec0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:48.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:48.659+0000 7f840c200700 1 --2- 172.21.15.60:0/2044765682 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f83ec0b2910 0x7f83ec0b4dd0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f840410aa50 tx=0x7f83f0009260 comp rx=0 tx=0).stop 2024-05-03T14:47:48.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:48.659+0000 7f840c200700 1 -- 172.21.15.60:0/2044765682 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84041056c0 msgr2=0x7f840419bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:48.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:48.659+0000 7f840c200700 1 --2- 172.21.15.60:0/2044765682 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84041056c0 0x7f840419bf10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f83f800d980 tx=0x7f83f800dc90 comp rx=0 tx=0).stop 2024-05-03T14:47:48.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:48.660+0000 7f840c200700 1 -- 172.21.15.60:0/2044765682 shutdown_connections 2024-05-03T14:47:48.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:48.660+0000 7f840c200700 1 --2- 172.21.15.60:0/2044765682 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f83ec0b2910 0x7f83ec0b4dd0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:48.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:48.660+0000 7f840c200700 1 --2- 172.21.15.60:0/2044765682 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f840410b6c0 0x7f84041a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:48.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:48.660+0000 7f840c200700 1 --2- 172.21.15.60:0/2044765682 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84041056c0 0x7f840419bf10 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:48.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:48.660+0000 7f840c200700 1 --2- 172.21.15.60:0/2044765682 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8404106080 0x7f840419c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:48.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:48.660+0000 7f840c200700 1 -- 172.21.15.60:0/2044765682 >> 172.21.15.60:0/2044765682 conn(0x7f84041011b0 msgr2=0x7f8404101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:47:48.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:48.660+0000 7f840c200700 1 -- 172.21.15.60:0/2044765682 shutdown_connections 2024-05-03T14:47:48.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:48.660+0000 7f840c200700 1 -- 172.21.15.60:0/2044765682 wait complete. 2024-05-03T14:47:49.510 INFO:tasks.cephadm.ceph_manager.ceph:need seq 180388626512 got 180388626517 for osd.6 2024-05-03T14:47:49.510 DEBUG:teuthology.parallel:result is None 2024-05-03T14:47:49.803 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:47:49 smithi060 ceph-mon[28873]: pgmap v431: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-03T14:47:49.803 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:47:49 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/2044765682' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-03T14:47:49.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:47:49 smithi090 ceph-mon[33568]: pgmap v431: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-03T14:47:49.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:47:49 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/2044765682' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-03T14:47:49.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:47:49 smithi149 ceph-mon[33332]: pgmap v431: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-03T14:47:49.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:47:49 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/2044765682' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-03T14:47:50.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:50.725+0000 7f6706600700 1 -- 172.21.15.60:0/1623086946 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6700109ad0 msgr2=0x7f6700109f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:50.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:50.725+0000 7f6706600700 1 --2- 172.21.15.60:0/1623086946 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6700109ad0 0x7f6700109f50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f66e80098f0 tx=0x7f66e8009c00 comp rx=0 tx=0).stop 2024-05-03T14:47:50.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:50.726+0000 7f6706600700 1 -- 172.21.15.60:0/1623086946 shutdown_connections 2024-05-03T14:47:50.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:50.726+0000 7f6706600700 1 --2- 172.21.15.60:0/1623086946 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f670010a490 0x7f67001169a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:50.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:50.726+0000 7f6706600700 1 --2- 172.21.15.60:0/1623086946 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f670010fb60 0x7f670010ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:50.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:50.726+0000 7f6706600700 1 --2- 172.21.15.60:0/1623086946 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6700109ad0 0x7f6700109f50 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:50.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:50.726+0000 7f6706600700 1 -- 172.21.15.60:0/1623086946 >> 172.21.15.60:0/1623086946 conn(0x7f6700076c70 msgr2=0x7f67001082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:47:50.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:50.726+0000 7f6706600700 1 -- 172.21.15.60:0/1623086946 shutdown_connections 2024-05-03T14:47:50.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:50.726+0000 7f6706600700 1 -- 172.21.15.60:0/1623086946 wait complete. 2024-05-03T14:47:50.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:50.727+0000 7f6706600700 1 Processor -- start 2024-05-03T14:47:50.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:50.727+0000 7f6706600700 1 -- start start 2024-05-03T14:47:50.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:50.727+0000 7f6706600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6700109ad0 0x7f67001a03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:50.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:50.727+0000 7f6706600700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f670010a490 0x7f67001a0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:50.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:50.728+0000 7f6706600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f670010fb60 0x7f67001a4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:50.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:50.728+0000 7f6706600700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f6700119f90 con 0x7f670010a490 2024-05-03T14:47:50.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:50.728+0000 7f6706600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6700119e10 con 0x7f670010fb60 2024-05-03T14:47:50.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:50.728+0000 7f6706600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f670011a110 con 0x7f6700109ad0 2024-05-03T14:47:50.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:50.728+0000 7f66f6a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f670010a490 0x7f67001a0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:50.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:50.728+0000 7f66ff400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6700109ad0 0x7f67001a03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:50.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:50.728+0000 7f66f6a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f670010a490 0x7f67001a0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53386/0 (socket says 172.21.15.60:53386) 2024-05-03T14:47:50.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:50.728+0000 7f66f6a00700 1 -- 172.21.15.60:0/3207578738 learned_addr learned my addr 172.21.15.60:0/3207578738 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:47:50.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:50.728+0000 7f66ff400700 1 --2- 172.21.15.60:0/3207578738 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6700109ad0 0x7f67001a03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.60:38648/0 (socket says 172.21.15.60:38648) 2024-05-03T14:47:50.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:50.728+0000 7f66ffe00700 1 --2- 172.21.15.60:0/3207578738 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f670010fb60 0x7f67001a4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:50.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:50.728+0000 7f66ff400700 1 -- 172.21.15.60:0/3207578738 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f670010fb60 msgr2=0x7f67001a4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:50.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:50.728+0000 7f66ff400700 1 --2- 172.21.15.60:0/3207578738 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f670010fb60 0x7f67001a4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:50.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:50.728+0000 7f66ff400700 1 -- 172.21.15.60:0/3207578738 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f670010a490 msgr2=0x7f67001a0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:50.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:50.728+0000 7f66ff400700 1 --2- 172.21.15.60:0/3207578738 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f670010a490 0x7f67001a0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:50.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:50.728+0000 7f66ff400700 1 -- 172.21.15.60:0/3207578738 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67001a53d0 con 0x7f6700109ad0 2024-05-03T14:47:50.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:50.728+0000 7f66f6a00700 1 --2- 172.21.15.60:0/3207578738 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f670010a490 0x7f67001a0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:47:50.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:50.729+0000 7f66ff400700 1 --2- 172.21.15.60:0/3207578738 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6700109ad0 0x7f67001a03c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f66f000ef10 tx=0x7f66f000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:50.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:50.729+0000 7f66fcc00700 1 -- 172.21.15.60:0/3207578738 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f000c9c0 con 0x7f6700109ad0 2024-05-03T14:47:50.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:50.729+0000 7f6706600700 1 -- 172.21.15.60:0/3207578738 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67001a5660 con 0x7f6700109ad0 2024-05-03T14:47:50.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:50.729+0000 7f66fcc00700 1 -- 172.21.15.60:0/3207578738 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f66f0019070 con 0x7f6700109ad0 2024-05-03T14:47:50.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:50.729+0000 7f66fcc00700 1 -- 172.21.15.60:0/3207578738 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f00146e0 con 0x7f6700109ad0 2024-05-03T14:47:50.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:50.729+0000 7f6706600700 1 -- 172.21.15.60:0/3207578738 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67001acf40 con 0x7f6700109ad0 2024-05-03T14:47:50.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:50.731+0000 7f6706600700 1 -- 172.21.15.60:0/3207578738 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f670006a190 con 0x7f6700109ad0 2024-05-03T14:47:50.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:50.732+0000 7f66fcc00700 1 -- 172.21.15.60:0/3207578738 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f66f0012050 con 0x7f6700109ad0 2024-05-03T14:47:50.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:50.734+0000 7f66fcc00700 1 --2- 172.21.15.60:0/3207578738 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f66e00b2910 0x7f66e00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:50.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:50.734+0000 7f66fcc00700 1 -- 172.21.15.60:0/3207578738 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f66f00e2680 con 0x7f6700109ad0 2024-05-03T14:47:50.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:50.734+0000 7f66f6a00700 1 --2- 172.21.15.60:0/3207578738 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f66e00b2910 0x7f66e00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:50.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:50.735+0000 7f66f6a00700 1 --2- 172.21.15.60:0/3207578738 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f66e00b2910 0x7f66e00b4dd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6700078b80 tx=0x7f66e8019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:50.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:50.738+0000 7f66fcc00700 1 -- 172.21.15.60:0/3207578738 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f66f0010040 con 0x7f6700109ad0 2024-05-03T14:47:50.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:50.978+0000 7f6706600700 1 -- 172.21.15.60:0/3207578738 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f670002e510 con 0x7f6700109ad0 2024-05-03T14:47:50.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:50.978+0000 7f66fcc00700 1 -- 172.21.15.60:0/3207578738 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f66f00a9e60 con 0x7f6700109ad0 2024-05-03T14:47:50.981 INFO:teuthology.orchestra.run.smithi060.stdout:34359738489 2024-05-03T14:47:50.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:50.982+0000 7f6706600700 1 -- 172.21.15.60:0/3207578738 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f66e00b2910 msgr2=0x7f66e00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:50.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:50.982+0000 7f6706600700 1 --2- 172.21.15.60:0/3207578738 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f66e00b2910 0x7f66e00b4dd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6700078b80 tx=0x7f66e8019040 comp rx=0 tx=0).stop 2024-05-03T14:47:50.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:50.982+0000 7f6706600700 1 -- 172.21.15.60:0/3207578738 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6700109ad0 msgr2=0x7f67001a03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:50.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:50.982+0000 7f6706600700 1 --2- 172.21.15.60:0/3207578738 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6700109ad0 0x7f67001a03c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f66f000ef10 tx=0x7f66f000c3a0 comp rx=0 tx=0).stop 2024-05-03T14:47:50.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:50.982+0000 7f6706600700 1 -- 172.21.15.60:0/3207578738 shutdown_connections 2024-05-03T14:47:50.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:50.982+0000 7f6706600700 1 --2- 172.21.15.60:0/3207578738 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f66e00b2910 0x7f66e00b4dd0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:50.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:50.982+0000 7f6706600700 1 --2- 172.21.15.60:0/3207578738 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f670010fb60 0x7f67001a4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:50.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:50.982+0000 7f6706600700 1 --2- 172.21.15.60:0/3207578738 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6700109ad0 0x7f67001a03c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:50.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:50.982+0000 7f6706600700 1 --2- 172.21.15.60:0/3207578738 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f670010a490 0x7f67001a0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:50.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:50.982+0000 7f6706600700 1 -- 172.21.15.60:0/3207578738 >> 172.21.15.60:0/3207578738 conn(0x7f6700076c70 msgr2=0x7f67000771b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:47:50.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:50.983+0000 7f6706600700 1 -- 172.21.15.60:0/3207578738 shutdown_connections 2024-05-03T14:47:50.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:50.983+0000 7f6706600700 1 -- 172.21.15.60:0/3207578738 wait complete. 2024-05-03T14:47:51.744 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738483 got 34359738489 for osd.0 2024-05-03T14:47:51.745 DEBUG:teuthology.parallel:result is None 2024-05-03T14:47:51.745 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-03T14:47:51.745 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-03T14:47:51.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:47:51 smithi090 ceph-mon[33568]: pgmap v432: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:47:51.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:47:51 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/3207578738' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-03T14:47:51.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:47:51 smithi149 ceph-mon[33332]: pgmap v432: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:47:51.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:47:51 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/3207578738' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-03T14:47:52.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:47:51 smithi060 ceph-mon[28873]: pgmap v432: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:47:52.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:47:51 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/3207578738' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-03T14:47:52.206 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:47:53.796 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:47:53 smithi060 ceph-mon[28873]: pgmap v433: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T14:47:53.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:47:53 smithi090 ceph-mon[33568]: pgmap v433: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T14:47:53.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:47:53 smithi149 ceph-mon[33332]: pgmap v433: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T14:47:54.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:54.310+0000 7f5069800700 1 -- 172.21.15.60:0/1891077111 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f506410b690 msgr2=0x7f506410ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:54.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:54.310+0000 7f5069800700 1 --2- 172.21.15.60:0/1891077111 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f506410b690 0x7f506410ba70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f50540098f0 tx=0x7f5054009c00 comp rx=0 tx=0).stop 2024-05-03T14:47:54.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:54.310+0000 7f5069800700 1 -- 172.21.15.60:0/1891077111 shutdown_connections 2024-05-03T14:47:54.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:54.310+0000 7f5069800700 1 --2- 172.21.15.60:0/1891077111 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5064106050 0x7f5064112560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:54.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:54.310+0000 7f5069800700 1 --2- 172.21.15.60:0/1891077111 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f506410b690 0x7f506410ba70 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:54.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:54.310+0000 7f5069800700 1 --2- 172.21.15.60:0/1891077111 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5064105690 0x7f5064105b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:54.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:54.310+0000 7f5069800700 1 -- 172.21.15.60:0/1891077111 >> 172.21.15.60:0/1891077111 conn(0x7f5064101180 msgr2=0x7f50641035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:47:54.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:54.310+0000 7f5069800700 1 -- 172.21.15.60:0/1891077111 shutdown_connections 2024-05-03T14:47:54.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:54.310+0000 7f5069800700 1 -- 172.21.15.60:0/1891077111 wait complete. 2024-05-03T14:47:54.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:54.311+0000 7f5069800700 1 Processor -- start 2024-05-03T14:47:54.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:54.311+0000 7f5069800700 1 -- start start 2024-05-03T14:47:54.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:54.312+0000 7f5069800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5064105690 0x7f506419bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:54.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:54.312+0000 7f5069800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5064106050 0x7f506419c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:54.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:54.312+0000 7f5069800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f506410b690 0x7f50641a07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:54.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:54.312+0000 7f5069800700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5064115b00 con 0x7f5064106050 2024-05-03T14:47:54.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:54.312+0000 7f5069800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5064115980 con 0x7f5064105690 2024-05-03T14:47:54.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:54.312+0000 7f5069800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5064115c80 con 0x7f506410b690 2024-05-03T14:47:54.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:54.312+0000 7f505a000700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5064106050 0x7f506419c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:54.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:54.312+0000 7f505a000700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5064106050 0x7f506419c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35626/0 (socket says 172.21.15.60:35626) 2024-05-03T14:47:54.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:54.312+0000 7f505a000700 1 -- 172.21.15.60:0/899211067 learned_addr learned my addr 172.21.15.60:0/899211067 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:47:54.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:54.312+0000 7f5062a00700 1 --2- 172.21.15.60:0/899211067 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5064105690 0x7f506419bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:54.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:54.312+0000 7f5063400700 1 --2- 172.21.15.60:0/899211067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f506410b690 0x7f50641a07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:54.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:54.313+0000 7f505a000700 1 -- 172.21.15.60:0/899211067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f506410b690 msgr2=0x7f50641a07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:54.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:54.313+0000 7f505a000700 1 --2- 172.21.15.60:0/899211067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f506410b690 0x7f50641a07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:54.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:54.313+0000 7f505a000700 1 -- 172.21.15.60:0/899211067 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5064105690 msgr2=0x7f506419bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:54.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:54.313+0000 7f505a000700 1 --2- 172.21.15.60:0/899211067 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5064105690 0x7f506419bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:54.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:54.313+0000 7f505a000700 1 -- 172.21.15.60:0/899211067 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50641a0ef0 con 0x7f5064106050 2024-05-03T14:47:54.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:54.313+0000 7f5062a00700 1 --2- 172.21.15.60:0/899211067 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5064105690 0x7f506419bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:47:54.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:54.313+0000 7f505a000700 1 --2- 172.21.15.60:0/899211067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5064106050 0x7f506419c470 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f504c00e9c0 tx=0x7f504c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:54.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:54.313+0000 7f505be00700 1 -- 172.21.15.60:0/899211067 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f504c004a30 con 0x7f5064106050 2024-05-03T14:47:54.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:54.314+0000 7f5069800700 1 -- 172.21.15.60:0/899211067 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50641a11e0 con 0x7f5064106050 2024-05-03T14:47:54.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:54.314+0000 7f505be00700 1 -- 172.21.15.60:0/899211067 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f504c00ccc0 con 0x7f5064106050 2024-05-03T14:47:54.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:54.314+0000 7f505be00700 1 -- 172.21.15.60:0/899211067 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f504c010430 con 0x7f5064106050 2024-05-03T14:47:54.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:54.314+0000 7f5069800700 1 -- 172.21.15.60:0/899211067 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50641a8a60 con 0x7f5064106050 2024-05-03T14:47:54.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:54.315+0000 7f5069800700 1 -- 172.21.15.60:0/899211067 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f506410dc20 con 0x7f5064106050 2024-05-03T14:47:54.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:54.317+0000 7f505be00700 1 -- 172.21.15.60:0/899211067 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f504c004d80 con 0x7f5064106050 2024-05-03T14:47:54.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:54.319+0000 7f505be00700 1 --2- 172.21.15.60:0/899211067 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f50480b2910 0x7f50480b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:54.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:54.319+0000 7f505be00700 1 -- 172.21.15.60:0/899211067 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f504c0e2a80 con 0x7f5064106050 2024-05-03T14:47:54.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:54.319+0000 7f5062a00700 1 --2- 172.21.15.60:0/899211067 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f50480b2910 0x7f50480b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:54.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:54.320+0000 7f5062a00700 1 --2- 172.21.15.60:0/899211067 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f50480b2910 0x7f50480b4dd0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f50540098c0 tx=0x7f5054019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:54.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:54.321+0000 7f505be00700 1 -- 172.21.15.60:0/899211067 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f504c054d70 con 0x7f5064106050 2024-05-03T14:47:54.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:54.556+0000 7f5069800700 1 -- 172.21.15.60:0/899211067 --> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f50640638e0 con 0x7f50480b2910 2024-05-03T14:47:54.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:54.573+0000 7f505be00700 1 -- 172.21.15.60:0/899211067 <== mgr.14152 v2:172.21.15.60:6800/3066142956 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340664 (secure 0 0 0) 0x7f50640638e0 con 0x7f50480b2910 2024-05-03T14:47:54.587 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:47:54.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:54.582+0000 7f5069800700 1 -- 172.21.15.60:0/899211067 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f50480b2910 msgr2=0x7f50480b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:54.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:54.582+0000 7f5069800700 1 --2- 172.21.15.60:0/899211067 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f50480b2910 0x7f50480b4dd0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f50540098c0 tx=0x7f5054019040 comp rx=0 tx=0).stop 2024-05-03T14:47:54.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:54.582+0000 7f5069800700 1 -- 172.21.15.60:0/899211067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5064106050 msgr2=0x7f506419c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:54.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:54.582+0000 7f5069800700 1 --2- 172.21.15.60:0/899211067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5064106050 0x7f506419c470 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f504c00e9c0 tx=0x7f504c00ecd0 comp rx=0 tx=0).stop 2024-05-03T14:47:54.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:54.582+0000 7f5069800700 1 -- 172.21.15.60:0/899211067 shutdown_connections 2024-05-03T14:47:54.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:54.582+0000 7f5069800700 1 --2- 172.21.15.60:0/899211067 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f50480b2910 0x7f50480b4dd0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:54.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:54.582+0000 7f5069800700 1 --2- 172.21.15.60:0/899211067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f506410b690 0x7f50641a07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:54.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:54.582+0000 7f5069800700 1 --2- 172.21.15.60:0/899211067 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5064105690 0x7f506419bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:54.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:54.582+0000 7f5069800700 1 --2- 172.21.15.60:0/899211067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5064106050 0x7f506419c470 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:54.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:54.582+0000 7f5069800700 1 -- 172.21.15.60:0/899211067 >> 172.21.15.60:0/899211067 conn(0x7f5064101180 msgr2=0x7f5064101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:47:54.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:54.583+0000 7f5069800700 1 -- 172.21.15.60:0/899211067 shutdown_connections 2024-05-03T14:47:54.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:54.583+0000 7f5069800700 1 -- 172.21.15.60:0/899211067 wait complete. 2024-05-03T14:47:54.591 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-05-03T14:47:55.367 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":434,"stamp":"2024-05-03T14:47:54.403464+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657592,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1252,"num_read_kb":723,"num_write":374,"num_write_kb":2292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":2952512,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":308,"ondisk_log_size":308,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361264,"kb_used_data":12104,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580560,"statfs":{"total":1151940427776,"available":1151570493440,"internally_reserved":0,"allocated":12394496,"data_stored":8127830,"data_compressed":496272,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19140,"internal_metadata":357414204},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":115,"num_read_kb":55,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005964"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110403+0000","last_change":"2024-05-03T14:44:31.452444+0000","last_active":"2024-05-03T14:46:06.110403+0000","last_peered":"2024-05-03T14:46:06.110403+0000","last_clean":"2024-05-03T14:46:06.110403+0000","last_became_active":"2024-05-03T14:44:31.451574+0000","last_became_peered":"2024-05-03T14:44:31.451574+0000","last_unstale":"2024-05-03T14:46:06.110403+0000","last_undegraded":"2024-05-03T14:46:06.110403+0000","last_fullsized":"2024-05-03T14:46:06.110403+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:14:31.009833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.590392+0000","last_change":"2024-05-03T14:44:31.461736+0000","last_active":"2024-05-03T14:46:05.590392+0000","last_peered":"2024-05-03T14:46:05.590392+0000","last_clean":"2024-05-03T14:46:05.590392+0000","last_became_active":"2024-05-03T14:44:31.457963+0000","last_became_peered":"2024-05-03T14:44:31.457963+0000","last_unstale":"2024-05-03T14:46:05.590392+0000","last_undegraded":"2024-05-03T14:46:05.590392+0000","last_fullsized":"2024-05-03T14:46:05.590392+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:59:16.029675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.590007+0000","last_change":"2024-05-03T14:44:31.457163+0000","last_active":"2024-05-03T14:46:05.590007+0000","last_peered":"2024-05-03T14:46:05.590007+0000","last_clean":"2024-05-03T14:46:05.590007+0000","last_became_active":"2024-05-03T14:44:31.455564+0000","last_became_peered":"2024-05-03T14:44:31.455564+0000","last_unstale":"2024-05-03T14:46:05.590007+0000","last_undegraded":"2024-05-03T14:46:05.590007+0000","last_fullsized":"2024-05-03T14:46:05.590007+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T16:14:00.761333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109879+0000","last_change":"2024-05-03T14:44:31.450623+0000","last_active":"2024-05-03T14:46:06.109879+0000","last_peered":"2024-05-03T14:46:06.109879+0000","last_clean":"2024-05-03T14:46:06.109879+0000","last_became_active":"2024-05-03T14:44:31.449944+0000","last_became_peered":"2024-05-03T14:44:31.449944+0000","last_unstale":"2024-05-03T14:46:06.109879+0000","last_undegraded":"2024-05-03T14:46:06.109879+0000","last_fullsized":"2024-05-03T14:46:06.109879+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:40:24.155342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.593956+0000","last_change":"2024-05-03T14:44:31.473624+0000","last_active":"2024-05-03T14:46:05.593956+0000","last_peered":"2024-05-03T14:46:05.593956+0000","last_clean":"2024-05-03T14:46:05.593956+0000","last_became_active":"2024-05-03T14:44:31.473339+0000","last_became_peered":"2024-05-03T14:44:31.473339+0000","last_unstale":"2024-05-03T14:46:05.593956+0000","last_undegraded":"2024-05-03T14:46:05.593956+0000","last_fullsized":"2024-05-03T14:46:05.593956+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:21:42.683546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.608139+0000","last_change":"2024-05-03T14:44:31.466037+0000","last_active":"2024-05-03T14:46:05.608139+0000","last_peered":"2024-05-03T14:46:05.608139+0000","last_clean":"2024-05-03T14:46:05.608139+0000","last_became_active":"2024-05-03T14:44:31.465202+0000","last_became_peered":"2024-05-03T14:44:31.465202+0000","last_unstale":"2024-05-03T14:46:05.608139+0000","last_undegraded":"2024-05-03T14:46:05.608139+0000","last_fullsized":"2024-05-03T14:46:05.608139+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:46:04.189133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.594286+0000","last_change":"2024-05-03T14:44:31.458708+0000","last_active":"2024-05-03T14:46:05.594286+0000","last_peered":"2024-05-03T14:46:05.594286+0000","last_clean":"2024-05-03T14:46:05.594286+0000","last_became_active":"2024-05-03T14:44:31.458292+0000","last_became_peered":"2024-05-03T14:44:31.458292+0000","last_unstale":"2024-05-03T14:46:05.594286+0000","last_undegraded":"2024-05-03T14:46:05.594286+0000","last_fullsized":"2024-05-03T14:46:05.594286+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:15:28.850661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.113349+0000","last_change":"2024-05-03T14:44:31.466051+0000","last_active":"2024-05-03T14:46:06.113349+0000","last_peered":"2024-05-03T14:46:06.113349+0000","last_clean":"2024-05-03T14:46:06.113349+0000","last_became_active":"2024-05-03T14:44:31.465620+0000","last_became_peered":"2024-05-03T14:44:31.465620+0000","last_unstale":"2024-05-03T14:46:06.113349+0000","last_undegraded":"2024-05-03T14:46:06.113349+0000","last_fullsized":"2024-05-03T14:46:06.113349+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:27:02.749377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109470+0000","last_change":"2024-05-03T14:44:31.451507+0000","last_active":"2024-05-03T14:46:06.109470+0000","last_peered":"2024-05-03T14:46:06.109470+0000","last_clean":"2024-05-03T14:46:06.109470+0000","last_became_active":"2024-05-03T14:44:31.451163+0000","last_became_peered":"2024-05-03T14:44:31.451163+0000","last_unstale":"2024-05-03T14:46:06.109470+0000","last_undegraded":"2024-05-03T14:46:06.109470+0000","last_fullsized":"2024-05-03T14:46:06.109470+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:56:50.415619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.608463+0000","last_change":"2024-05-03T14:45:05.421817+0000","last_active":"2024-05-03T14:46:05.608463+0000","last_peered":"2024-05-03T14:46:05.608463+0000","last_clean":"2024-05-03T14:46:05.608463+0000","last_became_active":"2024-05-03T14:45:05.421446+0000","last_became_peered":"2024-05-03T14:45:05.421446+0000","last_unstale":"2024-05-03T14:46:05.608463+0000","last_undegraded":"2024-05-03T14:46:05.608463+0000","last_fullsized":"2024-05-03T14:46:05.608463+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:31:36.867429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.594798+0000","last_change":"2024-05-03T14:44:31.454474+0000","last_active":"2024-05-03T14:46:05.594798+0000","last_peered":"2024-05-03T14:46:05.594798+0000","last_clean":"2024-05-03T14:46:05.594798+0000","last_became_active":"2024-05-03T14:44:31.454089+0000","last_became_peered":"2024-05-03T14:44:31.454089+0000","last_unstale":"2024-05-03T14:46:05.594798+0000","last_undegraded":"2024-05-03T14:46:05.594798+0000","last_fullsized":"2024-05-03T14:46:05.594798+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:54:59.739325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.592913+0000","last_change":"2024-05-03T14:45:05.617724+0000","last_active":"2024-05-03T14:46:05.592913+0000","last_peered":"2024-05-03T14:46:05.592913+0000","last_clean":"2024-05-03T14:46:05.592913+0000","last_became_active":"2024-05-03T14:45:05.616554+0000","last_became_peered":"2024-05-03T14:45:05.616554+0000","last_unstale":"2024-05-03T14:46:05.592913+0000","last_undegraded":"2024-05-03T14:46:05.592913+0000","last_fullsized":"2024-05-03T14:46:05.592913+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:24:20.890688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110669+0000","last_change":"2024-05-03T14:44:31.462676+0000","last_active":"2024-05-03T14:46:06.110669+0000","last_peered":"2024-05-03T14:46:06.110669+0000","last_clean":"2024-05-03T14:46:06.110669+0000","last_became_active":"2024-05-03T14:44:31.459326+0000","last_became_peered":"2024-05-03T14:44:31.459326+0000","last_unstale":"2024-05-03T14:46:06.110669+0000","last_undegraded":"2024-05-03T14:46:06.110669+0000","last_fullsized":"2024-05-03T14:46:06.110669+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:36:22.975112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.589768+0000","last_change":"2024-05-03T14:44:31.447665+0000","last_active":"2024-05-03T14:46:05.589768+0000","last_peered":"2024-05-03T14:46:05.589768+0000","last_clean":"2024-05-03T14:46:05.589768+0000","last_became_active":"2024-05-03T14:44:31.447204+0000","last_became_peered":"2024-05-03T14:44:31.447204+0000","last_unstale":"2024-05-03T14:46:05.589768+0000","last_undegraded":"2024-05-03T14:46:05.589768+0000","last_fullsized":"2024-05-03T14:46:05.589768+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:23:41.313986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.590246+0000","last_change":"2024-05-03T14:44:31.458869+0000","last_active":"2024-05-03T14:46:05.590246+0000","last_peered":"2024-05-03T14:46:05.590246+0000","last_clean":"2024-05-03T14:46:05.590246+0000","last_became_active":"2024-05-03T14:44:31.453157+0000","last_became_peered":"2024-05-03T14:44:31.453157+0000","last_unstale":"2024-05-03T14:46:05.590246+0000","last_undegraded":"2024-05-03T14:46:05.590246+0000","last_fullsized":"2024-05-03T14:46:05.590246+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:26:15.764522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.108585+0000","last_change":"2024-05-03T14:44:31.461320+0000","last_active":"2024-05-03T14:46:06.108585+0000","last_peered":"2024-05-03T14:46:06.108585+0000","last_clean":"2024-05-03T14:46:06.108585+0000","last_became_active":"2024-05-03T14:44:31.460935+0000","last_became_peered":"2024-05-03T14:44:31.460935+0000","last_unstale":"2024-05-03T14:46:06.108585+0000","last_undegraded":"2024-05-03T14:46:06.108585+0000","last_fullsized":"2024-05-03T14:46:06.108585+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:32:22.368935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110144+0000","last_change":"2024-05-03T14:44:31.452694+0000","last_active":"2024-05-03T14:46:06.110144+0000","last_peered":"2024-05-03T14:46:06.110144+0000","last_clean":"2024-05-03T14:46:06.110144+0000","last_became_active":"2024-05-03T14:44:31.452012+0000","last_became_peered":"2024-05-03T14:44:31.452012+0000","last_unstale":"2024-05-03T14:46:06.110144+0000","last_undegraded":"2024-05-03T14:46:06.110144+0000","last_fullsized":"2024-05-03T14:46:06.110144+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:59:34.721990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.591555+0000","last_change":"2024-05-03T14:44:31.461029+0000","last_active":"2024-05-03T14:46:05.591555+0000","last_peered":"2024-05-03T14:46:05.591555+0000","last_clean":"2024-05-03T14:46:05.591555+0000","last_became_active":"2024-05-03T14:44:31.460506+0000","last_became_peered":"2024-05-03T14:44:31.460506+0000","last_unstale":"2024-05-03T14:46:05.591555+0000","last_undegraded":"2024-05-03T14:46:05.591555+0000","last_fullsized":"2024-05-03T14:46:05.591555+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:09:23.903606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.591626+0000","last_change":"2024-05-03T14:44:31.467556+0000","last_active":"2024-05-03T14:46:05.591626+0000","last_peered":"2024-05-03T14:46:05.591626+0000","last_clean":"2024-05-03T14:46:05.591626+0000","last_became_active":"2024-05-03T14:44:31.467191+0000","last_became_peered":"2024-05-03T14:44:31.467191+0000","last_unstale":"2024-05-03T14:46:05.591626+0000","last_undegraded":"2024-05-03T14:46:05.591626+0000","last_fullsized":"2024-05-03T14:46:05.591626+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:33:42.980620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.591444+0000","last_change":"2024-05-03T14:45:05.618172+0000","last_active":"2024-05-03T14:46:05.591444+0000","last_peered":"2024-05-03T14:46:05.591444+0000","last_clean":"2024-05-03T14:46:05.591444+0000","last_became_active":"2024-05-03T14:45:05.617555+0000","last_became_peered":"2024-05-03T14:45:05.617555+0000","last_unstale":"2024-05-03T14:46:05.591444+0000","last_undegraded":"2024-05-03T14:46:05.591444+0000","last_fullsized":"2024-05-03T14:46:05.591444+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T16:44:40.610265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'13","reported_seq":194,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:47:51.361468+0000","last_change":"2024-05-03T14:46:06.129243+0000","last_active":"2024-05-03T14:47:51.361468+0000","last_peered":"2024-05-03T14:47:51.361468+0000","last_clean":"2024-05-03T14:47:51.361468+0000","last_became_active":"2024-05-03T14:46:06.112952+0000","last_became_peered":"2024-05-03T14:46:06.112952+0000","last_unstale":"2024-05-03T14:47:51.361468+0000","last_undegraded":"2024-05-03T14:47:51.361468+0000","last_fullsized":"2024-05-03T14:47:51.361468+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:12:22.401171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":114,"num_read_kb":114,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.113593+0000","last_change":"2024-05-03T14:44:31.467383+0000","last_active":"2024-05-03T14:46:06.113593+0000","last_peered":"2024-05-03T14:46:06.113593+0000","last_clean":"2024-05-03T14:46:06.113593+0000","last_became_active":"2024-05-03T14:44:31.466993+0000","last_became_peered":"2024-05-03T14:44:31.466993+0000","last_unstale":"2024-05-03T14:46:06.113593+0000","last_undegraded":"2024-05-03T14:46:06.113593+0000","last_fullsized":"2024-05-03T14:46:06.113593+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T16:43:50.237083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.108935+0000","last_change":"2024-05-03T14:44:44.583870+0000","last_active":"2024-05-03T14:46:06.108935+0000","last_peered":"2024-05-03T14:46:06.108935+0000","last_clean":"2024-05-03T14:46:06.108935+0000","last_became_active":"2024-05-03T14:44:44.583296+0000","last_became_peered":"2024-05-03T14:44:44.583296+0000","last_unstale":"2024-05-03T14:46:06.108935+0000","last_undegraded":"2024-05-03T14:46:06.108935+0000","last_fullsized":"2024-05-03T14:46:06.108935+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:55:54.345707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.115090+0000","last_change":"2024-05-03T14:44:27.415797+0000","last_active":"2024-05-03T14:46:06.115090+0000","last_peered":"2024-05-03T14:46:06.115090+0000","last_clean":"2024-05-03T14:46:06.115090+0000","last_became_active":"2024-05-03T14:44:27.414960+0000","last_became_peered":"2024-05-03T14:44:27.414960+0000","last_unstale":"2024-05-03T14:46:06.115090+0000","last_undegraded":"2024-05-03T14:46:06.115090+0000","last_fullsized":"2024-05-03T14:46:06.115090+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:15:54.374392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.590953+0000","last_change":"2024-05-03T14:44:31.470241+0000","last_active":"2024-05-03T14:46:05.590953+0000","last_peered":"2024-05-03T14:46:05.590953+0000","last_clean":"2024-05-03T14:46:05.590953+0000","last_became_active":"2024-05-03T14:44:31.469882+0000","last_became_peered":"2024-05-03T14:44:31.469882+0000","last_unstale":"2024-05-03T14:46:05.590953+0000","last_undegraded":"2024-05-03T14:46:05.590953+0000","last_fullsized":"2024-05-03T14:46:05.590953+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:40:43.970897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109107+0000","last_change":"2024-05-03T14:44:44.583776+0000","last_active":"2024-05-03T14:46:06.109107+0000","last_peered":"2024-05-03T14:46:06.109107+0000","last_clean":"2024-05-03T14:46:06.109107+0000","last_became_active":"2024-05-03T14:44:44.583117+0000","last_became_peered":"2024-05-03T14:44:44.583117+0000","last_unstale":"2024-05-03T14:46:06.109107+0000","last_undegraded":"2024-05-03T14:46:06.109107+0000","last_fullsized":"2024-05-03T14:46:06.109107+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:58:49.428720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.594222+0000","last_change":"2024-05-03T14:44:27.404806+0000","last_active":"2024-05-03T14:46:05.594222+0000","last_peered":"2024-05-03T14:46:05.594222+0000","last_clean":"2024-05-03T14:46:05.594222+0000","last_became_active":"2024-05-03T14:44:27.404415+0000","last_became_peered":"2024-05-03T14:44:27.404415+0000","last_unstale":"2024-05-03T14:46:05.594222+0000","last_undegraded":"2024-05-03T14:46:05.594222+0000","last_fullsized":"2024-05-03T14:46:05.594222+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:04:40.569591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.108885+0000","last_change":"2024-05-03T14:44:31.465714+0000","last_active":"2024-05-03T14:46:06.108885+0000","last_peered":"2024-05-03T14:46:06.108885+0000","last_clean":"2024-05-03T14:46:06.108885+0000","last_became_active":"2024-05-03T14:44:31.465264+0000","last_became_peered":"2024-05-03T14:44:31.465264+0000","last_unstale":"2024-05-03T14:46:06.108885+0000","last_undegraded":"2024-05-03T14:46:06.108885+0000","last_fullsized":"2024-05-03T14:46:06.108885+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:19:38.206564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109172+0000","last_change":"2024-05-03T14:44:44.583611+0000","last_active":"2024-05-03T14:46:06.109172+0000","last_peered":"2024-05-03T14:46:06.109172+0000","last_clean":"2024-05-03T14:46:06.109172+0000","last_became_active":"2024-05-03T14:44:44.583149+0000","last_became_peered":"2024-05-03T14:44:44.583149+0000","last_unstale":"2024-05-03T14:46:06.109172+0000","last_undegraded":"2024-05-03T14:46:06.109172+0000","last_fullsized":"2024-05-03T14:46:06.109172+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:25:23.550168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.594007+0000","last_change":"2024-05-03T14:44:27.405942+0000","last_active":"2024-05-03T14:46:05.594007+0000","last_peered":"2024-05-03T14:46:05.594007+0000","last_clean":"2024-05-03T14:46:05.594007+0000","last_became_active":"2024-05-03T14:44:27.405556+0000","last_became_peered":"2024-05-03T14:44:27.405556+0000","last_unstale":"2024-05-03T14:46:05.594007+0000","last_undegraded":"2024-05-03T14:46:05.594007+0000","last_fullsized":"2024-05-03T14:46:05.594007+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:16:07.798364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110336+0000","last_change":"2024-05-03T14:44:31.451421+0000","last_active":"2024-05-03T14:46:06.110336+0000","last_peered":"2024-05-03T14:46:06.110336+0000","last_clean":"2024-05-03T14:46:06.110336+0000","last_became_active":"2024-05-03T14:44:31.451072+0000","last_became_peered":"2024-05-03T14:44:31.451072+0000","last_unstale":"2024-05-03T14:46:06.110336+0000","last_undegraded":"2024-05-03T14:46:06.110336+0000","last_fullsized":"2024-05-03T14:46:06.110336+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:12:31.040518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.591429+0000","last_change":"2024-05-03T14:44:44.567926+0000","last_active":"2024-05-03T14:46:05.591429+0000","last_peered":"2024-05-03T14:46:05.591429+0000","last_clean":"2024-05-03T14:46:05.591429+0000","last_became_active":"2024-05-03T14:44:44.567494+0000","last_became_peered":"2024-05-03T14:44:44.567494+0000","last_unstale":"2024-05-03T14:46:05.591429+0000","last_undegraded":"2024-05-03T14:46:05.591429+0000","last_fullsized":"2024-05-03T14:46:05.591429+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:51:11.144925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'5","reported_seq":170,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:47:47.706050+0000","last_change":"2024-05-03T14:45:05.614704+0000","last_active":"2024-05-03T14:47:47.706050+0000","last_peered":"2024-05-03T14:47:47.706050+0000","last_clean":"2024-05-03T14:47:47.706050+0000","last_became_active":"2024-05-03T14:45:05.614173+0000","last_became_peered":"2024-05-03T14:45:05.614173+0000","last_unstale":"2024-05-03T14:47:47.706050+0000","last_undegraded":"2024-05-03T14:47:47.706050+0000","last_fullsized":"2024-05-03T14:47:47.706050+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:50:42.956239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.593723+0000","last_change":"2024-05-03T14:44:31.464666+0000","last_active":"2024-05-03T14:46:05.593723+0000","last_peered":"2024-05-03T14:46:05.593723+0000","last_clean":"2024-05-03T14:46:05.593723+0000","last_became_active":"2024-05-03T14:44:31.464298+0000","last_became_peered":"2024-05-03T14:44:31.464298+0000","last_unstale":"2024-05-03T14:46:05.593723+0000","last_undegraded":"2024-05-03T14:46:05.593723+0000","last_fullsized":"2024-05-03T14:46:05.593723+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:26:35.425908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.591342+0000","last_change":"2024-05-03T14:44:44.580268+0000","last_active":"2024-05-03T14:46:05.591342+0000","last_peered":"2024-05-03T14:46:05.591342+0000","last_clean":"2024-05-03T14:46:05.591342+0000","last_became_active":"2024-05-03T14:44:44.579617+0000","last_became_peered":"2024-05-03T14:44:44.579617+0000","last_unstale":"2024-05-03T14:46:05.591342+0000","last_undegraded":"2024-05-03T14:46:05.591342+0000","last_fullsized":"2024-05-03T14:46:05.591342+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:16:25.580698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.608165+0000","last_change":"2024-05-03T14:44:27.418578+0000","last_active":"2024-05-03T14:46:05.608165+0000","last_peered":"2024-05-03T14:46:05.608165+0000","last_clean":"2024-05-03T14:46:05.608165+0000","last_became_active":"2024-05-03T14:44:27.418206+0000","last_became_peered":"2024-05-03T14:44:27.418206+0000","last_unstale":"2024-05-03T14:46:05.608165+0000","last_undegraded":"2024-05-03T14:46:05.608165+0000","last_fullsized":"2024-05-03T14:46:05.608165+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:48:47.085451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110237+0000","last_change":"2024-05-03T14:44:31.467353+0000","last_active":"2024-05-03T14:46:06.110237+0000","last_peered":"2024-05-03T14:46:06.110237+0000","last_clean":"2024-05-03T14:46:06.110237+0000","last_became_active":"2024-05-03T14:44:31.466606+0000","last_became_peered":"2024-05-03T14:44:31.466606+0000","last_unstale":"2024-05-03T14:46:06.110237+0000","last_undegraded":"2024-05-03T14:46:06.110237+0000","last_fullsized":"2024-05-03T14:46:06.110237+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T14:50:22.624015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110027+0000","last_change":"2024-05-03T14:44:44.567367+0000","last_active":"2024-05-03T14:46:06.110027+0000","last_peered":"2024-05-03T14:46:06.110027+0000","last_clean":"2024-05-03T14:46:06.110027+0000","last_became_active":"2024-05-03T14:44:44.566916+0000","last_became_peered":"2024-05-03T14:44:44.566916+0000","last_unstale":"2024-05-03T14:46:06.110027+0000","last_undegraded":"2024-05-03T14:46:06.110027+0000","last_fullsized":"2024-05-03T14:46:06.110027+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:07:39.966644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110318+0000","last_change":"2024-05-03T14:44:27.424704+0000","last_active":"2024-05-03T14:46:06.110318+0000","last_peered":"2024-05-03T14:46:06.110318+0000","last_clean":"2024-05-03T14:46:06.110318+0000","last_became_active":"2024-05-03T14:44:27.424231+0000","last_became_peered":"2024-05-03T14:44:27.424231+0000","last_unstale":"2024-05-03T14:46:06.110318+0000","last_undegraded":"2024-05-03T14:46:06.110318+0000","last_fullsized":"2024-05-03T14:46:06.110318+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:00:22.076458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.113192+0000","last_change":"2024-05-03T14:44:31.474599+0000","last_active":"2024-05-03T14:46:06.113192+0000","last_peered":"2024-05-03T14:46:06.113192+0000","last_clean":"2024-05-03T14:46:06.113192+0000","last_became_active":"2024-05-03T14:44:31.473970+0000","last_became_peered":"2024-05-03T14:44:31.473970+0000","last_unstale":"2024-05-03T14:46:06.113192+0000","last_undegraded":"2024-05-03T14:46:06.113192+0000","last_fullsized":"2024-05-03T14:46:06.113192+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:20:14.259534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.590281+0000","last_change":"2024-05-03T14:44:44.580126+0000","last_active":"2024-05-03T14:46:05.590281+0000","last_peered":"2024-05-03T14:46:05.590281+0000","last_clean":"2024-05-03T14:46:05.590281+0000","last_became_active":"2024-05-03T14:44:44.578708+0000","last_became_peered":"2024-05-03T14:44:44.578708+0000","last_unstale":"2024-05-03T14:46:05.590281+0000","last_undegraded":"2024-05-03T14:46:05.590281+0000","last_fullsized":"2024-05-03T14:46:05.590281+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:42:43.279786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109232+0000","last_change":"2024-05-03T14:44:27.405332+0000","last_active":"2024-05-03T14:46:06.109232+0000","last_peered":"2024-05-03T14:46:06.109232+0000","last_clean":"2024-05-03T14:46:06.109232+0000","last_became_active":"2024-05-03T14:44:27.404994+0000","last_became_peered":"2024-05-03T14:44:27.404994+0000","last_unstale":"2024-05-03T14:46:06.109232+0000","last_undegraded":"2024-05-03T14:46:06.109232+0000","last_fullsized":"2024-05-03T14:46:06.109232+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:18:23.716014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.590839+0000","last_change":"2024-05-03T14:44:31.470396+0000","last_active":"2024-05-03T14:46:05.590839+0000","last_peered":"2024-05-03T14:46:05.590839+0000","last_clean":"2024-05-03T14:46:05.590839+0000","last_became_active":"2024-05-03T14:44:31.467973+0000","last_became_peered":"2024-05-03T14:44:31.467973+0000","last_unstale":"2024-05-03T14:46:05.590839+0000","last_undegraded":"2024-05-03T14:46:05.590839+0000","last_fullsized":"2024-05-03T14:46:05.590839+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:44:21.266589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.113155+0000","last_change":"2024-05-03T14:44:44.582678+0000","last_active":"2024-05-03T14:46:06.113155+0000","last_peered":"2024-05-03T14:46:06.113155+0000","last_clean":"2024-05-03T14:46:06.113155+0000","last_became_active":"2024-05-03T14:44:44.582155+0000","last_became_peered":"2024-05-03T14:44:44.582155+0000","last_unstale":"2024-05-03T14:46:06.113155+0000","last_undegraded":"2024-05-03T14:46:06.113155+0000","last_fullsized":"2024-05-03T14:46:06.113155+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T15:21:05.986627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":275,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:47:49.708310+0000","last_change":"2024-05-03T14:44:27.415880+0000","last_active":"2024-05-03T14:47:49.708310+0000","last_peered":"2024-05-03T14:47:49.708310+0000","last_clean":"2024-05-03T14:47:49.708310+0000","last_became_active":"2024-05-03T14:44:27.415354+0000","last_became_peered":"2024-05-03T14:44:27.415354+0000","last_unstale":"2024-05-03T14:47:49.708310+0000","last_undegraded":"2024-05-03T14:47:49.708310+0000","last_fullsized":"2024-05-03T14:47:49.708310+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T16:55:41.808314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":226,"num_read_kb":226,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109557+0000","last_change":"2024-05-03T14:44:31.462685+0000","last_active":"2024-05-03T14:46:06.109557+0000","last_peered":"2024-05-03T14:46:06.109557+0000","last_clean":"2024-05-03T14:46:06.109557+0000","last_became_active":"2024-05-03T14:44:31.462325+0000","last_became_peered":"2024-05-03T14:44:31.462325+0000","last_unstale":"2024-05-03T14:46:06.109557+0000","last_undegraded":"2024-05-03T14:46:06.109557+0000","last_fullsized":"2024-05-03T14:46:06.109557+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:59:54.715648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.590321+0000","last_change":"2024-05-03T14:44:44.573855+0000","last_active":"2024-05-03T14:46:05.590321+0000","last_peered":"2024-05-03T14:46:05.590321+0000","last_clean":"2024-05-03T14:46:05.590321+0000","last_became_active":"2024-05-03T14:44:44.573314+0000","last_became_peered":"2024-05-03T14:44:44.573314+0000","last_unstale":"2024-05-03T14:46:05.590321+0000","last_undegraded":"2024-05-03T14:46:05.590321+0000","last_fullsized":"2024-05-03T14:46:05.590321+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:04:24.282503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.594887+0000","last_change":"2024-05-03T14:45:05.613037+0000","last_active":"2024-05-03T14:46:05.594887+0000","last_peered":"2024-05-03T14:46:05.594887+0000","last_clean":"2024-05-03T14:46:05.594887+0000","last_became_active":"2024-05-03T14:45:05.612581+0000","last_became_peered":"2024-05-03T14:45:05.612581+0000","last_unstale":"2024-05-03T14:46:05.594887+0000","last_undegraded":"2024-05-03T14:46:05.594887+0000","last_fullsized":"2024-05-03T14:46:05.594887+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:34:07.365447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.590847+0000","last_change":"2024-05-03T14:44:31.451912+0000","last_active":"2024-05-03T14:46:05.590847+0000","last_peered":"2024-05-03T14:46:05.590847+0000","last_clean":"2024-05-03T14:46:05.590847+0000","last_became_active":"2024-05-03T14:44:31.449029+0000","last_became_peered":"2024-05-03T14:44:31.449029+0000","last_unstale":"2024-05-03T14:46:05.590847+0000","last_undegraded":"2024-05-03T14:46:05.590847+0000","last_fullsized":"2024-05-03T14:46:05.590847+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:27:13.018719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109453+0000","last_change":"2024-05-03T14:44:44.577001+0000","last_active":"2024-05-03T14:46:06.109453+0000","last_peered":"2024-05-03T14:46:06.109453+0000","last_clean":"2024-05-03T14:46:06.109453+0000","last_became_active":"2024-05-03T14:44:44.576224+0000","last_became_peered":"2024-05-03T14:44:44.576224+0000","last_unstale":"2024-05-03T14:46:06.109453+0000","last_undegraded":"2024-05-03T14:46:06.109453+0000","last_fullsized":"2024-05-03T14:46:06.109453+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T15:15:30.497801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110288+0000","last_change":"2024-05-03T14:44:27.402496+0000","last_active":"2024-05-03T14:46:06.110288+0000","last_peered":"2024-05-03T14:46:06.110288+0000","last_clean":"2024-05-03T14:46:06.110288+0000","last_became_active":"2024-05-03T14:44:27.402062+0000","last_became_peered":"2024-05-03T14:44:27.402062+0000","last_unstale":"2024-05-03T14:46:06.110288+0000","last_undegraded":"2024-05-03T14:46:06.110288+0000","last_fullsized":"2024-05-03T14:46:06.110288+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:47:10.090902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.108933+0000","last_change":"2024-05-03T14:44:31.452654+0000","last_active":"2024-05-03T14:46:06.108933+0000","last_peered":"2024-05-03T14:46:06.108933+0000","last_clean":"2024-05-03T14:46:06.108933+0000","last_became_active":"2024-05-03T14:44:31.452305+0000","last_became_peered":"2024-05-03T14:44:31.452305+0000","last_unstale":"2024-05-03T14:46:06.108933+0000","last_undegraded":"2024-05-03T14:46:06.108933+0000","last_fullsized":"2024-05-03T14:46:06.108933+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:20:45.816113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.591228+0000","last_change":"2024-05-03T14:44:44.576004+0000","last_active":"2024-05-03T14:46:05.591228+0000","last_peered":"2024-05-03T14:46:05.591228+0000","last_clean":"2024-05-03T14:46:05.591228+0000","last_became_active":"2024-05-03T14:44:44.572486+0000","last_became_peered":"2024-05-03T14:44:44.572486+0000","last_unstale":"2024-05-03T14:46:05.591228+0000","last_undegraded":"2024-05-03T14:46:05.591228+0000","last_fullsized":"2024-05-03T14:46:05.591228+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T15:11:51.125404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'38","reported_seq":319,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:47:49.708283+0000","last_change":"2024-05-03T14:44:27.405937+0000","last_active":"2024-05-03T14:47:49.708283+0000","last_peered":"2024-05-03T14:47:49.708283+0000","last_clean":"2024-05-03T14:47:49.708283+0000","last_became_active":"2024-05-03T14:44:27.405511+0000","last_became_peered":"2024-05-03T14:44:27.405511+0000","last_unstale":"2024-05-03T14:47:49.708283+0000","last_undegraded":"2024-05-03T14:47:49.708283+0000","last_fullsized":"2024-05-03T14:47:49.708283+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:00:56.824015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60568,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":236,"num_read_kb":177,"num_write":40,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.594109+0000","last_change":"2024-05-03T14:44:31.448712+0000","last_active":"2024-05-03T14:46:05.594109+0000","last_peered":"2024-05-03T14:46:05.594109+0000","last_clean":"2024-05-03T14:46:05.594109+0000","last_became_active":"2024-05-03T14:44:31.448346+0000","last_became_peered":"2024-05-03T14:44:31.448346+0000","last_unstale":"2024-05-03T14:46:05.594109+0000","last_undegraded":"2024-05-03T14:46:05.594109+0000","last_fullsized":"2024-05-03T14:46:05.594109+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T16:32:40.770487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110659+0000","last_change":"2024-05-03T14:44:44.585600+0000","last_active":"2024-05-03T14:46:06.110659+0000","last_peered":"2024-05-03T14:46:06.110659+0000","last_clean":"2024-05-03T14:46:06.110659+0000","last_became_active":"2024-05-03T14:44:44.585221+0000","last_became_peered":"2024-05-03T14:44:44.585221+0000","last_unstale":"2024-05-03T14:46:06.110659+0000","last_undegraded":"2024-05-03T14:46:06.110659+0000","last_fullsized":"2024-05-03T14:46:06.110659+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:28:45.804470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":56,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.591475+0000","last_change":"2024-05-03T14:44:27.416524+0000","last_active":"2024-05-03T14:46:05.591475+0000","last_peered":"2024-05-03T14:46:05.591475+0000","last_clean":"2024-05-03T14:46:05.591475+0000","last_became_active":"2024-05-03T14:44:27.416211+0000","last_became_peered":"2024-05-03T14:44:27.416211+0000","last_unstale":"2024-05-03T14:46:05.591475+0000","last_undegraded":"2024-05-03T14:46:05.591475+0000","last_fullsized":"2024-05-03T14:46:05.591475+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:00:26.361267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.111079+0000","last_change":"2024-05-03T14:44:31.468973+0000","last_active":"2024-05-03T14:46:06.111079+0000","last_peered":"2024-05-03T14:46:06.111079+0000","last_clean":"2024-05-03T14:46:06.111079+0000","last_became_active":"2024-05-03T14:44:31.468547+0000","last_became_peered":"2024-05-03T14:44:31.468547+0000","last_unstale":"2024-05-03T14:46:06.111079+0000","last_undegraded":"2024-05-03T14:46:06.111079+0000","last_fullsized":"2024-05-03T14:46:06.111079+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:16:16.223859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109343+0000","last_change":"2024-05-03T14:44:44.579157+0000","last_active":"2024-05-03T14:46:06.109343+0000","last_peered":"2024-05-03T14:46:06.109343+0000","last_clean":"2024-05-03T14:46:06.109343+0000","last_became_active":"2024-05-03T14:44:44.578718+0000","last_became_peered":"2024-05-03T14:44:44.578718+0000","last_unstale":"2024-05-03T14:46:06.109343+0000","last_undegraded":"2024-05-03T14:46:06.109343+0000","last_fullsized":"2024-05-03T14:46:06.109343+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:03:31.574958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110085+0000","last_change":"2024-05-03T14:44:27.415290+0000","last_active":"2024-05-03T14:46:06.110085+0000","last_peered":"2024-05-03T14:46:06.110085+0000","last_clean":"2024-05-03T14:46:06.110085+0000","last_became_active":"2024-05-03T14:44:27.414930+0000","last_became_peered":"2024-05-03T14:44:27.414930+0000","last_unstale":"2024-05-03T14:46:06.110085+0000","last_undegraded":"2024-05-03T14:46:06.110085+0000","last_fullsized":"2024-05-03T14:46:06.110085+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:06:40.675902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.594402+0000","last_change":"2024-05-03T14:44:31.464631+0000","last_active":"2024-05-03T14:46:05.594402+0000","last_peered":"2024-05-03T14:46:05.594402+0000","last_clean":"2024-05-03T14:46:05.594402+0000","last_became_active":"2024-05-03T14:44:31.464325+0000","last_became_peered":"2024-05-03T14:44:31.464325+0000","last_unstale":"2024-05-03T14:46:05.594402+0000","last_undegraded":"2024-05-03T14:46:05.594402+0000","last_fullsized":"2024-05-03T14:46:05.594402+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:09:39.140391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.591385+0000","last_change":"2024-05-03T14:44:44.583495+0000","last_active":"2024-05-03T14:46:05.591385+0000","last_peered":"2024-05-03T14:46:05.591385+0000","last_clean":"2024-05-03T14:46:05.591385+0000","last_became_active":"2024-05-03T14:44:44.582971+0000","last_became_peered":"2024-05-03T14:44:44.582971+0000","last_unstale":"2024-05-03T14:46:05.591385+0000","last_undegraded":"2024-05-03T14:46:05.591385+0000","last_fullsized":"2024-05-03T14:46:05.591385+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:23:12.086610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"87'5","reported_seq":168,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:47:51.792024+0000","last_change":"2024-05-03T14:44:27.403319+0000","last_active":"2024-05-03T14:47:51.792024+0000","last_peered":"2024-05-03T14:47:51.792024+0000","last_clean":"2024-05-03T14:47:51.792024+0000","last_became_active":"2024-05-03T14:44:27.402936+0000","last_became_peered":"2024-05-03T14:44:27.402936+0000","last_unstale":"2024-05-03T14:47:51.792024+0000","last_undegraded":"2024-05-03T14:47:51.792024+0000","last_fullsized":"2024-05-03T14:47:51.792024+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:06:24.851356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":119,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.594278+0000","last_change":"2024-05-03T14:44:31.453514+0000","last_active":"2024-05-03T14:46:05.594278+0000","last_peered":"2024-05-03T14:46:05.594278+0000","last_clean":"2024-05-03T14:46:05.594278+0000","last_became_active":"2024-05-03T14:44:31.452755+0000","last_became_peered":"2024-05-03T14:44:31.452755+0000","last_unstale":"2024-05-03T14:46:05.594278+0000","last_undegraded":"2024-05-03T14:46:05.594278+0000","last_fullsized":"2024-05-03T14:46:05.594278+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:29:45.095487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.113886+0000","last_change":"2024-05-03T14:44:44.587105+0000","last_active":"2024-05-03T14:46:06.113886+0000","last_peered":"2024-05-03T14:46:06.113886+0000","last_clean":"2024-05-03T14:46:06.113886+0000","last_became_active":"2024-05-03T14:44:44.586669+0000","last_became_peered":"2024-05-03T14:44:44.586669+0000","last_unstale":"2024-05-03T14:46:06.113886+0000","last_undegraded":"2024-05-03T14:46:06.113886+0000","last_fullsized":"2024-05-03T14:46:06.113886+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:18:16.364544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":63,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.593664+0000","last_change":"2024-05-03T14:44:27.412481+0000","last_active":"2024-05-03T14:46:05.593664+0000","last_peered":"2024-05-03T14:46:05.593664+0000","last_clean":"2024-05-03T14:46:05.593664+0000","last_became_active":"2024-05-03T14:44:27.412146+0000","last_became_peered":"2024-05-03T14:44:27.412146+0000","last_unstale":"2024-05-03T14:46:05.593664+0000","last_undegraded":"2024-05-03T14:46:05.593664+0000","last_fullsized":"2024-05-03T14:46:05.593664+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T16:03:40.205892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109800+0000","last_change":"2024-05-03T14:44:31.453208+0000","last_active":"2024-05-03T14:46:06.109800+0000","last_peered":"2024-05-03T14:46:06.109800+0000","last_clean":"2024-05-03T14:46:06.109800+0000","last_became_active":"2024-05-03T14:44:31.452788+0000","last_became_peered":"2024-05-03T14:44:31.452788+0000","last_unstale":"2024-05-03T14:46:06.109800+0000","last_undegraded":"2024-05-03T14:46:06.109800+0000","last_fullsized":"2024-05-03T14:46:06.109800+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:43:31.554510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109726+0000","last_change":"2024-05-03T14:44:44.582993+0000","last_active":"2024-05-03T14:46:06.109726+0000","last_peered":"2024-05-03T14:46:06.109726+0000","last_clean":"2024-05-03T14:46:06.109726+0000","last_became_active":"2024-05-03T14:44:44.582511+0000","last_became_peered":"2024-05-03T14:44:44.582511+0000","last_unstale":"2024-05-03T14:46:06.109726+0000","last_undegraded":"2024-05-03T14:46:06.109726+0000","last_fullsized":"2024-05-03T14:46:06.109726+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:22:26.380344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.594354+0000","last_change":"2024-05-03T14:44:27.401742+0000","last_active":"2024-05-03T14:46:05.594354+0000","last_peered":"2024-05-03T14:46:05.594354+0000","last_clean":"2024-05-03T14:46:05.594354+0000","last_became_active":"2024-05-03T14:44:27.401347+0000","last_became_peered":"2024-05-03T14:44:27.401347+0000","last_unstale":"2024-05-03T14:46:05.594354+0000","last_undegraded":"2024-05-03T14:46:05.594354+0000","last_fullsized":"2024-05-03T14:46:05.594354+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T15:39:41.144999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109391+0000","last_change":"2024-05-03T14:44:31.453480+0000","last_active":"2024-05-03T14:46:06.109391+0000","last_peered":"2024-05-03T14:46:06.109391+0000","last_clean":"2024-05-03T14:46:06.109391+0000","last_became_active":"2024-05-03T14:44:31.453171+0000","last_became_peered":"2024-05-03T14:44:31.453171+0000","last_unstale":"2024-05-03T14:46:06.109391+0000","last_undegraded":"2024-05-03T14:46:06.109391+0000","last_fullsized":"2024-05-03T14:46:06.109391+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:30:16.567938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.593660+0000","last_change":"2024-05-03T14:44:45.208587+0000","last_active":"2024-05-03T14:46:05.593660+0000","last_peered":"2024-05-03T14:46:05.593660+0000","last_clean":"2024-05-03T14:46:05.593660+0000","last_became_active":"2024-05-03T14:44:45.207987+0000","last_became_peered":"2024-05-03T14:44:45.207987+0000","last_unstale":"2024-05-03T14:46:05.593660+0000","last_undegraded":"2024-05-03T14:46:05.593660+0000","last_fullsized":"2024-05-03T14:46:05.593660+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:22:14.854390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.593134+0000","last_change":"2024-05-03T14:44:27.421333+0000","last_active":"2024-05-03T14:46:05.593134+0000","last_peered":"2024-05-03T14:46:05.593134+0000","last_clean":"2024-05-03T14:46:05.593134+0000","last_became_active":"2024-05-03T14:44:27.419745+0000","last_became_peered":"2024-05-03T14:44:27.419745+0000","last_unstale":"2024-05-03T14:46:05.593134+0000","last_undegraded":"2024-05-03T14:46:05.593134+0000","last_fullsized":"2024-05-03T14:46:05.593134+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:40:57.258260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.608152+0000","last_change":"2024-05-03T14:44:31.467871+0000","last_active":"2024-05-03T14:46:05.608152+0000","last_peered":"2024-05-03T14:46:05.608152+0000","last_clean":"2024-05-03T14:46:05.608152+0000","last_became_active":"2024-05-03T14:44:31.467408+0000","last_became_peered":"2024-05-03T14:44:31.467408+0000","last_unstale":"2024-05-03T14:46:05.608152+0000","last_undegraded":"2024-05-03T14:46:05.608152+0000","last_fullsized":"2024-05-03T14:46:05.608152+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:19:24.843591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110575+0000","last_change":"2024-05-03T14:44:44.583116+0000","last_active":"2024-05-03T14:46:06.110575+0000","last_peered":"2024-05-03T14:46:06.110575+0000","last_clean":"2024-05-03T14:46:06.110575+0000","last_became_active":"2024-05-03T14:44:44.581838+0000","last_became_peered":"2024-05-03T14:44:44.581838+0000","last_unstale":"2024-05-03T14:46:06.110575+0000","last_undegraded":"2024-05-03T14:46:06.110575+0000","last_fullsized":"2024-05-03T14:46:06.110575+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:14:24.608777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'5","reported_seq":164,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:47:49.357378+0000","last_change":"2024-05-03T14:44:27.416068+0000","last_active":"2024-05-03T14:47:49.357378+0000","last_peered":"2024-05-03T14:47:49.357378+0000","last_clean":"2024-05-03T14:47:49.357378+0000","last_became_active":"2024-05-03T14:44:27.415574+0000","last_became_peered":"2024-05-03T14:44:27.415574+0000","last_unstale":"2024-05-03T14:47:49.357378+0000","last_undegraded":"2024-05-03T14:47:49.357378+0000","last_fullsized":"2024-05-03T14:47:49.357378+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T15:31:40.186754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.608212+0000","last_change":"2024-05-03T14:44:31.453935+0000","last_active":"2024-05-03T14:46:05.608212+0000","last_peered":"2024-05-03T14:46:05.608212+0000","last_clean":"2024-05-03T14:46:05.608212+0000","last_became_active":"2024-05-03T14:44:31.453565+0000","last_became_peered":"2024-05-03T14:44:31.453565+0000","last_unstale":"2024-05-03T14:46:05.608212+0000","last_undegraded":"2024-05-03T14:46:05.608212+0000","last_fullsized":"2024-05-03T14:46:05.608212+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:34:04.855830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.111449+0000","last_change":"2024-05-03T14:44:45.210274+0000","last_active":"2024-05-03T14:46:06.111449+0000","last_peered":"2024-05-03T14:46:06.111449+0000","last_clean":"2024-05-03T14:46:06.111449+0000","last_became_active":"2024-05-03T14:44:45.208951+0000","last_became_peered":"2024-05-03T14:44:45.208951+0000","last_unstale":"2024-05-03T14:46:06.111449+0000","last_undegraded":"2024-05-03T14:46:06.111449+0000","last_fullsized":"2024-05-03T14:46:06.111449+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:56:30.953019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109801+0000","last_change":"2024-05-03T14:44:31.449470+0000","last_active":"2024-05-03T14:46:06.109801+0000","last_peered":"2024-05-03T14:46:06.109801+0000","last_clean":"2024-05-03T14:46:06.109801+0000","last_became_active":"2024-05-03T14:44:31.449031+0000","last_became_peered":"2024-05-03T14:44:31.449031+0000","last_unstale":"2024-05-03T14:46:06.109801+0000","last_undegraded":"2024-05-03T14:46:06.109801+0000","last_fullsized":"2024-05-03T14:46:06.109801+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T14:51:54.576685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":164,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:47:48.784714+0000","last_change":"2024-05-03T14:44:27.422378+0000","last_active":"2024-05-03T14:47:48.784714+0000","last_peered":"2024-05-03T14:47:48.784714+0000","last_clean":"2024-05-03T14:47:48.784714+0000","last_became_active":"2024-05-03T14:44:27.421988+0000","last_became_peered":"2024-05-03T14:44:27.421988+0000","last_unstale":"2024-05-03T14:47:48.784714+0000","last_undegraded":"2024-05-03T14:47:48.784714+0000","last_fullsized":"2024-05-03T14:47:48.784714+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:20:10.371418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":114,"num_read_kb":114,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:09.223079+0000","last_change":"2024-05-03T14:44:44.576084+0000","last_active":"2024-05-03T14:46:09.223079+0000","last_peered":"2024-05-03T14:46:09.223079+0000","last_clean":"2024-05-03T14:46:09.223079+0000","last_became_active":"2024-05-03T14:44:44.575633+0000","last_became_peered":"2024-05-03T14:44:44.575633+0000","last_unstale":"2024-05-03T14:46:09.223079+0000","last_undegraded":"2024-05-03T14:46:09.223079+0000","last_fullsized":"2024-05-03T14:46:09.223079+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:35:38.570413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.113766+0000","last_change":"2024-05-03T14:44:31.467024+0000","last_active":"2024-05-03T14:46:06.113766+0000","last_peered":"2024-05-03T14:46:06.113766+0000","last_clean":"2024-05-03T14:46:06.113766+0000","last_became_active":"2024-05-03T14:44:31.466545+0000","last_became_peered":"2024-05-03T14:44:31.466545+0000","last_unstale":"2024-05-03T14:46:06.113766+0000","last_undegraded":"2024-05-03T14:46:06.113766+0000","last_fullsized":"2024-05-03T14:46:06.113766+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:34:41.289638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109321+0000","last_change":"2024-05-03T14:44:27.410463+0000","last_active":"2024-05-03T14:46:06.109321+0000","last_peered":"2024-05-03T14:46:06.109321+0000","last_clean":"2024-05-03T14:46:06.109321+0000","last_became_active":"2024-05-03T14:44:27.410017+0000","last_became_peered":"2024-05-03T14:44:27.410017+0000","last_unstale":"2024-05-03T14:46:06.109321+0000","last_undegraded":"2024-05-03T14:46:06.109321+0000","last_fullsized":"2024-05-03T14:46:06.109321+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T16:44:40.376214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":120,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.594056+0000","last_change":"2024-05-03T14:42:51.347913+0000","last_active":"2024-05-03T14:46:05.594056+0000","last_peered":"2024-05-03T14:46:05.594056+0000","last_clean":"2024-05-03T14:46:05.594056+0000","last_became_active":"2024-05-03T14:42:51.341302+0000","last_became_peered":"2024-05-03T14:42:51.341302+0000","last_unstale":"2024-05-03T14:46:05.594056+0000","last_undegraded":"2024-05-03T14:46:05.594056+0000","last_fullsized":"2024-05-03T14:46:05.594056+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:38:55.096374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:38:55.096374+0000","last_clean_scrub_stamp":"2024-05-03T14:38:55.096374+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:35:46.708686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110588+0000","last_change":"2024-05-03T14:44:45.209369+0000","last_active":"2024-05-03T14:46:06.110588+0000","last_peered":"2024-05-03T14:46:06.110588+0000","last_clean":"2024-05-03T14:46:06.110588+0000","last_became_active":"2024-05-03T14:44:45.208835+0000","last_became_peered":"2024-05-03T14:44:45.208835+0000","last_unstale":"2024-05-03T14:46:06.110588+0000","last_undegraded":"2024-05-03T14:46:06.110588+0000","last_fullsized":"2024-05-03T14:46:06.110588+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:35:35.343030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110001+0000","last_change":"2024-05-03T14:44:31.462199+0000","last_active":"2024-05-03T14:46:06.110001+0000","last_peered":"2024-05-03T14:46:06.110001+0000","last_clean":"2024-05-03T14:46:06.110001+0000","last_became_active":"2024-05-03T14:44:31.460972+0000","last_became_peered":"2024-05-03T14:44:31.460972+0000","last_unstale":"2024-05-03T14:46:06.110001+0000","last_undegraded":"2024-05-03T14:46:06.110001+0000","last_fullsized":"2024-05-03T14:46:06.110001+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:21:49.619023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.594437+0000","last_change":"2024-05-03T14:44:27.403303+0000","last_active":"2024-05-03T14:46:05.594437+0000","last_peered":"2024-05-03T14:46:05.594437+0000","last_clean":"2024-05-03T14:46:05.594437+0000","last_became_active":"2024-05-03T14:44:27.402870+0000","last_became_peered":"2024-05-03T14:44:27.402870+0000","last_unstale":"2024-05-03T14:46:05.594437+0000","last_undegraded":"2024-05-03T14:46:05.594437+0000","last_fullsized":"2024-05-03T14:46:05.594437+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:57:23.209298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109302+0000","last_change":"2024-05-03T14:44:44.579532+0000","last_active":"2024-05-03T14:46:06.109302+0000","last_peered":"2024-05-03T14:46:06.109302+0000","last_clean":"2024-05-03T14:46:06.109302+0000","last_became_active":"2024-05-03T14:44:44.579052+0000","last_became_peered":"2024-05-03T14:44:44.579052+0000","last_unstale":"2024-05-03T14:46:06.109302+0000","last_undegraded":"2024-05-03T14:46:06.109302+0000","last_fullsized":"2024-05-03T14:46:06.109302+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:07:43.567880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109553+0000","last_change":"2024-05-03T14:45:05.422044+0000","last_active":"2024-05-03T14:46:06.109553+0000","last_peered":"2024-05-03T14:46:06.109553+0000","last_clean":"2024-05-03T14:46:06.109553+0000","last_became_active":"2024-05-03T14:45:05.421557+0000","last_became_peered":"2024-05-03T14:45:05.421557+0000","last_unstale":"2024-05-03T14:46:06.109553+0000","last_undegraded":"2024-05-03T14:46:06.109553+0000","last_fullsized":"2024-05-03T14:46:06.109553+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:44:03.074990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110911+0000","last_change":"2024-05-03T14:44:31.460150+0000","last_active":"2024-05-03T14:46:06.110911+0000","last_peered":"2024-05-03T14:46:06.110911+0000","last_clean":"2024-05-03T14:46:06.110911+0000","last_became_active":"2024-05-03T14:44:31.459550+0000","last_became_peered":"2024-05-03T14:44:31.459550+0000","last_unstale":"2024-05-03T14:46:06.110911+0000","last_undegraded":"2024-05-03T14:46:06.110911+0000","last_fullsized":"2024-05-03T14:46:06.110911+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:28:17.556068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.113036+0000","last_change":"2024-05-03T14:44:45.215035+0000","last_active":"2024-05-03T14:46:06.113036+0000","last_peered":"2024-05-03T14:46:06.113036+0000","last_clean":"2024-05-03T14:46:06.113036+0000","last_became_active":"2024-05-03T14:44:45.208667+0000","last_became_peered":"2024-05-03T14:44:45.208667+0000","last_unstale":"2024-05-03T14:46:06.113036+0000","last_undegraded":"2024-05-03T14:46:06.113036+0000","last_fullsized":"2024-05-03T14:46:06.113036+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:11:47.512884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'14","reported_seq":59,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:37.441771+0000","last_change":"2024-05-03T14:44:27.417377+0000","last_active":"2024-05-03T14:46:37.441771+0000","last_peered":"2024-05-03T14:46:37.441771+0000","last_clean":"2024-05-03T14:46:37.441771+0000","last_became_active":"2024-05-03T14:44:27.416983+0000","last_became_peered":"2024-05-03T14:44:27.416983+0000","last_unstale":"2024-05-03T14:46:37.441771+0000","last_undegraded":"2024-05-03T14:46:37.441771+0000","last_fullsized":"2024-05-03T14:46:37.441771+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T15:37:31.610517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.115275+0000","last_change":"2024-05-03T14:44:31.467316+0000","last_active":"2024-05-03T14:46:06.115275+0000","last_peered":"2024-05-03T14:46:06.115275+0000","last_clean":"2024-05-03T14:46:06.115275+0000","last_became_active":"2024-05-03T14:44:31.466025+0000","last_became_peered":"2024-05-03T14:44:31.466025+0000","last_unstale":"2024-05-03T14:46:06.115275+0000","last_undegraded":"2024-05-03T14:46:06.115275+0000","last_fullsized":"2024-05-03T14:46:06.115275+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:13:00.130689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.591023+0000","last_change":"2024-05-03T14:44:44.567031+0000","last_active":"2024-05-03T14:46:05.591023+0000","last_peered":"2024-05-03T14:46:05.591023+0000","last_clean":"2024-05-03T14:46:05.591023+0000","last_became_active":"2024-05-03T14:44:44.566562+0000","last_became_peered":"2024-05-03T14:44:44.566562+0000","last_unstale":"2024-05-03T14:46:05.591023+0000","last_undegraded":"2024-05-03T14:46:05.591023+0000","last_fullsized":"2024-05-03T14:46:05.591023+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T15:47:12.335283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110083+0000","last_change":"2024-05-03T14:44:28.279115+0000","last_active":"2024-05-03T14:46:06.110083+0000","last_peered":"2024-05-03T14:46:06.110083+0000","last_clean":"2024-05-03T14:46:06.110083+0000","last_became_active":"2024-05-03T14:44:28.278781+0000","last_became_peered":"2024-05-03T14:44:28.278781+0000","last_unstale":"2024-05-03T14:46:06.110083+0000","last_undegraded":"2024-05-03T14:46:06.110083+0000","last_fullsized":"2024-05-03T14:46:06.110083+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:40:41.273763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.590492+0000","last_change":"2024-05-03T14:44:31.475575+0000","last_active":"2024-05-03T14:46:05.590492+0000","last_peered":"2024-05-03T14:46:05.590492+0000","last_clean":"2024-05-03T14:46:05.590492+0000","last_became_active":"2024-05-03T14:44:31.474336+0000","last_became_peered":"2024-05-03T14:44:31.474336+0000","last_unstale":"2024-05-03T14:46:05.590492+0000","last_undegraded":"2024-05-03T14:46:05.590492+0000","last_fullsized":"2024-05-03T14:46:05.590492+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:22:48.742222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109679+0000","last_change":"2024-05-03T14:45:05.616983+0000","last_active":"2024-05-03T14:46:06.109679+0000","last_peered":"2024-05-03T14:46:06.109679+0000","last_clean":"2024-05-03T14:46:06.109679+0000","last_became_active":"2024-05-03T14:45:05.616475+0000","last_became_peered":"2024-05-03T14:45:05.616475+0000","last_unstale":"2024-05-03T14:46:06.109679+0000","last_undegraded":"2024-05-03T14:46:06.109679+0000","last_fullsized":"2024-05-03T14:46:06.109679+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:01:02.124133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.108773+0000","last_change":"2024-05-03T14:44:27.417092+0000","last_active":"2024-05-03T14:46:06.108773+0000","last_peered":"2024-05-03T14:46:06.108773+0000","last_clean":"2024-05-03T14:46:06.108773+0000","last_became_active":"2024-05-03T14:44:27.416765+0000","last_became_peered":"2024-05-03T14:44:27.416765+0000","last_unstale":"2024-05-03T14:46:06.108773+0000","last_undegraded":"2024-05-03T14:46:06.108773+0000","last_fullsized":"2024-05-03T14:46:06.108773+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T16:08:01.882063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.593648+0000","last_change":"2024-05-03T14:44:31.448394+0000","last_active":"2024-05-03T14:46:05.593648+0000","last_peered":"2024-05-03T14:46:05.593648+0000","last_clean":"2024-05-03T14:46:05.593648+0000","last_became_active":"2024-05-03T14:44:31.447752+0000","last_became_peered":"2024-05-03T14:44:31.447752+0000","last_unstale":"2024-05-03T14:46:05.593648+0000","last_undegraded":"2024-05-03T14:46:05.593648+0000","last_fullsized":"2024-05-03T14:46:05.593648+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:01:24.601259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.591583+0000","last_change":"2024-05-03T14:44:44.587688+0000","last_active":"2024-05-03T14:46:05.591583+0000","last_peered":"2024-05-03T14:46:05.591583+0000","last_clean":"2024-05-03T14:46:05.591583+0000","last_became_active":"2024-05-03T14:44:44.586982+0000","last_became_peered":"2024-05-03T14:44:44.586982+0000","last_unstale":"2024-05-03T14:46:05.591583+0000","last_undegraded":"2024-05-03T14:46:05.591583+0000","last_fullsized":"2024-05-03T14:46:05.591583+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:22:54.456635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.113246+0000","last_change":"2024-05-03T14:44:27.411828+0000","last_active":"2024-05-03T14:46:06.113246+0000","last_peered":"2024-05-03T14:46:06.113246+0000","last_clean":"2024-05-03T14:46:06.113246+0000","last_became_active":"2024-05-03T14:44:27.410490+0000","last_became_peered":"2024-05-03T14:44:27.410490+0000","last_unstale":"2024-05-03T14:46:06.113246+0000","last_undegraded":"2024-05-03T14:46:06.113246+0000","last_fullsized":"2024-05-03T14:46:06.113246+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:34:00.061773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109119+0000","last_change":"2024-05-03T14:44:31.448582+0000","last_active":"2024-05-03T14:46:06.109119+0000","last_peered":"2024-05-03T14:46:06.109119+0000","last_clean":"2024-05-03T14:46:06.109119+0000","last_became_active":"2024-05-03T14:44:31.448246+0000","last_became_peered":"2024-05-03T14:44:31.448246+0000","last_unstale":"2024-05-03T14:46:06.109119+0000","last_undegraded":"2024-05-03T14:46:06.109119+0000","last_fullsized":"2024-05-03T14:46:06.109119+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:12:19.856995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:17.125526+0000","last_change":"2024-05-03T14:45:05.616413+0000","last_active":"2024-05-03T14:46:17.125526+0000","last_peered":"2024-05-03T14:46:17.125526+0000","last_clean":"2024-05-03T14:46:17.125526+0000","last_became_active":"2024-05-03T14:45:05.615930+0000","last_became_peered":"2024-05-03T14:45:05.615930+0000","last_unstale":"2024-05-03T14:46:17.125526+0000","last_undegraded":"2024-05-03T14:46:17.125526+0000","last_fullsized":"2024-05-03T14:46:17.125526+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:32:43.344452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.591122+0000","last_change":"2024-05-03T14:44:27.417625+0000","last_active":"2024-05-03T14:46:05.591122+0000","last_peered":"2024-05-03T14:46:05.591122+0000","last_clean":"2024-05-03T14:46:05.591122+0000","last_became_active":"2024-05-03T14:44:27.417230+0000","last_became_peered":"2024-05-03T14:44:27.417230+0000","last_unstale":"2024-05-03T14:46:05.591122+0000","last_undegraded":"2024-05-03T14:46:05.591122+0000","last_fullsized":"2024-05-03T14:46:05.591122+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:56:13.661544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.114374+0000","last_change":"2024-05-03T14:44:31.475539+0000","last_active":"2024-05-03T14:46:06.114374+0000","last_peered":"2024-05-03T14:46:06.114374+0000","last_clean":"2024-05-03T14:46:06.114374+0000","last_became_active":"2024-05-03T14:44:31.474743+0000","last_became_peered":"2024-05-03T14:44:31.474743+0000","last_unstale":"2024-05-03T14:46:06.114374+0000","last_undegraded":"2024-05-03T14:46:06.114374+0000","last_fullsized":"2024-05-03T14:46:06.114374+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T15:52:19.705304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109488+0000","last_change":"2024-05-03T14:44:45.208377+0000","last_active":"2024-05-03T14:46:06.109488+0000","last_peered":"2024-05-03T14:46:06.109488+0000","last_clean":"2024-05-03T14:46:06.109488+0000","last_became_active":"2024-05-03T14:44:45.207778+0000","last_became_peered":"2024-05-03T14:44:45.207778+0000","last_unstale":"2024-05-03T14:46:06.109488+0000","last_undegraded":"2024-05-03T14:46:06.109488+0000","last_fullsized":"2024-05-03T14:46:06.109488+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T15:51:36.843263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.593889+0000","last_change":"2024-05-03T14:44:27.403571+0000","last_active":"2024-05-03T14:46:05.593889+0000","last_peered":"2024-05-03T14:46:05.593889+0000","last_clean":"2024-05-03T14:46:05.593889+0000","last_became_active":"2024-05-03T14:44:27.403173+0000","last_became_peered":"2024-05-03T14:44:27.403173+0000","last_unstale":"2024-05-03T14:46:05.593889+0000","last_undegraded":"2024-05-03T14:46:05.593889+0000","last_fullsized":"2024-05-03T14:46:05.593889+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T15:32:07.406763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.594841+0000","last_change":"2024-05-03T14:44:31.459573+0000","last_active":"2024-05-03T14:46:05.594841+0000","last_peered":"2024-05-03T14:46:05.594841+0000","last_clean":"2024-05-03T14:46:05.594841+0000","last_became_active":"2024-05-03T14:44:31.459200+0000","last_became_peered":"2024-05-03T14:44:31.459200+0000","last_unstale":"2024-05-03T14:46:05.594841+0000","last_undegraded":"2024-05-03T14:46:05.594841+0000","last_fullsized":"2024-05-03T14:46:05.594841+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:58:04.841787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.108797+0000","last_change":"2024-05-03T14:44:44.579130+0000","last_active":"2024-05-03T14:46:06.108797+0000","last_peered":"2024-05-03T14:46:06.108797+0000","last_clean":"2024-05-03T14:46:06.108797+0000","last_became_active":"2024-05-03T14:44:44.578696+0000","last_became_peered":"2024-05-03T14:44:44.578696+0000","last_unstale":"2024-05-03T14:46:06.108797+0000","last_undegraded":"2024-05-03T14:46:06.108797+0000","last_fullsized":"2024-05-03T14:46:06.108797+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:42:48.407127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'5","reported_seq":169,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:47:52.713498+0000","last_change":"2024-05-03T14:44:28.279181+0000","last_active":"2024-05-03T14:47:52.713498+0000","last_peered":"2024-05-03T14:47:52.713498+0000","last_clean":"2024-05-03T14:47:52.713498+0000","last_became_active":"2024-05-03T14:44:28.278790+0000","last_became_peered":"2024-05-03T14:44:28.278790+0000","last_unstale":"2024-05-03T14:47:52.713498+0000","last_undegraded":"2024-05-03T14:47:52.713498+0000","last_fullsized":"2024-05-03T14:47:52.713498+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:16:30.170750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":119,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.594905+0000","last_change":"2024-05-03T14:44:31.451646+0000","last_active":"2024-05-03T14:46:05.594905+0000","last_peered":"2024-05-03T14:46:05.594905+0000","last_clean":"2024-05-03T14:46:05.594905+0000","last_became_active":"2024-05-03T14:44:31.451319+0000","last_became_peered":"2024-05-03T14:44:31.451319+0000","last_unstale":"2024-05-03T14:46:05.594905+0000","last_undegraded":"2024-05-03T14:46:05.594905+0000","last_fullsized":"2024-05-03T14:46:05.594905+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:17:51.946028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.590751+0000","last_change":"2024-05-03T14:45:05.619282+0000","last_active":"2024-05-03T14:46:05.590751+0000","last_peered":"2024-05-03T14:46:05.590751+0000","last_clean":"2024-05-03T14:46:05.590751+0000","last_became_active":"2024-05-03T14:45:05.617776+0000","last_became_peered":"2024-05-03T14:45:05.617776+0000","last_unstale":"2024-05-03T14:46:05.590751+0000","last_undegraded":"2024-05-03T14:46:05.590751+0000","last_fullsized":"2024-05-03T14:46:05.590751+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:36:53.105226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110481+0000","last_change":"2024-05-03T14:44:27.415277+0000","last_active":"2024-05-03T14:46:06.110481+0000","last_peered":"2024-05-03T14:46:06.110481+0000","last_clean":"2024-05-03T14:46:06.110481+0000","last_became_active":"2024-05-03T14:44:27.414900+0000","last_became_peered":"2024-05-03T14:44:27.414900+0000","last_unstale":"2024-05-03T14:46:06.110481+0000","last_undegraded":"2024-05-03T14:46:06.110481+0000","last_fullsized":"2024-05-03T14:46:06.110481+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:41:00.065933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.594580+0000","last_change":"2024-05-03T14:44:31.467592+0000","last_active":"2024-05-03T14:46:05.594580+0000","last_peered":"2024-05-03T14:46:05.594580+0000","last_clean":"2024-05-03T14:46:05.594580+0000","last_became_active":"2024-05-03T14:44:31.467265+0000","last_became_peered":"2024-05-03T14:44:31.467265+0000","last_unstale":"2024-05-03T14:46:05.594580+0000","last_undegraded":"2024-05-03T14:46:05.594580+0000","last_fullsized":"2024-05-03T14:46:05.594580+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:28:26.672800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109129+0000","last_change":"2024-05-03T14:44:44.566154+0000","last_active":"2024-05-03T14:46:06.109129+0000","last_peered":"2024-05-03T14:46:06.109129+0000","last_clean":"2024-05-03T14:46:06.109129+0000","last_became_active":"2024-05-03T14:44:44.565714+0000","last_became_peered":"2024-05-03T14:44:44.565714+0000","last_unstale":"2024-05-03T14:46:06.109129+0000","last_undegraded":"2024-05-03T14:46:06.109129+0000","last_fullsized":"2024-05-03T14:46:06.109129+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:54:48.448838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.113683+0000","last_change":"2024-05-03T14:44:27.419293+0000","last_active":"2024-05-03T14:46:06.113683+0000","last_peered":"2024-05-03T14:46:06.113683+0000","last_clean":"2024-05-03T14:46:06.113683+0000","last_became_active":"2024-05-03T14:44:27.418902+0000","last_became_peered":"2024-05-03T14:44:27.418902+0000","last_unstale":"2024-05-03T14:46:06.113683+0000","last_undegraded":"2024-05-03T14:46:06.113683+0000","last_fullsized":"2024-05-03T14:46:06.113683+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:17:54.890553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.595091+0000","last_change":"2024-05-03T14:44:31.449682+0000","last_active":"2024-05-03T14:46:05.595091+0000","last_peered":"2024-05-03T14:46:05.595091+0000","last_clean":"2024-05-03T14:46:05.595091+0000","last_became_active":"2024-05-03T14:44:31.449192+0000","last_became_peered":"2024-05-03T14:44:31.449192+0000","last_unstale":"2024-05-03T14:46:05.595091+0000","last_undegraded":"2024-05-03T14:46:05.595091+0000","last_fullsized":"2024-05-03T14:46:05.595091+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:48:46.249154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109183+0000","last_change":"2024-05-03T14:45:05.422527+0000","last_active":"2024-05-03T14:46:06.109183+0000","last_peered":"2024-05-03T14:46:06.109183+0000","last_clean":"2024-05-03T14:46:06.109183+0000","last_became_active":"2024-05-03T14:45:05.422014+0000","last_became_peered":"2024-05-03T14:45:05.422014+0000","last_unstale":"2024-05-03T14:46:06.109183+0000","last_undegraded":"2024-05-03T14:46:06.109183+0000","last_fullsized":"2024-05-03T14:46:06.109183+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T15:13:47.216960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.591688+0000","last_change":"2024-05-03T14:44:31.450447+0000","last_active":"2024-05-03T14:46:05.591688+0000","last_peered":"2024-05-03T14:46:05.591688+0000","last_clean":"2024-05-03T14:46:05.591688+0000","last_became_active":"2024-05-03T14:44:31.450103+0000","last_became_peered":"2024-05-03T14:44:31.450103+0000","last_unstale":"2024-05-03T14:46:05.591688+0000","last_undegraded":"2024-05-03T14:46:05.591688+0000","last_fullsized":"2024-05-03T14:46:05.591688+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:43:06.840375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.591291+0000","last_change":"2024-05-03T14:44:31.467676+0000","last_active":"2024-05-03T14:46:05.591291+0000","last_peered":"2024-05-03T14:46:05.591291+0000","last_clean":"2024-05-03T14:46:05.591291+0000","last_became_active":"2024-05-03T14:44:31.467285+0000","last_became_peered":"2024-05-03T14:44:31.467285+0000","last_unstale":"2024-05-03T14:46:05.591291+0000","last_undegraded":"2024-05-03T14:46:05.591291+0000","last_fullsized":"2024-05-03T14:46:05.591291+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:39:37.400178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.591341+0000","last_change":"2024-05-03T14:44:31.469379+0000","last_active":"2024-05-03T14:46:05.591341+0000","last_peered":"2024-05-03T14:46:05.591341+0000","last_clean":"2024-05-03T14:46:05.591341+0000","last_became_active":"2024-05-03T14:44:31.469022+0000","last_became_peered":"2024-05-03T14:44:31.469022+0000","last_unstale":"2024-05-03T14:46:05.591341+0000","last_undegraded":"2024-05-03T14:46:05.591341+0000","last_fullsized":"2024-05-03T14:46:05.591341+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:33:54.227652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.111107+0000","last_change":"2024-05-03T14:44:31.463709+0000","last_active":"2024-05-03T14:46:06.111107+0000","last_peered":"2024-05-03T14:46:06.111107+0000","last_clean":"2024-05-03T14:46:06.111107+0000","last_became_active":"2024-05-03T14:44:31.463302+0000","last_became_peered":"2024-05-03T14:44:31.463302+0000","last_unstale":"2024-05-03T14:46:06.111107+0000","last_undegraded":"2024-05-03T14:46:06.111107+0000","last_fullsized":"2024-05-03T14:46:06.111107+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:44:39.692253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109824+0000","last_change":"2024-05-03T14:44:31.443381+0000","last_active":"2024-05-03T14:46:06.109824+0000","last_peered":"2024-05-03T14:46:06.109824+0000","last_clean":"2024-05-03T14:46:06.109824+0000","last_became_active":"2024-05-03T14:44:31.442981+0000","last_became_peered":"2024-05-03T14:44:31.442981+0000","last_unstale":"2024-05-03T14:46:06.109824+0000","last_undegraded":"2024-05-03T14:46:06.109824+0000","last_fullsized":"2024-05-03T14:46:06.109824+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:58:56.653006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.591689+0000","last_change":"2024-05-03T14:44:31.449229+0000","last_active":"2024-05-03T14:46:05.591689+0000","last_peered":"2024-05-03T14:46:05.591689+0000","last_clean":"2024-05-03T14:46:05.591689+0000","last_became_active":"2024-05-03T14:44:31.448870+0000","last_became_peered":"2024-05-03T14:44:31.448870+0000","last_unstale":"2024-05-03T14:46:05.591689+0000","last_undegraded":"2024-05-03T14:46:05.591689+0000","last_fullsized":"2024-05-03T14:46:05.591689+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:48:29.213680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.115435+0000","last_change":"2024-05-03T14:44:31.468266+0000","last_active":"2024-05-03T14:46:06.115435+0000","last_peered":"2024-05-03T14:46:06.115435+0000","last_clean":"2024-05-03T14:46:06.115435+0000","last_became_active":"2024-05-03T14:44:31.467784+0000","last_became_peered":"2024-05-03T14:44:31.467784+0000","last_unstale":"2024-05-03T14:46:06.115435+0000","last_undegraded":"2024-05-03T14:46:06.115435+0000","last_fullsized":"2024-05-03T14:46:06.115435+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:46:18.979331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109250+0000","last_change":"2024-05-03T14:44:31.467526+0000","last_active":"2024-05-03T14:46:06.109250+0000","last_peered":"2024-05-03T14:46:06.109250+0000","last_clean":"2024-05-03T14:46:06.109250+0000","last_became_active":"2024-05-03T14:44:31.466849+0000","last_became_peered":"2024-05-03T14:44:31.466849+0000","last_unstale":"2024-05-03T14:46:06.109250+0000","last_undegraded":"2024-05-03T14:46:06.109250+0000","last_fullsized":"2024-05-03T14:46:06.109250+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:57:11.147130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.113267+0000","last_change":"2024-05-03T14:44:31.468251+0000","last_active":"2024-05-03T14:46:06.113267+0000","last_peered":"2024-05-03T14:46:06.113267+0000","last_clean":"2024-05-03T14:46:06.113267+0000","last_became_active":"2024-05-03T14:44:31.467812+0000","last_became_peered":"2024-05-03T14:44:31.467812+0000","last_unstale":"2024-05-03T14:46:06.113267+0000","last_undegraded":"2024-05-03T14:46:06.113267+0000","last_fullsized":"2024-05-03T14:46:06.113267+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:52:45.585331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.108911+0000","last_change":"2024-05-03T14:44:31.466378+0000","last_active":"2024-05-03T14:46:06.108911+0000","last_peered":"2024-05-03T14:46:06.108911+0000","last_clean":"2024-05-03T14:46:06.108911+0000","last_became_active":"2024-05-03T14:44:31.466052+0000","last_became_peered":"2024-05-03T14:44:31.466052+0000","last_unstale":"2024-05-03T14:46:06.108911+0000","last_undegraded":"2024-05-03T14:46:06.108911+0000","last_fullsized":"2024-05-03T14:46:06.108911+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:34:48.401934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110503+0000","last_change":"2024-05-03T14:44:31.463771+0000","last_active":"2024-05-03T14:46:06.110503+0000","last_peered":"2024-05-03T14:46:06.110503+0000","last_clean":"2024-05-03T14:46:06.110503+0000","last_became_active":"2024-05-03T14:44:31.463204+0000","last_became_peered":"2024-05-03T14:44:31.463204+0000","last_unstale":"2024-05-03T14:46:06.110503+0000","last_undegraded":"2024-05-03T14:46:06.110503+0000","last_fullsized":"2024-05-03T14:46:06.110503+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:47:33.044835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109581+0000","last_change":"2024-05-03T14:44:31.450950+0000","last_active":"2024-05-03T14:46:06.109581+0000","last_peered":"2024-05-03T14:46:06.109581+0000","last_clean":"2024-05-03T14:46:06.109581+0000","last_became_active":"2024-05-03T14:44:31.450595+0000","last_became_peered":"2024-05-03T14:44:31.450595+0000","last_unstale":"2024-05-03T14:46:06.109581+0000","last_undegraded":"2024-05-03T14:46:06.109581+0000","last_fullsized":"2024-05-03T14:46:06.109581+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:42:41.240430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":250062,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66990,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1176,"num_read_kb":659,"num_write":249,"num_write_kb":140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":200970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":214,"ondisk_log_size":214,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710778,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":516321,"data_compressed":32309,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907007,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28128,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717024,"statfs":{"total":95995035648,"available":95966232576,"internally_reserved":0,"allocated":1138688,"data_stored":1167017,"data_compressed":68497,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070534,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":515467,"data_compressed":32309,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266764,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":999424,"data_stored":515733,"data_compressed":32309,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462992,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":1111279,"data_compressed":68497,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626518,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":987136,"data_stored":510971,"data_compressed":32309,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574962,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":573191,"data_compressed":32309,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738489,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":512427,"data_compressed":32309,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411440,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1064960,"data_stored":572167,"data_compressed":32309,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182504,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":510679,"data_compressed":32309,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986275,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":995328,"data_stored":515045,"data_compressed":32309,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822749,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1069056,"data_stored":1107533,"data_compressed":68497,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34561,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34561,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36188,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36188,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36188,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36188,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36188,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":62942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":61918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":62304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-03T14:47:55.373 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-03T14:47:55.830 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:47:55.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:47:55 smithi090 ceph-mon[33568]: pgmap v434: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:47:55.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:47:55 smithi090 ceph-mon[33568]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T14:47:55.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:47:55 smithi149 ceph-mon[33332]: pgmap v434: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:47:55.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:47:55 smithi149 ceph-mon[33332]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T14:47:55.978 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:47:55 smithi060 ceph-mon[28873]: pgmap v434: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:47:55.979 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:47:55 smithi060 ceph-mon[28873]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T14:47:57.783 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:47:57 smithi060 ceph-mon[28873]: pgmap v435: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:47:57.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:47:57 smithi090 ceph-mon[33568]: pgmap v435: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:47:57.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:47:57 smithi149 ceph-mon[33332]: pgmap v435: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:47:57.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:57.954+0000 7f7162200700 1 -- 172.21.15.60:0/4172179638 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f715c10e8b0 msgr2=0x7f715c10ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:57.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:57.954+0000 7f7162200700 1 --2- 172.21.15.60:0/4172179638 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f715c10e8b0 0x7f715c10ec90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f71500098f0 tx=0x7f7150009c00 comp rx=0 tx=0).stop 2024-05-03T14:47:57.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:57.954+0000 7f7162200700 1 -- 172.21.15.60:0/4172179638 shutdown_connections 2024-05-03T14:47:57.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:57.954+0000 7f7162200700 1 --2- 172.21.15.60:0/4172179638 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f715c10a3a0 0x7f715c116840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:57.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:57.954+0000 7f7162200700 1 --2- 172.21.15.60:0/4172179638 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f715c10e8b0 0x7f715c10ec90 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:57.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:57.954+0000 7f7162200700 1 --2- 172.21.15.60:0/4172179638 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f715c1099e0 0x7f715c109e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:57.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:57.954+0000 7f7162200700 1 -- 172.21.15.60:0/4172179638 >> 172.21.15.60:0/4172179638 conn(0x7f715c079230 msgr2=0x7f715c1082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:47:57.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:57.954+0000 7f7162200700 1 -- 172.21.15.60:0/4172179638 shutdown_connections 2024-05-03T14:47:57.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:57.954+0000 7f7162200700 1 -- 172.21.15.60:0/4172179638 wait complete. 2024-05-03T14:47:57.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:57.956+0000 7f7162200700 1 Processor -- start 2024-05-03T14:47:57.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:57.956+0000 7f7162200700 1 -- start start 2024-05-03T14:47:57.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:57.957+0000 7f7162200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f715c1099e0 0x7f715c1a02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:57.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:57.957+0000 7f7162200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f715c10a3a0 0x7f715c1a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:57.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:57.957+0000 7f7162200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f715c10e8b0 0x7f715c1a4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:57.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:57.957+0000 7f7162200700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f715c119e60 con 0x7f715c10e8b0 2024-05-03T14:47:57.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:57.957+0000 7f7162200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f715c119ce0 con 0x7f715c1099e0 2024-05-03T14:47:57.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:57.957+0000 7f7162200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f715c119fe0 con 0x7f715c10a3a0 2024-05-03T14:47:57.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:57.957+0000 7f715be00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f715c10a3a0 0x7f715c1a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:57.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:57.957+0000 7f715be00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f715c10a3a0 0x7f715c1a07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.60:57228/0 (socket says 172.21.15.60:57228) 2024-05-03T14:47:57.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:57.957+0000 7f715be00700 1 -- 172.21.15.60:0/711584582 learned_addr learned my addr 172.21.15.60:0/711584582 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:47:57.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:57.957+0000 7f7161800700 1 --2- 172.21.15.60:0/711584582 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f715c10e8b0 0x7f715c1a4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:57.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:57.957+0000 7f7160e00700 1 --2- 172.21.15.60:0/711584582 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f715c1099e0 0x7f715c1a02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:57.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:57.958+0000 7f715be00700 1 -- 172.21.15.60:0/711584582 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f715c1099e0 msgr2=0x7f715c1a02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:57.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:57.958+0000 7f715be00700 1 --2- 172.21.15.60:0/711584582 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f715c1099e0 0x7f715c1a02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:57.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:57.958+0000 7f715be00700 1 -- 172.21.15.60:0/711584582 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f715c10e8b0 msgr2=0x7f715c1a4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:57.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:57.958+0000 7f715be00700 1 --2- 172.21.15.60:0/711584582 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f715c10e8b0 0x7f715c1a4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:57.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:57.958+0000 7f715be00700 1 -- 172.21.15.60:0/711584582 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f715c1a52c0 con 0x7f715c10a3a0 2024-05-03T14:47:57.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:57.958+0000 7f715be00700 1 --2- 172.21.15.60:0/711584582 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f715c10a3a0 0x7f715c1a07f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f714c00e9c0 tx=0x7f714c00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:57.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:57.958+0000 7f7159600700 1 -- 172.21.15.60:0/711584582 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f714c004a30 con 0x7f715c10a3a0 2024-05-03T14:47:57.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:57.959+0000 7f7162200700 1 -- 172.21.15.60:0/711584582 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f715c1a55b0 con 0x7f715c10a3a0 2024-05-03T14:47:57.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:57.959+0000 7f7159600700 1 -- 172.21.15.60:0/711584582 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f714c00ccc0 con 0x7f715c10a3a0 2024-05-03T14:47:57.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:57.959+0000 7f7159600700 1 -- 172.21.15.60:0/711584582 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f714c010430 con 0x7f715c10a3a0 2024-05-03T14:47:57.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:57.959+0000 7f7162200700 1 -- 172.21.15.60:0/711584582 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f715c1ace30 con 0x7f715c10a3a0 2024-05-03T14:47:57.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:57.960+0000 7f7162200700 1 -- 172.21.15.60:0/711584582 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f715c06a190 con 0x7f715c10a3a0 2024-05-03T14:47:57.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:57.962+0000 7f7159600700 1 -- 172.21.15.60:0/711584582 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f714c004d80 con 0x7f715c10a3a0 2024-05-03T14:47:57.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:57.964+0000 7f7159600700 1 --2- 172.21.15.60:0/711584582 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f71440b2990 0x7f71440b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:47:57.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:57.964+0000 7f7159600700 1 -- 172.21.15.60:0/711584582 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f714c0e29e0 con 0x7f715c10a3a0 2024-05-03T14:47:57.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:57.964+0000 7f7160e00700 1 --2- 172.21.15.60:0/711584582 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f71440b2990 0x7f71440b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:47:57.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:57.965+0000 7f7160e00700 1 --2- 172.21.15.60:0/711584582 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f71440b2990 0x7f71440b4e50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7150009810 tx=0x7f7150019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:47:57.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:57.966+0000 7f7159600700 1 -- 172.21.15.60:0/711584582 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f714c054d70 con 0x7f715c10a3a0 2024-05-03T14:47:58.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:58.197+0000 7f7162200700 1 -- 172.21.15.60:0/711584582 --> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f715c1a1610 con 0x7f71440b2990 2024-05-03T14:47:58.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:58.227+0000 7f7159600700 1 -- 172.21.15.60:0/711584582 <== mgr.14152 v2:172.21.15.60:6800/3066142956 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340664 (secure 0 0 0) 0x7f715007a2d0 con 0x7f71440b2990 2024-05-03T14:47:58.233 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:47:58.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:58.236+0000 7f7162200700 1 -- 172.21.15.60:0/711584582 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f71440b2990 msgr2=0x7f71440b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:58.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:58.236+0000 7f7162200700 1 --2- 172.21.15.60:0/711584582 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f71440b2990 0x7f71440b4e50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7150009810 tx=0x7f7150019040 comp rx=0 tx=0).stop 2024-05-03T14:47:58.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:58.236+0000 7f7162200700 1 -- 172.21.15.60:0/711584582 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f715c10a3a0 msgr2=0x7f715c1a07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:47:58.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:58.236+0000 7f7162200700 1 --2- 172.21.15.60:0/711584582 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f715c10a3a0 0x7f715c1a07f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f714c00e9c0 tx=0x7f714c00ecd0 comp rx=0 tx=0).stop 2024-05-03T14:47:58.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:58.236+0000 7f7162200700 1 -- 172.21.15.60:0/711584582 shutdown_connections 2024-05-03T14:47:58.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:58.236+0000 7f7162200700 1 --2- 172.21.15.60:0/711584582 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f71440b2990 0x7f71440b4e50 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:58.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:58.236+0000 7f7162200700 1 --2- 172.21.15.60:0/711584582 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f715c10e8b0 0x7f715c1a4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:58.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:58.236+0000 7f7162200700 1 --2- 172.21.15.60:0/711584582 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f715c1099e0 0x7f715c1a02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:58.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:58.236+0000 7f7162200700 1 --2- 172.21.15.60:0/711584582 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f715c10a3a0 0x7f715c1a07f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:47:58.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:58.236+0000 7f7162200700 1 -- 172.21.15.60:0/711584582 >> 172.21.15.60:0/711584582 conn(0x7f715c079230 msgr2=0x7f715c0797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:47:58.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:58.236+0000 7f7162200700 1 -- 172.21.15.60:0/711584582 shutdown_connections 2024-05-03T14:47:58.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:47:58.236+0000 7f7162200700 1 -- 172.21.15.60:0/711584582 wait complete. 2024-05-03T14:47:58.241 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-05-03T14:47:59.059 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":435,"stamp":"2024-05-03T14:47:56.404476+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657592,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1287,"num_read_kb":743,"num_write":374,"num_write_kb":2292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":2952512,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":308,"ondisk_log_size":308,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361264,"kb_used_data":12104,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580560,"statfs":{"total":1151940427776,"available":1151570493440,"internally_reserved":0,"allocated":12394496,"data_stored":8127830,"data_compressed":496272,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19140,"internal_metadata":357414204},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":135,"num_read_kb":70,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006225"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110403+0000","last_change":"2024-05-03T14:44:31.452444+0000","last_active":"2024-05-03T14:46:06.110403+0000","last_peered":"2024-05-03T14:46:06.110403+0000","last_clean":"2024-05-03T14:46:06.110403+0000","last_became_active":"2024-05-03T14:44:31.451574+0000","last_became_peered":"2024-05-03T14:44:31.451574+0000","last_unstale":"2024-05-03T14:46:06.110403+0000","last_undegraded":"2024-05-03T14:46:06.110403+0000","last_fullsized":"2024-05-03T14:46:06.110403+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:14:31.009833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.590392+0000","last_change":"2024-05-03T14:44:31.461736+0000","last_active":"2024-05-03T14:46:05.590392+0000","last_peered":"2024-05-03T14:46:05.590392+0000","last_clean":"2024-05-03T14:46:05.590392+0000","last_became_active":"2024-05-03T14:44:31.457963+0000","last_became_peered":"2024-05-03T14:44:31.457963+0000","last_unstale":"2024-05-03T14:46:05.590392+0000","last_undegraded":"2024-05-03T14:46:05.590392+0000","last_fullsized":"2024-05-03T14:46:05.590392+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:59:16.029675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.590007+0000","last_change":"2024-05-03T14:44:31.457163+0000","last_active":"2024-05-03T14:46:05.590007+0000","last_peered":"2024-05-03T14:46:05.590007+0000","last_clean":"2024-05-03T14:46:05.590007+0000","last_became_active":"2024-05-03T14:44:31.455564+0000","last_became_peered":"2024-05-03T14:44:31.455564+0000","last_unstale":"2024-05-03T14:46:05.590007+0000","last_undegraded":"2024-05-03T14:46:05.590007+0000","last_fullsized":"2024-05-03T14:46:05.590007+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T16:14:00.761333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109879+0000","last_change":"2024-05-03T14:44:31.450623+0000","last_active":"2024-05-03T14:46:06.109879+0000","last_peered":"2024-05-03T14:46:06.109879+0000","last_clean":"2024-05-03T14:46:06.109879+0000","last_became_active":"2024-05-03T14:44:31.449944+0000","last_became_peered":"2024-05-03T14:44:31.449944+0000","last_unstale":"2024-05-03T14:46:06.109879+0000","last_undegraded":"2024-05-03T14:46:06.109879+0000","last_fullsized":"2024-05-03T14:46:06.109879+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:40:24.155342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.593956+0000","last_change":"2024-05-03T14:44:31.473624+0000","last_active":"2024-05-03T14:46:05.593956+0000","last_peered":"2024-05-03T14:46:05.593956+0000","last_clean":"2024-05-03T14:46:05.593956+0000","last_became_active":"2024-05-03T14:44:31.473339+0000","last_became_peered":"2024-05-03T14:44:31.473339+0000","last_unstale":"2024-05-03T14:46:05.593956+0000","last_undegraded":"2024-05-03T14:46:05.593956+0000","last_fullsized":"2024-05-03T14:46:05.593956+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:21:42.683546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.608139+0000","last_change":"2024-05-03T14:44:31.466037+0000","last_active":"2024-05-03T14:46:05.608139+0000","last_peered":"2024-05-03T14:46:05.608139+0000","last_clean":"2024-05-03T14:46:05.608139+0000","last_became_active":"2024-05-03T14:44:31.465202+0000","last_became_peered":"2024-05-03T14:44:31.465202+0000","last_unstale":"2024-05-03T14:46:05.608139+0000","last_undegraded":"2024-05-03T14:46:05.608139+0000","last_fullsized":"2024-05-03T14:46:05.608139+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:46:04.189133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.594286+0000","last_change":"2024-05-03T14:44:31.458708+0000","last_active":"2024-05-03T14:46:05.594286+0000","last_peered":"2024-05-03T14:46:05.594286+0000","last_clean":"2024-05-03T14:46:05.594286+0000","last_became_active":"2024-05-03T14:44:31.458292+0000","last_became_peered":"2024-05-03T14:44:31.458292+0000","last_unstale":"2024-05-03T14:46:05.594286+0000","last_undegraded":"2024-05-03T14:46:05.594286+0000","last_fullsized":"2024-05-03T14:46:05.594286+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:15:28.850661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.113349+0000","last_change":"2024-05-03T14:44:31.466051+0000","last_active":"2024-05-03T14:46:06.113349+0000","last_peered":"2024-05-03T14:46:06.113349+0000","last_clean":"2024-05-03T14:46:06.113349+0000","last_became_active":"2024-05-03T14:44:31.465620+0000","last_became_peered":"2024-05-03T14:44:31.465620+0000","last_unstale":"2024-05-03T14:46:06.113349+0000","last_undegraded":"2024-05-03T14:46:06.113349+0000","last_fullsized":"2024-05-03T14:46:06.113349+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:27:02.749377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109470+0000","last_change":"2024-05-03T14:44:31.451507+0000","last_active":"2024-05-03T14:46:06.109470+0000","last_peered":"2024-05-03T14:46:06.109470+0000","last_clean":"2024-05-03T14:46:06.109470+0000","last_became_active":"2024-05-03T14:44:31.451163+0000","last_became_peered":"2024-05-03T14:44:31.451163+0000","last_unstale":"2024-05-03T14:46:06.109470+0000","last_undegraded":"2024-05-03T14:46:06.109470+0000","last_fullsized":"2024-05-03T14:46:06.109470+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:56:50.415619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.608463+0000","last_change":"2024-05-03T14:45:05.421817+0000","last_active":"2024-05-03T14:46:05.608463+0000","last_peered":"2024-05-03T14:46:05.608463+0000","last_clean":"2024-05-03T14:46:05.608463+0000","last_became_active":"2024-05-03T14:45:05.421446+0000","last_became_peered":"2024-05-03T14:45:05.421446+0000","last_unstale":"2024-05-03T14:46:05.608463+0000","last_undegraded":"2024-05-03T14:46:05.608463+0000","last_fullsized":"2024-05-03T14:46:05.608463+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:31:36.867429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.594798+0000","last_change":"2024-05-03T14:44:31.454474+0000","last_active":"2024-05-03T14:46:05.594798+0000","last_peered":"2024-05-03T14:46:05.594798+0000","last_clean":"2024-05-03T14:46:05.594798+0000","last_became_active":"2024-05-03T14:44:31.454089+0000","last_became_peered":"2024-05-03T14:44:31.454089+0000","last_unstale":"2024-05-03T14:46:05.594798+0000","last_undegraded":"2024-05-03T14:46:05.594798+0000","last_fullsized":"2024-05-03T14:46:05.594798+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:54:59.739325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.592913+0000","last_change":"2024-05-03T14:45:05.617724+0000","last_active":"2024-05-03T14:46:05.592913+0000","last_peered":"2024-05-03T14:46:05.592913+0000","last_clean":"2024-05-03T14:46:05.592913+0000","last_became_active":"2024-05-03T14:45:05.616554+0000","last_became_peered":"2024-05-03T14:45:05.616554+0000","last_unstale":"2024-05-03T14:46:05.592913+0000","last_undegraded":"2024-05-03T14:46:05.592913+0000","last_fullsized":"2024-05-03T14:46:05.592913+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:24:20.890688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110669+0000","last_change":"2024-05-03T14:44:31.462676+0000","last_active":"2024-05-03T14:46:06.110669+0000","last_peered":"2024-05-03T14:46:06.110669+0000","last_clean":"2024-05-03T14:46:06.110669+0000","last_became_active":"2024-05-03T14:44:31.459326+0000","last_became_peered":"2024-05-03T14:44:31.459326+0000","last_unstale":"2024-05-03T14:46:06.110669+0000","last_undegraded":"2024-05-03T14:46:06.110669+0000","last_fullsized":"2024-05-03T14:46:06.110669+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:36:22.975112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.589768+0000","last_change":"2024-05-03T14:44:31.447665+0000","last_active":"2024-05-03T14:46:05.589768+0000","last_peered":"2024-05-03T14:46:05.589768+0000","last_clean":"2024-05-03T14:46:05.589768+0000","last_became_active":"2024-05-03T14:44:31.447204+0000","last_became_peered":"2024-05-03T14:44:31.447204+0000","last_unstale":"2024-05-03T14:46:05.589768+0000","last_undegraded":"2024-05-03T14:46:05.589768+0000","last_fullsized":"2024-05-03T14:46:05.589768+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:23:41.313986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.590246+0000","last_change":"2024-05-03T14:44:31.458869+0000","last_active":"2024-05-03T14:46:05.590246+0000","last_peered":"2024-05-03T14:46:05.590246+0000","last_clean":"2024-05-03T14:46:05.590246+0000","last_became_active":"2024-05-03T14:44:31.453157+0000","last_became_peered":"2024-05-03T14:44:31.453157+0000","last_unstale":"2024-05-03T14:46:05.590246+0000","last_undegraded":"2024-05-03T14:46:05.590246+0000","last_fullsized":"2024-05-03T14:46:05.590246+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:26:15.764522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.108585+0000","last_change":"2024-05-03T14:44:31.461320+0000","last_active":"2024-05-03T14:46:06.108585+0000","last_peered":"2024-05-03T14:46:06.108585+0000","last_clean":"2024-05-03T14:46:06.108585+0000","last_became_active":"2024-05-03T14:44:31.460935+0000","last_became_peered":"2024-05-03T14:44:31.460935+0000","last_unstale":"2024-05-03T14:46:06.108585+0000","last_undegraded":"2024-05-03T14:46:06.108585+0000","last_fullsized":"2024-05-03T14:46:06.108585+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:32:22.368935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110144+0000","last_change":"2024-05-03T14:44:31.452694+0000","last_active":"2024-05-03T14:46:06.110144+0000","last_peered":"2024-05-03T14:46:06.110144+0000","last_clean":"2024-05-03T14:46:06.110144+0000","last_became_active":"2024-05-03T14:44:31.452012+0000","last_became_peered":"2024-05-03T14:44:31.452012+0000","last_unstale":"2024-05-03T14:46:06.110144+0000","last_undegraded":"2024-05-03T14:46:06.110144+0000","last_fullsized":"2024-05-03T14:46:06.110144+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:59:34.721990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.591555+0000","last_change":"2024-05-03T14:44:31.461029+0000","last_active":"2024-05-03T14:46:05.591555+0000","last_peered":"2024-05-03T14:46:05.591555+0000","last_clean":"2024-05-03T14:46:05.591555+0000","last_became_active":"2024-05-03T14:44:31.460506+0000","last_became_peered":"2024-05-03T14:44:31.460506+0000","last_unstale":"2024-05-03T14:46:05.591555+0000","last_undegraded":"2024-05-03T14:46:05.591555+0000","last_fullsized":"2024-05-03T14:46:05.591555+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:09:23.903606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.591626+0000","last_change":"2024-05-03T14:44:31.467556+0000","last_active":"2024-05-03T14:46:05.591626+0000","last_peered":"2024-05-03T14:46:05.591626+0000","last_clean":"2024-05-03T14:46:05.591626+0000","last_became_active":"2024-05-03T14:44:31.467191+0000","last_became_peered":"2024-05-03T14:44:31.467191+0000","last_unstale":"2024-05-03T14:46:05.591626+0000","last_undegraded":"2024-05-03T14:46:05.591626+0000","last_fullsized":"2024-05-03T14:46:05.591626+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:33:42.980620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.591444+0000","last_change":"2024-05-03T14:45:05.618172+0000","last_active":"2024-05-03T14:46:05.591444+0000","last_peered":"2024-05-03T14:46:05.591444+0000","last_clean":"2024-05-03T14:46:05.591444+0000","last_became_active":"2024-05-03T14:45:05.617555+0000","last_became_peered":"2024-05-03T14:45:05.617555+0000","last_unstale":"2024-05-03T14:46:05.591444+0000","last_undegraded":"2024-05-03T14:46:05.591444+0000","last_fullsized":"2024-05-03T14:46:05.591444+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T16:44:40.610265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'13","reported_seq":194,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:47:51.361468+0000","last_change":"2024-05-03T14:46:06.129243+0000","last_active":"2024-05-03T14:47:51.361468+0000","last_peered":"2024-05-03T14:47:51.361468+0000","last_clean":"2024-05-03T14:47:51.361468+0000","last_became_active":"2024-05-03T14:46:06.112952+0000","last_became_peered":"2024-05-03T14:46:06.112952+0000","last_unstale":"2024-05-03T14:47:51.361468+0000","last_undegraded":"2024-05-03T14:47:51.361468+0000","last_fullsized":"2024-05-03T14:47:51.361468+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:12:22.401171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":114,"num_read_kb":114,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.113593+0000","last_change":"2024-05-03T14:44:31.467383+0000","last_active":"2024-05-03T14:46:06.113593+0000","last_peered":"2024-05-03T14:46:06.113593+0000","last_clean":"2024-05-03T14:46:06.113593+0000","last_became_active":"2024-05-03T14:44:31.466993+0000","last_became_peered":"2024-05-03T14:44:31.466993+0000","last_unstale":"2024-05-03T14:46:06.113593+0000","last_undegraded":"2024-05-03T14:46:06.113593+0000","last_fullsized":"2024-05-03T14:46:06.113593+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T16:43:50.237083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.108935+0000","last_change":"2024-05-03T14:44:44.583870+0000","last_active":"2024-05-03T14:46:06.108935+0000","last_peered":"2024-05-03T14:46:06.108935+0000","last_clean":"2024-05-03T14:46:06.108935+0000","last_became_active":"2024-05-03T14:44:44.583296+0000","last_became_peered":"2024-05-03T14:44:44.583296+0000","last_unstale":"2024-05-03T14:46:06.108935+0000","last_undegraded":"2024-05-03T14:46:06.108935+0000","last_fullsized":"2024-05-03T14:46:06.108935+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:55:54.345707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.115090+0000","last_change":"2024-05-03T14:44:27.415797+0000","last_active":"2024-05-03T14:46:06.115090+0000","last_peered":"2024-05-03T14:46:06.115090+0000","last_clean":"2024-05-03T14:46:06.115090+0000","last_became_active":"2024-05-03T14:44:27.414960+0000","last_became_peered":"2024-05-03T14:44:27.414960+0000","last_unstale":"2024-05-03T14:46:06.115090+0000","last_undegraded":"2024-05-03T14:46:06.115090+0000","last_fullsized":"2024-05-03T14:46:06.115090+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:15:54.374392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.590953+0000","last_change":"2024-05-03T14:44:31.470241+0000","last_active":"2024-05-03T14:46:05.590953+0000","last_peered":"2024-05-03T14:46:05.590953+0000","last_clean":"2024-05-03T14:46:05.590953+0000","last_became_active":"2024-05-03T14:44:31.469882+0000","last_became_peered":"2024-05-03T14:44:31.469882+0000","last_unstale":"2024-05-03T14:46:05.590953+0000","last_undegraded":"2024-05-03T14:46:05.590953+0000","last_fullsized":"2024-05-03T14:46:05.590953+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:40:43.970897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109107+0000","last_change":"2024-05-03T14:44:44.583776+0000","last_active":"2024-05-03T14:46:06.109107+0000","last_peered":"2024-05-03T14:46:06.109107+0000","last_clean":"2024-05-03T14:46:06.109107+0000","last_became_active":"2024-05-03T14:44:44.583117+0000","last_became_peered":"2024-05-03T14:44:44.583117+0000","last_unstale":"2024-05-03T14:46:06.109107+0000","last_undegraded":"2024-05-03T14:46:06.109107+0000","last_fullsized":"2024-05-03T14:46:06.109107+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:58:49.428720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.594222+0000","last_change":"2024-05-03T14:44:27.404806+0000","last_active":"2024-05-03T14:46:05.594222+0000","last_peered":"2024-05-03T14:46:05.594222+0000","last_clean":"2024-05-03T14:46:05.594222+0000","last_became_active":"2024-05-03T14:44:27.404415+0000","last_became_peered":"2024-05-03T14:44:27.404415+0000","last_unstale":"2024-05-03T14:46:05.594222+0000","last_undegraded":"2024-05-03T14:46:05.594222+0000","last_fullsized":"2024-05-03T14:46:05.594222+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:04:40.569591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.108885+0000","last_change":"2024-05-03T14:44:31.465714+0000","last_active":"2024-05-03T14:46:06.108885+0000","last_peered":"2024-05-03T14:46:06.108885+0000","last_clean":"2024-05-03T14:46:06.108885+0000","last_became_active":"2024-05-03T14:44:31.465264+0000","last_became_peered":"2024-05-03T14:44:31.465264+0000","last_unstale":"2024-05-03T14:46:06.108885+0000","last_undegraded":"2024-05-03T14:46:06.108885+0000","last_fullsized":"2024-05-03T14:46:06.108885+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:19:38.206564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109172+0000","last_change":"2024-05-03T14:44:44.583611+0000","last_active":"2024-05-03T14:46:06.109172+0000","last_peered":"2024-05-03T14:46:06.109172+0000","last_clean":"2024-05-03T14:46:06.109172+0000","last_became_active":"2024-05-03T14:44:44.583149+0000","last_became_peered":"2024-05-03T14:44:44.583149+0000","last_unstale":"2024-05-03T14:46:06.109172+0000","last_undegraded":"2024-05-03T14:46:06.109172+0000","last_fullsized":"2024-05-03T14:46:06.109172+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:25:23.550168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.594007+0000","last_change":"2024-05-03T14:44:27.405942+0000","last_active":"2024-05-03T14:46:05.594007+0000","last_peered":"2024-05-03T14:46:05.594007+0000","last_clean":"2024-05-03T14:46:05.594007+0000","last_became_active":"2024-05-03T14:44:27.405556+0000","last_became_peered":"2024-05-03T14:44:27.405556+0000","last_unstale":"2024-05-03T14:46:05.594007+0000","last_undegraded":"2024-05-03T14:46:05.594007+0000","last_fullsized":"2024-05-03T14:46:05.594007+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:16:07.798364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110336+0000","last_change":"2024-05-03T14:44:31.451421+0000","last_active":"2024-05-03T14:46:06.110336+0000","last_peered":"2024-05-03T14:46:06.110336+0000","last_clean":"2024-05-03T14:46:06.110336+0000","last_became_active":"2024-05-03T14:44:31.451072+0000","last_became_peered":"2024-05-03T14:44:31.451072+0000","last_unstale":"2024-05-03T14:46:06.110336+0000","last_undegraded":"2024-05-03T14:46:06.110336+0000","last_fullsized":"2024-05-03T14:46:06.110336+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:12:31.040518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.591429+0000","last_change":"2024-05-03T14:44:44.567926+0000","last_active":"2024-05-03T14:46:05.591429+0000","last_peered":"2024-05-03T14:46:05.591429+0000","last_clean":"2024-05-03T14:46:05.591429+0000","last_became_active":"2024-05-03T14:44:44.567494+0000","last_became_peered":"2024-05-03T14:44:44.567494+0000","last_unstale":"2024-05-03T14:46:05.591429+0000","last_undegraded":"2024-05-03T14:46:05.591429+0000","last_fullsized":"2024-05-03T14:46:05.591429+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:51:11.144925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'5","reported_seq":175,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:47:52.713343+0000","last_change":"2024-05-03T14:45:05.614704+0000","last_active":"2024-05-03T14:47:52.713343+0000","last_peered":"2024-05-03T14:47:52.713343+0000","last_clean":"2024-05-03T14:47:52.713343+0000","last_became_active":"2024-05-03T14:45:05.614173+0000","last_became_peered":"2024-05-03T14:45:05.614173+0000","last_unstale":"2024-05-03T14:47:52.713343+0000","last_undegraded":"2024-05-03T14:47:52.713343+0000","last_fullsized":"2024-05-03T14:47:52.713343+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:50:42.956239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.593723+0000","last_change":"2024-05-03T14:44:31.464666+0000","last_active":"2024-05-03T14:46:05.593723+0000","last_peered":"2024-05-03T14:46:05.593723+0000","last_clean":"2024-05-03T14:46:05.593723+0000","last_became_active":"2024-05-03T14:44:31.464298+0000","last_became_peered":"2024-05-03T14:44:31.464298+0000","last_unstale":"2024-05-03T14:46:05.593723+0000","last_undegraded":"2024-05-03T14:46:05.593723+0000","last_fullsized":"2024-05-03T14:46:05.593723+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:26:35.425908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.591342+0000","last_change":"2024-05-03T14:44:44.580268+0000","last_active":"2024-05-03T14:46:05.591342+0000","last_peered":"2024-05-03T14:46:05.591342+0000","last_clean":"2024-05-03T14:46:05.591342+0000","last_became_active":"2024-05-03T14:44:44.579617+0000","last_became_peered":"2024-05-03T14:44:44.579617+0000","last_unstale":"2024-05-03T14:46:05.591342+0000","last_undegraded":"2024-05-03T14:46:05.591342+0000","last_fullsized":"2024-05-03T14:46:05.591342+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:16:25.580698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.608165+0000","last_change":"2024-05-03T14:44:27.418578+0000","last_active":"2024-05-03T14:46:05.608165+0000","last_peered":"2024-05-03T14:46:05.608165+0000","last_clean":"2024-05-03T14:46:05.608165+0000","last_became_active":"2024-05-03T14:44:27.418206+0000","last_became_peered":"2024-05-03T14:44:27.418206+0000","last_unstale":"2024-05-03T14:46:05.608165+0000","last_undegraded":"2024-05-03T14:46:05.608165+0000","last_fullsized":"2024-05-03T14:46:05.608165+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:48:47.085451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110237+0000","last_change":"2024-05-03T14:44:31.467353+0000","last_active":"2024-05-03T14:46:06.110237+0000","last_peered":"2024-05-03T14:46:06.110237+0000","last_clean":"2024-05-03T14:46:06.110237+0000","last_became_active":"2024-05-03T14:44:31.466606+0000","last_became_peered":"2024-05-03T14:44:31.466606+0000","last_unstale":"2024-05-03T14:46:06.110237+0000","last_undegraded":"2024-05-03T14:46:06.110237+0000","last_fullsized":"2024-05-03T14:46:06.110237+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T14:50:22.624015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110027+0000","last_change":"2024-05-03T14:44:44.567367+0000","last_active":"2024-05-03T14:46:06.110027+0000","last_peered":"2024-05-03T14:46:06.110027+0000","last_clean":"2024-05-03T14:46:06.110027+0000","last_became_active":"2024-05-03T14:44:44.566916+0000","last_became_peered":"2024-05-03T14:44:44.566916+0000","last_unstale":"2024-05-03T14:46:06.110027+0000","last_undegraded":"2024-05-03T14:46:06.110027+0000","last_fullsized":"2024-05-03T14:46:06.110027+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:07:39.966644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110318+0000","last_change":"2024-05-03T14:44:27.424704+0000","last_active":"2024-05-03T14:46:06.110318+0000","last_peered":"2024-05-03T14:46:06.110318+0000","last_clean":"2024-05-03T14:46:06.110318+0000","last_became_active":"2024-05-03T14:44:27.424231+0000","last_became_peered":"2024-05-03T14:44:27.424231+0000","last_unstale":"2024-05-03T14:46:06.110318+0000","last_undegraded":"2024-05-03T14:46:06.110318+0000","last_fullsized":"2024-05-03T14:46:06.110318+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:00:22.076458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.113192+0000","last_change":"2024-05-03T14:44:31.474599+0000","last_active":"2024-05-03T14:46:06.113192+0000","last_peered":"2024-05-03T14:46:06.113192+0000","last_clean":"2024-05-03T14:46:06.113192+0000","last_became_active":"2024-05-03T14:44:31.473970+0000","last_became_peered":"2024-05-03T14:44:31.473970+0000","last_unstale":"2024-05-03T14:46:06.113192+0000","last_undegraded":"2024-05-03T14:46:06.113192+0000","last_fullsized":"2024-05-03T14:46:06.113192+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:20:14.259534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.590281+0000","last_change":"2024-05-03T14:44:44.580126+0000","last_active":"2024-05-03T14:46:05.590281+0000","last_peered":"2024-05-03T14:46:05.590281+0000","last_clean":"2024-05-03T14:46:05.590281+0000","last_became_active":"2024-05-03T14:44:44.578708+0000","last_became_peered":"2024-05-03T14:44:44.578708+0000","last_unstale":"2024-05-03T14:46:05.590281+0000","last_undegraded":"2024-05-03T14:46:05.590281+0000","last_fullsized":"2024-05-03T14:46:05.590281+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:42:43.279786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109232+0000","last_change":"2024-05-03T14:44:27.405332+0000","last_active":"2024-05-03T14:46:06.109232+0000","last_peered":"2024-05-03T14:46:06.109232+0000","last_clean":"2024-05-03T14:46:06.109232+0000","last_became_active":"2024-05-03T14:44:27.404994+0000","last_became_peered":"2024-05-03T14:44:27.404994+0000","last_unstale":"2024-05-03T14:46:06.109232+0000","last_undegraded":"2024-05-03T14:46:06.109232+0000","last_fullsized":"2024-05-03T14:46:06.109232+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:18:23.716014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.590839+0000","last_change":"2024-05-03T14:44:31.470396+0000","last_active":"2024-05-03T14:46:05.590839+0000","last_peered":"2024-05-03T14:46:05.590839+0000","last_clean":"2024-05-03T14:46:05.590839+0000","last_became_active":"2024-05-03T14:44:31.467973+0000","last_became_peered":"2024-05-03T14:44:31.467973+0000","last_unstale":"2024-05-03T14:46:05.590839+0000","last_undegraded":"2024-05-03T14:46:05.590839+0000","last_fullsized":"2024-05-03T14:46:05.590839+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:44:21.266589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.113155+0000","last_change":"2024-05-03T14:44:44.582678+0000","last_active":"2024-05-03T14:46:06.113155+0000","last_peered":"2024-05-03T14:46:06.113155+0000","last_clean":"2024-05-03T14:46:06.113155+0000","last_became_active":"2024-05-03T14:44:44.582155+0000","last_became_peered":"2024-05-03T14:44:44.582155+0000","last_unstale":"2024-05-03T14:46:06.113155+0000","last_undegraded":"2024-05-03T14:46:06.113155+0000","last_fullsized":"2024-05-03T14:46:06.113155+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T15:21:05.986627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":285,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:47:54.715360+0000","last_change":"2024-05-03T14:44:27.415880+0000","last_active":"2024-05-03T14:47:54.715360+0000","last_peered":"2024-05-03T14:47:54.715360+0000","last_clean":"2024-05-03T14:47:54.715360+0000","last_became_active":"2024-05-03T14:44:27.415354+0000","last_became_peered":"2024-05-03T14:44:27.415354+0000","last_unstale":"2024-05-03T14:47:54.715360+0000","last_undegraded":"2024-05-03T14:47:54.715360+0000","last_fullsized":"2024-05-03T14:47:54.715360+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T16:55:41.808314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":236,"num_read_kb":236,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109557+0000","last_change":"2024-05-03T14:44:31.462685+0000","last_active":"2024-05-03T14:46:06.109557+0000","last_peered":"2024-05-03T14:46:06.109557+0000","last_clean":"2024-05-03T14:46:06.109557+0000","last_became_active":"2024-05-03T14:44:31.462325+0000","last_became_peered":"2024-05-03T14:44:31.462325+0000","last_unstale":"2024-05-03T14:46:06.109557+0000","last_undegraded":"2024-05-03T14:46:06.109557+0000","last_fullsized":"2024-05-03T14:46:06.109557+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:59:54.715648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.590321+0000","last_change":"2024-05-03T14:44:44.573855+0000","last_active":"2024-05-03T14:46:05.590321+0000","last_peered":"2024-05-03T14:46:05.590321+0000","last_clean":"2024-05-03T14:46:05.590321+0000","last_became_active":"2024-05-03T14:44:44.573314+0000","last_became_peered":"2024-05-03T14:44:44.573314+0000","last_unstale":"2024-05-03T14:46:05.590321+0000","last_undegraded":"2024-05-03T14:46:05.590321+0000","last_fullsized":"2024-05-03T14:46:05.590321+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:04:24.282503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.594887+0000","last_change":"2024-05-03T14:45:05.613037+0000","last_active":"2024-05-03T14:46:05.594887+0000","last_peered":"2024-05-03T14:46:05.594887+0000","last_clean":"2024-05-03T14:46:05.594887+0000","last_became_active":"2024-05-03T14:45:05.612581+0000","last_became_peered":"2024-05-03T14:45:05.612581+0000","last_unstale":"2024-05-03T14:46:05.594887+0000","last_undegraded":"2024-05-03T14:46:05.594887+0000","last_fullsized":"2024-05-03T14:46:05.594887+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:34:07.365447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.590847+0000","last_change":"2024-05-03T14:44:31.451912+0000","last_active":"2024-05-03T14:46:05.590847+0000","last_peered":"2024-05-03T14:46:05.590847+0000","last_clean":"2024-05-03T14:46:05.590847+0000","last_became_active":"2024-05-03T14:44:31.449029+0000","last_became_peered":"2024-05-03T14:44:31.449029+0000","last_unstale":"2024-05-03T14:46:05.590847+0000","last_undegraded":"2024-05-03T14:46:05.590847+0000","last_fullsized":"2024-05-03T14:46:05.590847+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:27:13.018719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109453+0000","last_change":"2024-05-03T14:44:44.577001+0000","last_active":"2024-05-03T14:46:06.109453+0000","last_peered":"2024-05-03T14:46:06.109453+0000","last_clean":"2024-05-03T14:46:06.109453+0000","last_became_active":"2024-05-03T14:44:44.576224+0000","last_became_peered":"2024-05-03T14:44:44.576224+0000","last_unstale":"2024-05-03T14:46:06.109453+0000","last_undegraded":"2024-05-03T14:46:06.109453+0000","last_fullsized":"2024-05-03T14:46:06.109453+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T15:15:30.497801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110288+0000","last_change":"2024-05-03T14:44:27.402496+0000","last_active":"2024-05-03T14:46:06.110288+0000","last_peered":"2024-05-03T14:46:06.110288+0000","last_clean":"2024-05-03T14:46:06.110288+0000","last_became_active":"2024-05-03T14:44:27.402062+0000","last_became_peered":"2024-05-03T14:44:27.402062+0000","last_unstale":"2024-05-03T14:46:06.110288+0000","last_undegraded":"2024-05-03T14:46:06.110288+0000","last_fullsized":"2024-05-03T14:46:06.110288+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:47:10.090902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.108933+0000","last_change":"2024-05-03T14:44:31.452654+0000","last_active":"2024-05-03T14:46:06.108933+0000","last_peered":"2024-05-03T14:46:06.108933+0000","last_clean":"2024-05-03T14:46:06.108933+0000","last_became_active":"2024-05-03T14:44:31.452305+0000","last_became_peered":"2024-05-03T14:44:31.452305+0000","last_unstale":"2024-05-03T14:46:06.108933+0000","last_undegraded":"2024-05-03T14:46:06.108933+0000","last_fullsized":"2024-05-03T14:46:06.108933+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:20:45.816113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.591228+0000","last_change":"2024-05-03T14:44:44.576004+0000","last_active":"2024-05-03T14:46:05.591228+0000","last_peered":"2024-05-03T14:46:05.591228+0000","last_clean":"2024-05-03T14:46:05.591228+0000","last_became_active":"2024-05-03T14:44:44.572486+0000","last_became_peered":"2024-05-03T14:44:44.572486+0000","last_unstale":"2024-05-03T14:46:05.591228+0000","last_undegraded":"2024-05-03T14:46:05.591228+0000","last_fullsized":"2024-05-03T14:46:05.591228+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T15:11:51.125404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'38","reported_seq":329,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:47:54.715378+0000","last_change":"2024-05-03T14:44:27.405937+0000","last_active":"2024-05-03T14:47:54.715378+0000","last_peered":"2024-05-03T14:47:54.715378+0000","last_clean":"2024-05-03T14:47:54.715378+0000","last_became_active":"2024-05-03T14:44:27.405511+0000","last_became_peered":"2024-05-03T14:44:27.405511+0000","last_unstale":"2024-05-03T14:47:54.715378+0000","last_undegraded":"2024-05-03T14:47:54.715378+0000","last_fullsized":"2024-05-03T14:47:54.715378+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:00:56.824015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60568,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":246,"num_read_kb":182,"num_write":40,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.594109+0000","last_change":"2024-05-03T14:44:31.448712+0000","last_active":"2024-05-03T14:46:05.594109+0000","last_peered":"2024-05-03T14:46:05.594109+0000","last_clean":"2024-05-03T14:46:05.594109+0000","last_became_active":"2024-05-03T14:44:31.448346+0000","last_became_peered":"2024-05-03T14:44:31.448346+0000","last_unstale":"2024-05-03T14:46:05.594109+0000","last_undegraded":"2024-05-03T14:46:05.594109+0000","last_fullsized":"2024-05-03T14:46:05.594109+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T16:32:40.770487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110659+0000","last_change":"2024-05-03T14:44:44.585600+0000","last_active":"2024-05-03T14:46:06.110659+0000","last_peered":"2024-05-03T14:46:06.110659+0000","last_clean":"2024-05-03T14:46:06.110659+0000","last_became_active":"2024-05-03T14:44:44.585221+0000","last_became_peered":"2024-05-03T14:44:44.585221+0000","last_unstale":"2024-05-03T14:46:06.110659+0000","last_undegraded":"2024-05-03T14:46:06.110659+0000","last_fullsized":"2024-05-03T14:46:06.110659+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:28:45.804470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":56,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.591475+0000","last_change":"2024-05-03T14:44:27.416524+0000","last_active":"2024-05-03T14:46:05.591475+0000","last_peered":"2024-05-03T14:46:05.591475+0000","last_clean":"2024-05-03T14:46:05.591475+0000","last_became_active":"2024-05-03T14:44:27.416211+0000","last_became_peered":"2024-05-03T14:44:27.416211+0000","last_unstale":"2024-05-03T14:46:05.591475+0000","last_undegraded":"2024-05-03T14:46:05.591475+0000","last_fullsized":"2024-05-03T14:46:05.591475+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:00:26.361267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.111079+0000","last_change":"2024-05-03T14:44:31.468973+0000","last_active":"2024-05-03T14:46:06.111079+0000","last_peered":"2024-05-03T14:46:06.111079+0000","last_clean":"2024-05-03T14:46:06.111079+0000","last_became_active":"2024-05-03T14:44:31.468547+0000","last_became_peered":"2024-05-03T14:44:31.468547+0000","last_unstale":"2024-05-03T14:46:06.111079+0000","last_undegraded":"2024-05-03T14:46:06.111079+0000","last_fullsized":"2024-05-03T14:46:06.111079+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:16:16.223859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109343+0000","last_change":"2024-05-03T14:44:44.579157+0000","last_active":"2024-05-03T14:46:06.109343+0000","last_peered":"2024-05-03T14:46:06.109343+0000","last_clean":"2024-05-03T14:46:06.109343+0000","last_became_active":"2024-05-03T14:44:44.578718+0000","last_became_peered":"2024-05-03T14:44:44.578718+0000","last_unstale":"2024-05-03T14:46:06.109343+0000","last_undegraded":"2024-05-03T14:46:06.109343+0000","last_fullsized":"2024-05-03T14:46:06.109343+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:03:31.574958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110085+0000","last_change":"2024-05-03T14:44:27.415290+0000","last_active":"2024-05-03T14:46:06.110085+0000","last_peered":"2024-05-03T14:46:06.110085+0000","last_clean":"2024-05-03T14:46:06.110085+0000","last_became_active":"2024-05-03T14:44:27.414930+0000","last_became_peered":"2024-05-03T14:44:27.414930+0000","last_unstale":"2024-05-03T14:46:06.110085+0000","last_undegraded":"2024-05-03T14:46:06.110085+0000","last_fullsized":"2024-05-03T14:46:06.110085+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:06:40.675902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.594402+0000","last_change":"2024-05-03T14:44:31.464631+0000","last_active":"2024-05-03T14:46:05.594402+0000","last_peered":"2024-05-03T14:46:05.594402+0000","last_clean":"2024-05-03T14:46:05.594402+0000","last_became_active":"2024-05-03T14:44:31.464325+0000","last_became_peered":"2024-05-03T14:44:31.464325+0000","last_unstale":"2024-05-03T14:46:05.594402+0000","last_undegraded":"2024-05-03T14:46:05.594402+0000","last_fullsized":"2024-05-03T14:46:05.594402+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:09:39.140391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.591385+0000","last_change":"2024-05-03T14:44:44.583495+0000","last_active":"2024-05-03T14:46:05.591385+0000","last_peered":"2024-05-03T14:46:05.591385+0000","last_clean":"2024-05-03T14:46:05.591385+0000","last_became_active":"2024-05-03T14:44:44.582971+0000","last_became_peered":"2024-05-03T14:44:44.582971+0000","last_unstale":"2024-05-03T14:46:05.591385+0000","last_undegraded":"2024-05-03T14:46:05.591385+0000","last_fullsized":"2024-05-03T14:46:05.591385+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:23:12.086610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"87'5","reported_seq":168,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:47:51.792024+0000","last_change":"2024-05-03T14:44:27.403319+0000","last_active":"2024-05-03T14:47:51.792024+0000","last_peered":"2024-05-03T14:47:51.792024+0000","last_clean":"2024-05-03T14:47:51.792024+0000","last_became_active":"2024-05-03T14:44:27.402936+0000","last_became_peered":"2024-05-03T14:44:27.402936+0000","last_unstale":"2024-05-03T14:47:51.792024+0000","last_undegraded":"2024-05-03T14:47:51.792024+0000","last_fullsized":"2024-05-03T14:47:51.792024+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:06:24.851356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":119,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.594278+0000","last_change":"2024-05-03T14:44:31.453514+0000","last_active":"2024-05-03T14:46:05.594278+0000","last_peered":"2024-05-03T14:46:05.594278+0000","last_clean":"2024-05-03T14:46:05.594278+0000","last_became_active":"2024-05-03T14:44:31.452755+0000","last_became_peered":"2024-05-03T14:44:31.452755+0000","last_unstale":"2024-05-03T14:46:05.594278+0000","last_undegraded":"2024-05-03T14:46:05.594278+0000","last_fullsized":"2024-05-03T14:46:05.594278+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:29:45.095487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.113886+0000","last_change":"2024-05-03T14:44:44.587105+0000","last_active":"2024-05-03T14:46:06.113886+0000","last_peered":"2024-05-03T14:46:06.113886+0000","last_clean":"2024-05-03T14:46:06.113886+0000","last_became_active":"2024-05-03T14:44:44.586669+0000","last_became_peered":"2024-05-03T14:44:44.586669+0000","last_unstale":"2024-05-03T14:46:06.113886+0000","last_undegraded":"2024-05-03T14:46:06.113886+0000","last_fullsized":"2024-05-03T14:46:06.113886+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:18:16.364544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":63,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.593664+0000","last_change":"2024-05-03T14:44:27.412481+0000","last_active":"2024-05-03T14:46:05.593664+0000","last_peered":"2024-05-03T14:46:05.593664+0000","last_clean":"2024-05-03T14:46:05.593664+0000","last_became_active":"2024-05-03T14:44:27.412146+0000","last_became_peered":"2024-05-03T14:44:27.412146+0000","last_unstale":"2024-05-03T14:46:05.593664+0000","last_undegraded":"2024-05-03T14:46:05.593664+0000","last_fullsized":"2024-05-03T14:46:05.593664+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T16:03:40.205892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109800+0000","last_change":"2024-05-03T14:44:31.453208+0000","last_active":"2024-05-03T14:46:06.109800+0000","last_peered":"2024-05-03T14:46:06.109800+0000","last_clean":"2024-05-03T14:46:06.109800+0000","last_became_active":"2024-05-03T14:44:31.452788+0000","last_became_peered":"2024-05-03T14:44:31.452788+0000","last_unstale":"2024-05-03T14:46:06.109800+0000","last_undegraded":"2024-05-03T14:46:06.109800+0000","last_fullsized":"2024-05-03T14:46:06.109800+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:43:31.554510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109726+0000","last_change":"2024-05-03T14:44:44.582993+0000","last_active":"2024-05-03T14:46:06.109726+0000","last_peered":"2024-05-03T14:46:06.109726+0000","last_clean":"2024-05-03T14:46:06.109726+0000","last_became_active":"2024-05-03T14:44:44.582511+0000","last_became_peered":"2024-05-03T14:44:44.582511+0000","last_unstale":"2024-05-03T14:46:06.109726+0000","last_undegraded":"2024-05-03T14:46:06.109726+0000","last_fullsized":"2024-05-03T14:46:06.109726+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:22:26.380344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.594354+0000","last_change":"2024-05-03T14:44:27.401742+0000","last_active":"2024-05-03T14:46:05.594354+0000","last_peered":"2024-05-03T14:46:05.594354+0000","last_clean":"2024-05-03T14:46:05.594354+0000","last_became_active":"2024-05-03T14:44:27.401347+0000","last_became_peered":"2024-05-03T14:44:27.401347+0000","last_unstale":"2024-05-03T14:46:05.594354+0000","last_undegraded":"2024-05-03T14:46:05.594354+0000","last_fullsized":"2024-05-03T14:46:05.594354+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T15:39:41.144999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109391+0000","last_change":"2024-05-03T14:44:31.453480+0000","last_active":"2024-05-03T14:46:06.109391+0000","last_peered":"2024-05-03T14:46:06.109391+0000","last_clean":"2024-05-03T14:46:06.109391+0000","last_became_active":"2024-05-03T14:44:31.453171+0000","last_became_peered":"2024-05-03T14:44:31.453171+0000","last_unstale":"2024-05-03T14:46:06.109391+0000","last_undegraded":"2024-05-03T14:46:06.109391+0000","last_fullsized":"2024-05-03T14:46:06.109391+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:30:16.567938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.593660+0000","last_change":"2024-05-03T14:44:45.208587+0000","last_active":"2024-05-03T14:46:05.593660+0000","last_peered":"2024-05-03T14:46:05.593660+0000","last_clean":"2024-05-03T14:46:05.593660+0000","last_became_active":"2024-05-03T14:44:45.207987+0000","last_became_peered":"2024-05-03T14:44:45.207987+0000","last_unstale":"2024-05-03T14:46:05.593660+0000","last_undegraded":"2024-05-03T14:46:05.593660+0000","last_fullsized":"2024-05-03T14:46:05.593660+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:22:14.854390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.593134+0000","last_change":"2024-05-03T14:44:27.421333+0000","last_active":"2024-05-03T14:46:05.593134+0000","last_peered":"2024-05-03T14:46:05.593134+0000","last_clean":"2024-05-03T14:46:05.593134+0000","last_became_active":"2024-05-03T14:44:27.419745+0000","last_became_peered":"2024-05-03T14:44:27.419745+0000","last_unstale":"2024-05-03T14:46:05.593134+0000","last_undegraded":"2024-05-03T14:46:05.593134+0000","last_fullsized":"2024-05-03T14:46:05.593134+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:40:57.258260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.608152+0000","last_change":"2024-05-03T14:44:31.467871+0000","last_active":"2024-05-03T14:46:05.608152+0000","last_peered":"2024-05-03T14:46:05.608152+0000","last_clean":"2024-05-03T14:46:05.608152+0000","last_became_active":"2024-05-03T14:44:31.467408+0000","last_became_peered":"2024-05-03T14:44:31.467408+0000","last_unstale":"2024-05-03T14:46:05.608152+0000","last_undegraded":"2024-05-03T14:46:05.608152+0000","last_fullsized":"2024-05-03T14:46:05.608152+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:19:24.843591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110575+0000","last_change":"2024-05-03T14:44:44.583116+0000","last_active":"2024-05-03T14:46:06.110575+0000","last_peered":"2024-05-03T14:46:06.110575+0000","last_clean":"2024-05-03T14:46:06.110575+0000","last_became_active":"2024-05-03T14:44:44.581838+0000","last_became_peered":"2024-05-03T14:44:44.581838+0000","last_unstale":"2024-05-03T14:46:06.110575+0000","last_undegraded":"2024-05-03T14:46:06.110575+0000","last_fullsized":"2024-05-03T14:46:06.110575+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:14:24.608777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'5","reported_seq":169,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:47:54.370260+0000","last_change":"2024-05-03T14:44:27.416068+0000","last_active":"2024-05-03T14:47:54.370260+0000","last_peered":"2024-05-03T14:47:54.370260+0000","last_clean":"2024-05-03T14:47:54.370260+0000","last_became_active":"2024-05-03T14:44:27.415574+0000","last_became_peered":"2024-05-03T14:44:27.415574+0000","last_unstale":"2024-05-03T14:47:54.370260+0000","last_undegraded":"2024-05-03T14:47:54.370260+0000","last_fullsized":"2024-05-03T14:47:54.370260+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T15:31:40.186754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.608212+0000","last_change":"2024-05-03T14:44:31.453935+0000","last_active":"2024-05-03T14:46:05.608212+0000","last_peered":"2024-05-03T14:46:05.608212+0000","last_clean":"2024-05-03T14:46:05.608212+0000","last_became_active":"2024-05-03T14:44:31.453565+0000","last_became_peered":"2024-05-03T14:44:31.453565+0000","last_unstale":"2024-05-03T14:46:05.608212+0000","last_undegraded":"2024-05-03T14:46:05.608212+0000","last_fullsized":"2024-05-03T14:46:05.608212+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:34:04.855830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.111449+0000","last_change":"2024-05-03T14:44:45.210274+0000","last_active":"2024-05-03T14:46:06.111449+0000","last_peered":"2024-05-03T14:46:06.111449+0000","last_clean":"2024-05-03T14:46:06.111449+0000","last_became_active":"2024-05-03T14:44:45.208951+0000","last_became_peered":"2024-05-03T14:44:45.208951+0000","last_unstale":"2024-05-03T14:46:06.111449+0000","last_undegraded":"2024-05-03T14:46:06.111449+0000","last_fullsized":"2024-05-03T14:46:06.111449+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:56:30.953019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109801+0000","last_change":"2024-05-03T14:44:31.449470+0000","last_active":"2024-05-03T14:46:06.109801+0000","last_peered":"2024-05-03T14:46:06.109801+0000","last_clean":"2024-05-03T14:46:06.109801+0000","last_became_active":"2024-05-03T14:44:31.449031+0000","last_became_peered":"2024-05-03T14:44:31.449031+0000","last_unstale":"2024-05-03T14:46:06.109801+0000","last_undegraded":"2024-05-03T14:46:06.109801+0000","last_fullsized":"2024-05-03T14:46:06.109801+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T14:51:54.576685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":169,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:47:53.795883+0000","last_change":"2024-05-03T14:44:27.422378+0000","last_active":"2024-05-03T14:47:53.795883+0000","last_peered":"2024-05-03T14:47:53.795883+0000","last_clean":"2024-05-03T14:47:53.795883+0000","last_became_active":"2024-05-03T14:44:27.421988+0000","last_became_peered":"2024-05-03T14:44:27.421988+0000","last_unstale":"2024-05-03T14:47:53.795883+0000","last_undegraded":"2024-05-03T14:47:53.795883+0000","last_fullsized":"2024-05-03T14:47:53.795883+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:20:10.371418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":119,"num_read_kb":119,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:09.223079+0000","last_change":"2024-05-03T14:44:44.576084+0000","last_active":"2024-05-03T14:46:09.223079+0000","last_peered":"2024-05-03T14:46:09.223079+0000","last_clean":"2024-05-03T14:46:09.223079+0000","last_became_active":"2024-05-03T14:44:44.575633+0000","last_became_peered":"2024-05-03T14:44:44.575633+0000","last_unstale":"2024-05-03T14:46:09.223079+0000","last_undegraded":"2024-05-03T14:46:09.223079+0000","last_fullsized":"2024-05-03T14:46:09.223079+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:35:38.570413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.113766+0000","last_change":"2024-05-03T14:44:31.467024+0000","last_active":"2024-05-03T14:46:06.113766+0000","last_peered":"2024-05-03T14:46:06.113766+0000","last_clean":"2024-05-03T14:46:06.113766+0000","last_became_active":"2024-05-03T14:44:31.466545+0000","last_became_peered":"2024-05-03T14:44:31.466545+0000","last_unstale":"2024-05-03T14:46:06.113766+0000","last_undegraded":"2024-05-03T14:46:06.113766+0000","last_fullsized":"2024-05-03T14:46:06.113766+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:34:41.289638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109321+0000","last_change":"2024-05-03T14:44:27.410463+0000","last_active":"2024-05-03T14:46:06.109321+0000","last_peered":"2024-05-03T14:46:06.109321+0000","last_clean":"2024-05-03T14:46:06.109321+0000","last_became_active":"2024-05-03T14:44:27.410017+0000","last_became_peered":"2024-05-03T14:44:27.410017+0000","last_unstale":"2024-05-03T14:46:06.109321+0000","last_undegraded":"2024-05-03T14:46:06.109321+0000","last_fullsized":"2024-05-03T14:46:06.109321+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T16:44:40.376214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":120,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.594056+0000","last_change":"2024-05-03T14:42:51.347913+0000","last_active":"2024-05-03T14:46:05.594056+0000","last_peered":"2024-05-03T14:46:05.594056+0000","last_clean":"2024-05-03T14:46:05.594056+0000","last_became_active":"2024-05-03T14:42:51.341302+0000","last_became_peered":"2024-05-03T14:42:51.341302+0000","last_unstale":"2024-05-03T14:46:05.594056+0000","last_undegraded":"2024-05-03T14:46:05.594056+0000","last_fullsized":"2024-05-03T14:46:05.594056+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:38:55.096374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:38:55.096374+0000","last_clean_scrub_stamp":"2024-05-03T14:38:55.096374+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:35:46.708686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110588+0000","last_change":"2024-05-03T14:44:45.209369+0000","last_active":"2024-05-03T14:46:06.110588+0000","last_peered":"2024-05-03T14:46:06.110588+0000","last_clean":"2024-05-03T14:46:06.110588+0000","last_became_active":"2024-05-03T14:44:45.208835+0000","last_became_peered":"2024-05-03T14:44:45.208835+0000","last_unstale":"2024-05-03T14:46:06.110588+0000","last_undegraded":"2024-05-03T14:46:06.110588+0000","last_fullsized":"2024-05-03T14:46:06.110588+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:35:35.343030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110001+0000","last_change":"2024-05-03T14:44:31.462199+0000","last_active":"2024-05-03T14:46:06.110001+0000","last_peered":"2024-05-03T14:46:06.110001+0000","last_clean":"2024-05-03T14:46:06.110001+0000","last_became_active":"2024-05-03T14:44:31.460972+0000","last_became_peered":"2024-05-03T14:44:31.460972+0000","last_unstale":"2024-05-03T14:46:06.110001+0000","last_undegraded":"2024-05-03T14:46:06.110001+0000","last_fullsized":"2024-05-03T14:46:06.110001+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:21:49.619023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.594437+0000","last_change":"2024-05-03T14:44:27.403303+0000","last_active":"2024-05-03T14:46:05.594437+0000","last_peered":"2024-05-03T14:46:05.594437+0000","last_clean":"2024-05-03T14:46:05.594437+0000","last_became_active":"2024-05-03T14:44:27.402870+0000","last_became_peered":"2024-05-03T14:44:27.402870+0000","last_unstale":"2024-05-03T14:46:05.594437+0000","last_undegraded":"2024-05-03T14:46:05.594437+0000","last_fullsized":"2024-05-03T14:46:05.594437+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:57:23.209298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109302+0000","last_change":"2024-05-03T14:44:44.579532+0000","last_active":"2024-05-03T14:46:06.109302+0000","last_peered":"2024-05-03T14:46:06.109302+0000","last_clean":"2024-05-03T14:46:06.109302+0000","last_became_active":"2024-05-03T14:44:44.579052+0000","last_became_peered":"2024-05-03T14:44:44.579052+0000","last_unstale":"2024-05-03T14:46:06.109302+0000","last_undegraded":"2024-05-03T14:46:06.109302+0000","last_fullsized":"2024-05-03T14:46:06.109302+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:07:43.567880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109553+0000","last_change":"2024-05-03T14:45:05.422044+0000","last_active":"2024-05-03T14:46:06.109553+0000","last_peered":"2024-05-03T14:46:06.109553+0000","last_clean":"2024-05-03T14:46:06.109553+0000","last_became_active":"2024-05-03T14:45:05.421557+0000","last_became_peered":"2024-05-03T14:45:05.421557+0000","last_unstale":"2024-05-03T14:46:06.109553+0000","last_undegraded":"2024-05-03T14:46:06.109553+0000","last_fullsized":"2024-05-03T14:46:06.109553+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:44:03.074990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110911+0000","last_change":"2024-05-03T14:44:31.460150+0000","last_active":"2024-05-03T14:46:06.110911+0000","last_peered":"2024-05-03T14:46:06.110911+0000","last_clean":"2024-05-03T14:46:06.110911+0000","last_became_active":"2024-05-03T14:44:31.459550+0000","last_became_peered":"2024-05-03T14:44:31.459550+0000","last_unstale":"2024-05-03T14:46:06.110911+0000","last_undegraded":"2024-05-03T14:46:06.110911+0000","last_fullsized":"2024-05-03T14:46:06.110911+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:28:17.556068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.113036+0000","last_change":"2024-05-03T14:44:45.215035+0000","last_active":"2024-05-03T14:46:06.113036+0000","last_peered":"2024-05-03T14:46:06.113036+0000","last_clean":"2024-05-03T14:46:06.113036+0000","last_became_active":"2024-05-03T14:44:45.208667+0000","last_became_peered":"2024-05-03T14:44:45.208667+0000","last_unstale":"2024-05-03T14:46:06.113036+0000","last_undegraded":"2024-05-03T14:46:06.113036+0000","last_fullsized":"2024-05-03T14:46:06.113036+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:11:47.512884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'14","reported_seq":59,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:37.441771+0000","last_change":"2024-05-03T14:44:27.417377+0000","last_active":"2024-05-03T14:46:37.441771+0000","last_peered":"2024-05-03T14:46:37.441771+0000","last_clean":"2024-05-03T14:46:37.441771+0000","last_became_active":"2024-05-03T14:44:27.416983+0000","last_became_peered":"2024-05-03T14:44:27.416983+0000","last_unstale":"2024-05-03T14:46:37.441771+0000","last_undegraded":"2024-05-03T14:46:37.441771+0000","last_fullsized":"2024-05-03T14:46:37.441771+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T15:37:31.610517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.115275+0000","last_change":"2024-05-03T14:44:31.467316+0000","last_active":"2024-05-03T14:46:06.115275+0000","last_peered":"2024-05-03T14:46:06.115275+0000","last_clean":"2024-05-03T14:46:06.115275+0000","last_became_active":"2024-05-03T14:44:31.466025+0000","last_became_peered":"2024-05-03T14:44:31.466025+0000","last_unstale":"2024-05-03T14:46:06.115275+0000","last_undegraded":"2024-05-03T14:46:06.115275+0000","last_fullsized":"2024-05-03T14:46:06.115275+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:13:00.130689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.591023+0000","last_change":"2024-05-03T14:44:44.567031+0000","last_active":"2024-05-03T14:46:05.591023+0000","last_peered":"2024-05-03T14:46:05.591023+0000","last_clean":"2024-05-03T14:46:05.591023+0000","last_became_active":"2024-05-03T14:44:44.566562+0000","last_became_peered":"2024-05-03T14:44:44.566562+0000","last_unstale":"2024-05-03T14:46:05.591023+0000","last_undegraded":"2024-05-03T14:46:05.591023+0000","last_fullsized":"2024-05-03T14:46:05.591023+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T15:47:12.335283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110083+0000","last_change":"2024-05-03T14:44:28.279115+0000","last_active":"2024-05-03T14:46:06.110083+0000","last_peered":"2024-05-03T14:46:06.110083+0000","last_clean":"2024-05-03T14:46:06.110083+0000","last_became_active":"2024-05-03T14:44:28.278781+0000","last_became_peered":"2024-05-03T14:44:28.278781+0000","last_unstale":"2024-05-03T14:46:06.110083+0000","last_undegraded":"2024-05-03T14:46:06.110083+0000","last_fullsized":"2024-05-03T14:46:06.110083+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:40:41.273763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.590492+0000","last_change":"2024-05-03T14:44:31.475575+0000","last_active":"2024-05-03T14:46:05.590492+0000","last_peered":"2024-05-03T14:46:05.590492+0000","last_clean":"2024-05-03T14:46:05.590492+0000","last_became_active":"2024-05-03T14:44:31.474336+0000","last_became_peered":"2024-05-03T14:44:31.474336+0000","last_unstale":"2024-05-03T14:46:05.590492+0000","last_undegraded":"2024-05-03T14:46:05.590492+0000","last_fullsized":"2024-05-03T14:46:05.590492+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:22:48.742222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109679+0000","last_change":"2024-05-03T14:45:05.616983+0000","last_active":"2024-05-03T14:46:06.109679+0000","last_peered":"2024-05-03T14:46:06.109679+0000","last_clean":"2024-05-03T14:46:06.109679+0000","last_became_active":"2024-05-03T14:45:05.616475+0000","last_became_peered":"2024-05-03T14:45:05.616475+0000","last_unstale":"2024-05-03T14:46:06.109679+0000","last_undegraded":"2024-05-03T14:46:06.109679+0000","last_fullsized":"2024-05-03T14:46:06.109679+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:01:02.124133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.108773+0000","last_change":"2024-05-03T14:44:27.417092+0000","last_active":"2024-05-03T14:46:06.108773+0000","last_peered":"2024-05-03T14:46:06.108773+0000","last_clean":"2024-05-03T14:46:06.108773+0000","last_became_active":"2024-05-03T14:44:27.416765+0000","last_became_peered":"2024-05-03T14:44:27.416765+0000","last_unstale":"2024-05-03T14:46:06.108773+0000","last_undegraded":"2024-05-03T14:46:06.108773+0000","last_fullsized":"2024-05-03T14:46:06.108773+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T16:08:01.882063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.593648+0000","last_change":"2024-05-03T14:44:31.448394+0000","last_active":"2024-05-03T14:46:05.593648+0000","last_peered":"2024-05-03T14:46:05.593648+0000","last_clean":"2024-05-03T14:46:05.593648+0000","last_became_active":"2024-05-03T14:44:31.447752+0000","last_became_peered":"2024-05-03T14:44:31.447752+0000","last_unstale":"2024-05-03T14:46:05.593648+0000","last_undegraded":"2024-05-03T14:46:05.593648+0000","last_fullsized":"2024-05-03T14:46:05.593648+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:01:24.601259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.591583+0000","last_change":"2024-05-03T14:44:44.587688+0000","last_active":"2024-05-03T14:46:05.591583+0000","last_peered":"2024-05-03T14:46:05.591583+0000","last_clean":"2024-05-03T14:46:05.591583+0000","last_became_active":"2024-05-03T14:44:44.586982+0000","last_became_peered":"2024-05-03T14:44:44.586982+0000","last_unstale":"2024-05-03T14:46:05.591583+0000","last_undegraded":"2024-05-03T14:46:05.591583+0000","last_fullsized":"2024-05-03T14:46:05.591583+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:22:54.456635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.113246+0000","last_change":"2024-05-03T14:44:27.411828+0000","last_active":"2024-05-03T14:46:06.113246+0000","last_peered":"2024-05-03T14:46:06.113246+0000","last_clean":"2024-05-03T14:46:06.113246+0000","last_became_active":"2024-05-03T14:44:27.410490+0000","last_became_peered":"2024-05-03T14:44:27.410490+0000","last_unstale":"2024-05-03T14:46:06.113246+0000","last_undegraded":"2024-05-03T14:46:06.113246+0000","last_fullsized":"2024-05-03T14:46:06.113246+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:34:00.061773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109119+0000","last_change":"2024-05-03T14:44:31.448582+0000","last_active":"2024-05-03T14:46:06.109119+0000","last_peered":"2024-05-03T14:46:06.109119+0000","last_clean":"2024-05-03T14:46:06.109119+0000","last_became_active":"2024-05-03T14:44:31.448246+0000","last_became_peered":"2024-05-03T14:44:31.448246+0000","last_unstale":"2024-05-03T14:46:06.109119+0000","last_undegraded":"2024-05-03T14:46:06.109119+0000","last_fullsized":"2024-05-03T14:46:06.109119+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:12:19.856995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:17.125526+0000","last_change":"2024-05-03T14:45:05.616413+0000","last_active":"2024-05-03T14:46:17.125526+0000","last_peered":"2024-05-03T14:46:17.125526+0000","last_clean":"2024-05-03T14:46:17.125526+0000","last_became_active":"2024-05-03T14:45:05.615930+0000","last_became_peered":"2024-05-03T14:45:05.615930+0000","last_unstale":"2024-05-03T14:46:17.125526+0000","last_undegraded":"2024-05-03T14:46:17.125526+0000","last_fullsized":"2024-05-03T14:46:17.125526+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:32:43.344452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.591122+0000","last_change":"2024-05-03T14:44:27.417625+0000","last_active":"2024-05-03T14:46:05.591122+0000","last_peered":"2024-05-03T14:46:05.591122+0000","last_clean":"2024-05-03T14:46:05.591122+0000","last_became_active":"2024-05-03T14:44:27.417230+0000","last_became_peered":"2024-05-03T14:44:27.417230+0000","last_unstale":"2024-05-03T14:46:05.591122+0000","last_undegraded":"2024-05-03T14:46:05.591122+0000","last_fullsized":"2024-05-03T14:46:05.591122+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:56:13.661544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.114374+0000","last_change":"2024-05-03T14:44:31.475539+0000","last_active":"2024-05-03T14:46:06.114374+0000","last_peered":"2024-05-03T14:46:06.114374+0000","last_clean":"2024-05-03T14:46:06.114374+0000","last_became_active":"2024-05-03T14:44:31.474743+0000","last_became_peered":"2024-05-03T14:44:31.474743+0000","last_unstale":"2024-05-03T14:46:06.114374+0000","last_undegraded":"2024-05-03T14:46:06.114374+0000","last_fullsized":"2024-05-03T14:46:06.114374+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T15:52:19.705304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109488+0000","last_change":"2024-05-03T14:44:45.208377+0000","last_active":"2024-05-03T14:46:06.109488+0000","last_peered":"2024-05-03T14:46:06.109488+0000","last_clean":"2024-05-03T14:46:06.109488+0000","last_became_active":"2024-05-03T14:44:45.207778+0000","last_became_peered":"2024-05-03T14:44:45.207778+0000","last_unstale":"2024-05-03T14:46:06.109488+0000","last_undegraded":"2024-05-03T14:46:06.109488+0000","last_fullsized":"2024-05-03T14:46:06.109488+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T15:51:36.843263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.593889+0000","last_change":"2024-05-03T14:44:27.403571+0000","last_active":"2024-05-03T14:46:05.593889+0000","last_peered":"2024-05-03T14:46:05.593889+0000","last_clean":"2024-05-03T14:46:05.593889+0000","last_became_active":"2024-05-03T14:44:27.403173+0000","last_became_peered":"2024-05-03T14:44:27.403173+0000","last_unstale":"2024-05-03T14:46:05.593889+0000","last_undegraded":"2024-05-03T14:46:05.593889+0000","last_fullsized":"2024-05-03T14:46:05.593889+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T15:32:07.406763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.594841+0000","last_change":"2024-05-03T14:44:31.459573+0000","last_active":"2024-05-03T14:46:05.594841+0000","last_peered":"2024-05-03T14:46:05.594841+0000","last_clean":"2024-05-03T14:46:05.594841+0000","last_became_active":"2024-05-03T14:44:31.459200+0000","last_became_peered":"2024-05-03T14:44:31.459200+0000","last_unstale":"2024-05-03T14:46:05.594841+0000","last_undegraded":"2024-05-03T14:46:05.594841+0000","last_fullsized":"2024-05-03T14:46:05.594841+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:58:04.841787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.108797+0000","last_change":"2024-05-03T14:44:44.579130+0000","last_active":"2024-05-03T14:46:06.108797+0000","last_peered":"2024-05-03T14:46:06.108797+0000","last_clean":"2024-05-03T14:46:06.108797+0000","last_became_active":"2024-05-03T14:44:44.578696+0000","last_became_peered":"2024-05-03T14:44:44.578696+0000","last_unstale":"2024-05-03T14:46:06.108797+0000","last_undegraded":"2024-05-03T14:46:06.108797+0000","last_fullsized":"2024-05-03T14:46:06.108797+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:42:48.407127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'5","reported_seq":169,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:47:52.713498+0000","last_change":"2024-05-03T14:44:28.279181+0000","last_active":"2024-05-03T14:47:52.713498+0000","last_peered":"2024-05-03T14:47:52.713498+0000","last_clean":"2024-05-03T14:47:52.713498+0000","last_became_active":"2024-05-03T14:44:28.278790+0000","last_became_peered":"2024-05-03T14:44:28.278790+0000","last_unstale":"2024-05-03T14:47:52.713498+0000","last_undegraded":"2024-05-03T14:47:52.713498+0000","last_fullsized":"2024-05-03T14:47:52.713498+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:16:30.170750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":119,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.594905+0000","last_change":"2024-05-03T14:44:31.451646+0000","last_active":"2024-05-03T14:46:05.594905+0000","last_peered":"2024-05-03T14:46:05.594905+0000","last_clean":"2024-05-03T14:46:05.594905+0000","last_became_active":"2024-05-03T14:44:31.451319+0000","last_became_peered":"2024-05-03T14:44:31.451319+0000","last_unstale":"2024-05-03T14:46:05.594905+0000","last_undegraded":"2024-05-03T14:46:05.594905+0000","last_fullsized":"2024-05-03T14:46:05.594905+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:17:51.946028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.590751+0000","last_change":"2024-05-03T14:45:05.619282+0000","last_active":"2024-05-03T14:46:05.590751+0000","last_peered":"2024-05-03T14:46:05.590751+0000","last_clean":"2024-05-03T14:46:05.590751+0000","last_became_active":"2024-05-03T14:45:05.617776+0000","last_became_peered":"2024-05-03T14:45:05.617776+0000","last_unstale":"2024-05-03T14:46:05.590751+0000","last_undegraded":"2024-05-03T14:46:05.590751+0000","last_fullsized":"2024-05-03T14:46:05.590751+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:36:53.105226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110481+0000","last_change":"2024-05-03T14:44:27.415277+0000","last_active":"2024-05-03T14:46:06.110481+0000","last_peered":"2024-05-03T14:46:06.110481+0000","last_clean":"2024-05-03T14:46:06.110481+0000","last_became_active":"2024-05-03T14:44:27.414900+0000","last_became_peered":"2024-05-03T14:44:27.414900+0000","last_unstale":"2024-05-03T14:46:06.110481+0000","last_undegraded":"2024-05-03T14:46:06.110481+0000","last_fullsized":"2024-05-03T14:46:06.110481+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:41:00.065933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.594580+0000","last_change":"2024-05-03T14:44:31.467592+0000","last_active":"2024-05-03T14:46:05.594580+0000","last_peered":"2024-05-03T14:46:05.594580+0000","last_clean":"2024-05-03T14:46:05.594580+0000","last_became_active":"2024-05-03T14:44:31.467265+0000","last_became_peered":"2024-05-03T14:44:31.467265+0000","last_unstale":"2024-05-03T14:46:05.594580+0000","last_undegraded":"2024-05-03T14:46:05.594580+0000","last_fullsized":"2024-05-03T14:46:05.594580+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:28:26.672800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109129+0000","last_change":"2024-05-03T14:44:44.566154+0000","last_active":"2024-05-03T14:46:06.109129+0000","last_peered":"2024-05-03T14:46:06.109129+0000","last_clean":"2024-05-03T14:46:06.109129+0000","last_became_active":"2024-05-03T14:44:44.565714+0000","last_became_peered":"2024-05-03T14:44:44.565714+0000","last_unstale":"2024-05-03T14:46:06.109129+0000","last_undegraded":"2024-05-03T14:46:06.109129+0000","last_fullsized":"2024-05-03T14:46:06.109129+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:43.524153+0000","last_clean_scrub_stamp":"2024-05-03T14:44:43.524153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:54:48.448838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.113683+0000","last_change":"2024-05-03T14:44:27.419293+0000","last_active":"2024-05-03T14:46:06.113683+0000","last_peered":"2024-05-03T14:46:06.113683+0000","last_clean":"2024-05-03T14:46:06.113683+0000","last_became_active":"2024-05-03T14:44:27.418902+0000","last_became_peered":"2024-05-03T14:44:27.418902+0000","last_unstale":"2024-05-03T14:46:06.113683+0000","last_undegraded":"2024-05-03T14:46:06.113683+0000","last_fullsized":"2024-05-03T14:46:06.113683+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:26.360361+0000","last_clean_scrub_stamp":"2024-05-03T14:44:26.360361+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:17:54.890553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.595091+0000","last_change":"2024-05-03T14:44:31.449682+0000","last_active":"2024-05-03T14:46:05.595091+0000","last_peered":"2024-05-03T14:46:05.595091+0000","last_clean":"2024-05-03T14:46:05.595091+0000","last_became_active":"2024-05-03T14:44:31.449192+0000","last_became_peered":"2024-05-03T14:44:31.449192+0000","last_unstale":"2024-05-03T14:46:05.595091+0000","last_undegraded":"2024-05-03T14:46:05.595091+0000","last_fullsized":"2024-05-03T14:46:05.595091+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:48:46.249154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109183+0000","last_change":"2024-05-03T14:45:05.422527+0000","last_active":"2024-05-03T14:46:06.109183+0000","last_peered":"2024-05-03T14:46:06.109183+0000","last_clean":"2024-05-03T14:46:06.109183+0000","last_became_active":"2024-05-03T14:45:05.422014+0000","last_became_peered":"2024-05-03T14:45:05.422014+0000","last_unstale":"2024-05-03T14:46:06.109183+0000","last_undegraded":"2024-05-03T14:46:06.109183+0000","last_fullsized":"2024-05-03T14:46:06.109183+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T15:13:47.216960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.591688+0000","last_change":"2024-05-03T14:44:31.450447+0000","last_active":"2024-05-03T14:46:05.591688+0000","last_peered":"2024-05-03T14:46:05.591688+0000","last_clean":"2024-05-03T14:46:05.591688+0000","last_became_active":"2024-05-03T14:44:31.450103+0000","last_became_peered":"2024-05-03T14:44:31.450103+0000","last_unstale":"2024-05-03T14:46:05.591688+0000","last_undegraded":"2024-05-03T14:46:05.591688+0000","last_fullsized":"2024-05-03T14:46:05.591688+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:43:06.840375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.591291+0000","last_change":"2024-05-03T14:44:31.467676+0000","last_active":"2024-05-03T14:46:05.591291+0000","last_peered":"2024-05-03T14:46:05.591291+0000","last_clean":"2024-05-03T14:46:05.591291+0000","last_became_active":"2024-05-03T14:44:31.467285+0000","last_became_peered":"2024-05-03T14:44:31.467285+0000","last_unstale":"2024-05-03T14:46:05.591291+0000","last_undegraded":"2024-05-03T14:46:05.591291+0000","last_fullsized":"2024-05-03T14:46:05.591291+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:39:37.400178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.591341+0000","last_change":"2024-05-03T14:44:31.469379+0000","last_active":"2024-05-03T14:46:05.591341+0000","last_peered":"2024-05-03T14:46:05.591341+0000","last_clean":"2024-05-03T14:46:05.591341+0000","last_became_active":"2024-05-03T14:44:31.469022+0000","last_became_peered":"2024-05-03T14:44:31.469022+0000","last_unstale":"2024-05-03T14:46:05.591341+0000","last_undegraded":"2024-05-03T14:46:05.591341+0000","last_fullsized":"2024-05-03T14:46:05.591341+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:33:54.227652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.111107+0000","last_change":"2024-05-03T14:44:31.463709+0000","last_active":"2024-05-03T14:46:06.111107+0000","last_peered":"2024-05-03T14:46:06.111107+0000","last_clean":"2024-05-03T14:46:06.111107+0000","last_became_active":"2024-05-03T14:44:31.463302+0000","last_became_peered":"2024-05-03T14:44:31.463302+0000","last_unstale":"2024-05-03T14:46:06.111107+0000","last_undegraded":"2024-05-03T14:46:06.111107+0000","last_fullsized":"2024-05-03T14:46:06.111107+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:44:39.692253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109824+0000","last_change":"2024-05-03T14:44:31.443381+0000","last_active":"2024-05-03T14:46:06.109824+0000","last_peered":"2024-05-03T14:46:06.109824+0000","last_clean":"2024-05-03T14:46:06.109824+0000","last_became_active":"2024-05-03T14:44:31.442981+0000","last_became_peered":"2024-05-03T14:44:31.442981+0000","last_unstale":"2024-05-03T14:46:06.109824+0000","last_undegraded":"2024-05-03T14:46:06.109824+0000","last_fullsized":"2024-05-03T14:46:06.109824+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:58:56.653006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:05.591689+0000","last_change":"2024-05-03T14:44:31.449229+0000","last_active":"2024-05-03T14:46:05.591689+0000","last_peered":"2024-05-03T14:46:05.591689+0000","last_clean":"2024-05-03T14:46:05.591689+0000","last_became_active":"2024-05-03T14:44:31.448870+0000","last_became_peered":"2024-05-03T14:44:31.448870+0000","last_unstale":"2024-05-03T14:46:05.591689+0000","last_undegraded":"2024-05-03T14:46:05.591689+0000","last_fullsized":"2024-05-03T14:46:05.591689+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:48:29.213680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.115435+0000","last_change":"2024-05-03T14:44:31.468266+0000","last_active":"2024-05-03T14:46:06.115435+0000","last_peered":"2024-05-03T14:46:06.115435+0000","last_clean":"2024-05-03T14:46:06.115435+0000","last_became_active":"2024-05-03T14:44:31.467784+0000","last_became_peered":"2024-05-03T14:44:31.467784+0000","last_unstale":"2024-05-03T14:46:06.115435+0000","last_undegraded":"2024-05-03T14:46:06.115435+0000","last_fullsized":"2024-05-03T14:46:06.115435+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:46:18.979331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109250+0000","last_change":"2024-05-03T14:44:31.467526+0000","last_active":"2024-05-03T14:46:06.109250+0000","last_peered":"2024-05-03T14:46:06.109250+0000","last_clean":"2024-05-03T14:46:06.109250+0000","last_became_active":"2024-05-03T14:44:31.466849+0000","last_became_peered":"2024-05-03T14:44:31.466849+0000","last_unstale":"2024-05-03T14:46:06.109250+0000","last_undegraded":"2024-05-03T14:46:06.109250+0000","last_fullsized":"2024-05-03T14:46:06.109250+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:57:11.147130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.113267+0000","last_change":"2024-05-03T14:44:31.468251+0000","last_active":"2024-05-03T14:46:06.113267+0000","last_peered":"2024-05-03T14:46:06.113267+0000","last_clean":"2024-05-03T14:46:06.113267+0000","last_became_active":"2024-05-03T14:44:31.467812+0000","last_became_peered":"2024-05-03T14:44:31.467812+0000","last_unstale":"2024-05-03T14:46:06.113267+0000","last_undegraded":"2024-05-03T14:46:06.113267+0000","last_fullsized":"2024-05-03T14:46:06.113267+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:52:45.585331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.108911+0000","last_change":"2024-05-03T14:44:31.466378+0000","last_active":"2024-05-03T14:46:06.108911+0000","last_peered":"2024-05-03T14:46:06.108911+0000","last_clean":"2024-05-03T14:46:06.108911+0000","last_became_active":"2024-05-03T14:44:31.466052+0000","last_became_peered":"2024-05-03T14:44:31.466052+0000","last_unstale":"2024-05-03T14:46:06.108911+0000","last_undegraded":"2024-05-03T14:46:06.108911+0000","last_fullsized":"2024-05-03T14:46:06.108911+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:34:48.401934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.110503+0000","last_change":"2024-05-03T14:44:31.463771+0000","last_active":"2024-05-03T14:46:06.110503+0000","last_peered":"2024-05-03T14:46:06.110503+0000","last_clean":"2024-05-03T14:46:06.110503+0000","last_became_active":"2024-05-03T14:44:31.463204+0000","last_became_peered":"2024-05-03T14:44:31.463204+0000","last_unstale":"2024-05-03T14:46:06.110503+0000","last_undegraded":"2024-05-03T14:46:06.110503+0000","last_fullsized":"2024-05-03T14:46:06.110503+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T17:47:33.044835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T14:46:06.109581+0000","last_change":"2024-05-03T14:44:31.450950+0000","last_active":"2024-05-03T14:46:06.109581+0000","last_peered":"2024-05-03T14:46:06.109581+0000","last_clean":"2024-05-03T14:46:06.109581+0000","last_became_active":"2024-05-03T14:44:31.450595+0000","last_became_peered":"2024-05-03T14:44:31.450595+0000","last_unstale":"2024-05-03T14:46:06.109581+0000","last_undegraded":"2024-05-03T14:46:06.109581+0000","last_fullsized":"2024-05-03T14:46:06.109581+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T14:44:30.404038+0000","last_clean_scrub_stamp":"2024-05-03T14:44:30.404038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:42:41.240430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":250062,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66990,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1211,"num_read_kb":679,"num_write":249,"num_write_kb":140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":200970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":214,"ondisk_log_size":214,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710779,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":516321,"data_compressed":32309,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907008,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28128,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717024,"statfs":{"total":95995035648,"available":95966232576,"internally_reserved":0,"allocated":1138688,"data_stored":1167017,"data_compressed":68497,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070534,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":515467,"data_compressed":32309,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266765,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":999424,"data_stored":515733,"data_compressed":32309,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462993,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":1111279,"data_compressed":68497,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626519,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":987136,"data_stored":510971,"data_compressed":32309,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574963,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":573191,"data_compressed":32309,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738490,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":512427,"data_compressed":32309,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411440,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1064960,"data_stored":572167,"data_compressed":32309,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182504,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":510679,"data_compressed":32309,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986275,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":995328,"data_stored":515045,"data_compressed":32309,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822750,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1069056,"data_stored":1107533,"data_compressed":68497,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34561,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34561,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36188,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36188,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36188,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36188,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36188,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":62942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":61918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":62304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-03T14:47:59.064 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-03T14:47:59.065 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-03T14:47:59.065 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-03T14:47:59.065 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph health --format=json 2024-05-03T14:47:59.579 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:47:59.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:47:59 smithi090 ceph-mon[33568]: from='client.24919 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T14:47:59.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:47:59 smithi090 ceph-mon[33568]: pgmap v436: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:47:59.896 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:47:59 smithi060 ceph-mon[28873]: from='client.24919 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T14:47:59.896 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:47:59 smithi060 ceph-mon[28873]: pgmap v436: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:47:59.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:47:59 smithi149 ceph-mon[33332]: from='client.24919 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T14:47:59.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:47:59 smithi149 ceph-mon[33332]: pgmap v436: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:48:01.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:01.719+0000 7f3a3e200700 1 -- 172.21.15.60:0/2123487651 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a381086f0 msgr2=0x7f3a381102c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:01.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:01.719+0000 7f3a3e200700 1 --2- 172.21.15.60:0/2123487651 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a381086f0 0x7f3a381102c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3a30009fc0 tx=0x7f3a3000b330 comp rx=0 tx=0).stop 2024-05-03T14:48:01.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:01.720+0000 7f3a3e200700 1 -- 172.21.15.60:0/2123487651 shutdown_connections 2024-05-03T14:48:01.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:01.720+0000 7f3a3e200700 1 --2- 172.21.15.60:0/2123487651 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a381086f0 0x7f3a381102c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:01.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:01.720+0000 7f3a3e200700 1 --2- 172.21.15.60:0/2123487651 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a38103610 0x7f3a381039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:01.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:01.720+0000 7f3a3e200700 1 --2- 172.21.15.60:0/2123487651 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a38103fc0 0x7f3a38108010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:01.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:01.720+0000 7f3a3e200700 1 -- 172.21.15.60:0/2123487651 >> 172.21.15.60:0/2123487651 conn(0x7f3a380feec0 msgr2=0x7f3a381012e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:48:01.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:01.720+0000 7f3a3e200700 1 -- 172.21.15.60:0/2123487651 shutdown_connections 2024-05-03T14:48:01.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:01.720+0000 7f3a3e200700 1 -- 172.21.15.60:0/2123487651 wait complete. 2024-05-03T14:48:01.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:01.721+0000 7f3a3e200700 1 Processor -- start 2024-05-03T14:48:01.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:01.721+0000 7f3a3e200700 1 -- start start 2024-05-03T14:48:01.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:01.721+0000 7f3a3e200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a38103610 0x7f3a3819bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:01.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:01.721+0000 7f3a3e200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a38103fc0 0x7f3a3819c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:01.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:01.722+0000 7f3a3e200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a381086f0 0x7f3a381a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:01.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:01.722+0000 7f3a3e200700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f3a38113870 con 0x7f3a381086f0 2024-05-03T14:48:01.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:01.722+0000 7f3a3e200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3a381136f0 con 0x7f3a38103610 2024-05-03T14:48:01.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:01.722+0000 7f3a3e200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3a381139f0 con 0x7f3a38103fc0 2024-05-03T14:48:01.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:01.722+0000 7f3a37e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a38103fc0 0x7f3a3819c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:01.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:01.722+0000 7f3a37e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a38103fc0 0x7f3a3819c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.60:57234/0 (socket says 172.21.15.60:57234) 2024-05-03T14:48:01.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:01.722+0000 7f3a3ce00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a38103610 0x7f3a3819bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:01.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:01.722+0000 7f3a37e00700 1 -- 172.21.15.60:0/3005125756 learned_addr learned my addr 172.21.15.60:0/3005125756 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:48:01.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:01.722+0000 7f3a3d800700 1 --2- 172.21.15.60:0/3005125756 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a381086f0 0x7f3a381a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:01.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:01.722+0000 7f3a3ce00700 1 -- 172.21.15.60:0/3005125756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a38103fc0 msgr2=0x7f3a3819c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:01.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:01.722+0000 7f3a3ce00700 1 --2- 172.21.15.60:0/3005125756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a38103fc0 0x7f3a3819c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:01.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:01.722+0000 7f3a3ce00700 1 -- 172.21.15.60:0/3005125756 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a381086f0 msgr2=0x7f3a381a0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:01.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:01.722+0000 7f3a3ce00700 1 --2- 172.21.15.60:0/3005125756 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a381086f0 0x7f3a381a0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:01.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:01.722+0000 7f3a3ce00700 1 -- 172.21.15.60:0/3005125756 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a381a0eb0 con 0x7f3a38103610 2024-05-03T14:48:01.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:01.722+0000 7f3a37e00700 1 --2- 172.21.15.60:0/3005125756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a38103fc0 0x7f3a3819c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:48:01.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:01.723+0000 7f3a3ce00700 1 --2- 172.21.15.60:0/3005125756 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a38103610 0x7f3a3819bea0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c00e9f0 tx=0x7f3a2c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:48:01.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:01.723+0000 7f3a35600700 1 -- 172.21.15.60:0/3005125756 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a2c00c9c0 con 0x7f3a38103610 2024-05-03T14:48:01.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:01.723+0000 7f3a3e200700 1 -- 172.21.15.60:0/3005125756 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a381a11a0 con 0x7f3a38103610 2024-05-03T14:48:01.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:01.723+0000 7f3a35600700 1 -- 172.21.15.60:0/3005125756 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f3a2c00cb60 con 0x7f3a38103610 2024-05-03T14:48:01.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:01.723+0000 7f3a35600700 1 -- 172.21.15.60:0/3005125756 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a2c018660 con 0x7f3a38103610 2024-05-03T14:48:01.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:01.723+0000 7f3a3e200700 1 -- 172.21.15.60:0/3005125756 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a381a8a20 con 0x7f3a38103610 2024-05-03T14:48:01.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:01.724+0000 7f3a3e200700 1 -- 172.21.15.60:0/3005125756 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a3806a190 con 0x7f3a38103610 2024-05-03T14:48:01.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:01.727+0000 7f3a35600700 1 -- 172.21.15.60:0/3005125756 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f3a2c00cd10 con 0x7f3a38103610 2024-05-03T14:48:01.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:01.728+0000 7f3a35600700 1 --2- 172.21.15.60:0/3005125756 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f3a200b2990 0x7f3a200b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:01.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:01.728+0000 7f3a35600700 1 -- 172.21.15.60:0/3005125756 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f3a2c0e2f20 con 0x7f3a38103610 2024-05-03T14:48:01.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:01.729+0000 7f3a37e00700 1 --2- 172.21.15.60:0/3005125756 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f3a200b2990 0x7f3a200b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:01.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:01.729+0000 7f3a37e00700 1 --2- 172.21.15.60:0/3005125756 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f3a200b2990 0x7f3a200b4e50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3a38110650 tx=0x7f3a28009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:48:01.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:01.730+0000 7f3a35600700 1 -- 172.21.15.60:0/3005125756 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f3a2c0aa700 con 0x7f3a38103610 2024-05-03T14:48:01.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:48:01 smithi060 ceph-mon[28873]: pgmap v437: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:48:01.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:48:01 smithi090 ceph-mon[33568]: pgmap v437: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:48:01.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:48:01 smithi149 ceph-mon[33332]: pgmap v437: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:48:02.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:02.025+0000 7f3a3e200700 1 -- 172.21.15.60:0/3005125756 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f3a3819d200 con 0x7f3a38103610 2024-05-03T14:48:02.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:02.026+0000 7f3a35600700 1 -- 172.21.15.60:0/3005125756 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f3a2c0aa000 con 0x7f3a38103610 2024-05-03T14:48:02.029 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:48:02.029 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-03T14:48:02.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:02.029+0000 7f3a3e200700 1 -- 172.21.15.60:0/3005125756 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f3a200b2990 msgr2=0x7f3a200b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:02.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:02.029+0000 7f3a3e200700 1 --2- 172.21.15.60:0/3005125756 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f3a200b2990 0x7f3a200b4e50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3a38110650 tx=0x7f3a28009310 comp rx=0 tx=0).stop 2024-05-03T14:48:02.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:02.029+0000 7f3a3e200700 1 -- 172.21.15.60:0/3005125756 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a38103610 msgr2=0x7f3a3819bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:02.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:02.029+0000 7f3a3e200700 1 --2- 172.21.15.60:0/3005125756 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a38103610 0x7f3a3819bea0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c00e9f0 tx=0x7f3a2c00ed00 comp rx=0 tx=0).stop 2024-05-03T14:48:02.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:02.029+0000 7f3a3e200700 1 -- 172.21.15.60:0/3005125756 shutdown_connections 2024-05-03T14:48:02.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:02.029+0000 7f3a3e200700 1 --2- 172.21.15.60:0/3005125756 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f3a200b2990 0x7f3a200b4e50 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:02.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:02.029+0000 7f3a3e200700 1 --2- 172.21.15.60:0/3005125756 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a381086f0 0x7f3a381a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:02.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:02.029+0000 7f3a3e200700 1 --2- 172.21.15.60:0/3005125756 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a38103610 0x7f3a3819bea0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:02.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:02.029+0000 7f3a3e200700 1 --2- 172.21.15.60:0/3005125756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a38103fc0 0x7f3a3819c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:02.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:02.029+0000 7f3a3e200700 1 -- 172.21.15.60:0/3005125756 >> 172.21.15.60:0/3005125756 conn(0x7f3a380feec0 msgr2=0x7f3a38075b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:48:02.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:02.030+0000 7f3a3e200700 1 -- 172.21.15.60:0/3005125756 shutdown_connections 2024-05-03T14:48:02.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:02.030+0000 7f3a3e200700 1 -- 172.21.15.60:0/3005125756 wait complete. 2024-05-03T14:48:02.779 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:48:02 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/3005125756' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-03T14:48:02.841 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-03T14:48:02.842 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T14:48:02.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:48:02 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/3005125756' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-03T14:48:02.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:48:02 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/3005125756' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-03T14:48:03.327 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:48:03.797 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:48:03 smithi060 ceph-mon[28873]: pgmap v438: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T14:48:03.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:48:03 smithi149 ceph-mon[33332]: pgmap v438: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T14:48:04.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:48:03 smithi090 ceph-mon[33568]: pgmap v438: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T14:48:05.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:05.460+0000 7f66b4800700 1 -- 172.21.15.60:0/2480024837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66ac10a490 msgr2=0x7f66ac1169a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:05.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:05.460+0000 7f66b4800700 1 --2- 172.21.15.60:0/2480024837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66ac10a490 0x7f66ac1169a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f66a800b0a0 tx=0x7f66a800b3b0 comp rx=0 tx=0).stop 2024-05-03T14:48:05.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:05.461+0000 7f66b4800700 1 -- 172.21.15.60:0/2480024837 shutdown_connections 2024-05-03T14:48:05.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:05.461+0000 7f66b4800700 1 --2- 172.21.15.60:0/2480024837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66ac10a490 0x7f66ac1169a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:05.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:05.461+0000 7f66b4800700 1 --2- 172.21.15.60:0/2480024837 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66ac10fb60 0x7f66ac10ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:05.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:05.461+0000 7f66b4800700 1 --2- 172.21.15.60:0/2480024837 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66ac109ad0 0x7f66ac109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:05.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:05.461+0000 7f66b4800700 1 -- 172.21.15.60:0/2480024837 >> 172.21.15.60:0/2480024837 conn(0x7f66ac076c70 msgr2=0x7f66ac1082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:48:05.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:05.461+0000 7f66b4800700 1 -- 172.21.15.60:0/2480024837 shutdown_connections 2024-05-03T14:48:05.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:05.461+0000 7f66b4800700 1 -- 172.21.15.60:0/2480024837 wait complete. 2024-05-03T14:48:05.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:05.462+0000 7f66b4800700 1 Processor -- start 2024-05-03T14:48:05.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:05.462+0000 7f66b4800700 1 -- start start 2024-05-03T14:48:05.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:05.463+0000 7f66b4800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66ac109ad0 0x7f66ac1a02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:05.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:05.463+0000 7f66b4800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66ac10a490 0x7f66ac1a0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:05.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:05.463+0000 7f66b4800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66ac10fb60 0x7f66ac1a4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:05.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:05.463+0000 7f66b4800700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f66ac119f40 con 0x7f66ac109ad0 2024-05-03T14:48:05.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:05.463+0000 7f66b4800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f66ac119dc0 con 0x7f66ac10a490 2024-05-03T14:48:05.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:05.463+0000 7f66b4800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f66ac11a0c0 con 0x7f66ac10fb60 2024-05-03T14:48:05.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:05.463+0000 7f66b1e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66ac109ad0 0x7f66ac1a02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:05.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:05.463+0000 7f66b1e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66ac109ad0 0x7f66ac1a02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37804/0 (socket says 172.21.15.60:37804) 2024-05-03T14:48:05.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:05.463+0000 7f66b1e00700 1 -- 172.21.15.60:0/1029584902 learned_addr learned my addr 172.21.15.60:0/1029584902 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:48:05.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:05.463+0000 7f66b1400700 1 --2- 172.21.15.60:0/1029584902 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66ac10a490 0x7f66ac1a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:05.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:05.464+0000 7f66b2800700 1 --2- 172.21.15.60:0/1029584902 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66ac10fb60 0x7f66ac1a4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:05.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:05.464+0000 7f66b1e00700 1 -- 172.21.15.60:0/1029584902 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66ac10fb60 msgr2=0x7f66ac1a4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:05.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:05.464+0000 7f66b1e00700 1 --2- 172.21.15.60:0/1029584902 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66ac10fb60 0x7f66ac1a4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:05.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:05.464+0000 7f66b1e00700 1 -- 172.21.15.60:0/1029584902 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66ac10a490 msgr2=0x7f66ac1a0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:05.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:05.464+0000 7f66b1e00700 1 --2- 172.21.15.60:0/1029584902 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66ac10a490 0x7f66ac1a0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:05.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:05.464+0000 7f66b1e00700 1 -- 172.21.15.60:0/1029584902 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66ac1a52b0 con 0x7f66ac109ad0 2024-05-03T14:48:05.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:05.464+0000 7f66b1e00700 1 --2- 172.21.15.60:0/1029584902 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66ac109ad0 0x7f66ac1a02f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f66a000eb30 tx=0x7f66a000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:48:05.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:05.464+0000 7f66a6a00700 1 -- 172.21.15.60:0/1029584902 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a000cc90 con 0x7f66ac109ad0 2024-05-03T14:48:05.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:05.464+0000 7f66a6a00700 1 -- 172.21.15.60:0/1029584902 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f66a0004520 con 0x7f66ac109ad0 2024-05-03T14:48:05.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:05.464+0000 7f66b4800700 1 -- 172.21.15.60:0/1029584902 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66ac1a55a0 con 0x7f66ac109ad0 2024-05-03T14:48:05.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:05.464+0000 7f66a6a00700 1 -- 172.21.15.60:0/1029584902 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a0010690 con 0x7f66ac109ad0 2024-05-03T14:48:05.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:05.464+0000 7f66b4800700 1 -- 172.21.15.60:0/1029584902 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66ac1ace20 con 0x7f66ac109ad0 2024-05-03T14:48:05.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:05.466+0000 7f66b4800700 1 -- 172.21.15.60:0/1029584902 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66ac06a190 con 0x7f66ac109ad0 2024-05-03T14:48:05.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:05.468+0000 7f66a6a00700 1 -- 172.21.15.60:0/1029584902 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f66a0003e10 con 0x7f66ac109ad0 2024-05-03T14:48:05.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:05.471+0000 7f66a6a00700 1 --2- 172.21.15.60:0/1029584902 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f66940b2990 0x7f66940b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:05.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:05.471+0000 7f66a6a00700 1 -- 172.21.15.60:0/1029584902 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f66a00e2a20 con 0x7f66ac109ad0 2024-05-03T14:48:05.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:05.471+0000 7f66b1400700 1 --2- 172.21.15.60:0/1029584902 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f66940b2990 0x7f66940b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:05.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:05.472+0000 7f66b1400700 1 --2- 172.21.15.60:0/1029584902 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f66940b2990 0x7f66940b4e50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f66ac078a30 tx=0x7f669c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:48:05.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:05.474+0000 7f66a6a00700 1 -- 172.21.15.60:0/1029584902 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f66a00aa290 con 0x7f66ac109ad0 2024-05-03T14:48:05.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:05.772+0000 7f66b4800700 1 -- 172.21.15.60:0/1029584902 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f66ac02e510 con 0x7f66ac109ad0 2024-05-03T14:48:05.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:05.774+0000 7f66a6a00700 1 -- 172.21.15.60:0/1029584902 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11929 (secure 0 0 0) 0x7f66a00a9b90 con 0x7f66ac109ad0 2024-05-03T14:48:05.777 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:48:05.777 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1849151522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1849151522},{"type":"v1","addr":"172.21.15.60:6835","nonce":1849151522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/523710346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":523710346},{"type":"v1","addr":"172.21.15.60:6837","nonce":523710346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T14:44:34.226052+0000","modified":"2024-05-03T14:46:01.367913+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24511,"mds_1":24509,"mds_2":24487,"mds_3":24475,"mds_4":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14397":{"gid":14397,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.60:6839/3846205424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3846205424},{"type":"v1","addr":"172.21.15.60:6839","nonce":3846205424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6833/3033424631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":3033424631},{"type":"v1","addr":"172.21.15.90:6833","nonce":3033424631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6833/3454021291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3454021291},{"type":"v1","addr":"172.21.15.149:6833","nonce":3454021291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6835/1057049585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1057049585},{"type":"v1","addr":"172.21.15.90:6835","nonce":1057049585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/882239924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":882239924},{"type":"v1","addr":"172.21.15.149:6835","nonce":882239924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":31,"addr":"172.21.15.90:6837/2243702844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":2243702844},{"type":"v1","addr":"172.21.15.90:6837","nonce":2243702844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":32,"addr":"172.21.15.149:6837/2611278690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2611278690},{"type":"v1","addr":"172.21.15.149:6837","nonce":2611278690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.149:6839/2527622212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2527622212},{"type":"v1","addr":"172.21.15.149:6839","nonce":2527622212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.90:6839/3512381172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3512381172},{"type":"v1","addr":"172.21.15.90:6839","nonce":3512381172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.60:6841/924163182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":924163182},{"type":"v1","addr":"172.21.15.60:6841","nonce":924163182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24511,"qdb_cluster":[24470,24475,24487,24509,24511]},"id":1}]} 2024-05-03T14:48:05.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:05.777+0000 7f66b4800700 1 -- 172.21.15.60:0/1029584902 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f66940b2990 msgr2=0x7f66940b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:05.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:05.777+0000 7f66b4800700 1 --2- 172.21.15.60:0/1029584902 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f66940b2990 0x7f66940b4e50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f66ac078a30 tx=0x7f669c009260 comp rx=0 tx=0).stop 2024-05-03T14:48:05.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:05.777+0000 7f66b4800700 1 -- 172.21.15.60:0/1029584902 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66ac109ad0 msgr2=0x7f66ac1a02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:05.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:05.777+0000 7f66b4800700 1 --2- 172.21.15.60:0/1029584902 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66ac109ad0 0x7f66ac1a02f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f66a000eb30 tx=0x7f66a000ee40 comp rx=0 tx=0).stop 2024-05-03T14:48:05.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:05.777+0000 7f66b4800700 1 -- 172.21.15.60:0/1029584902 shutdown_connections 2024-05-03T14:48:05.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:05.777+0000 7f66b4800700 1 --2- 172.21.15.60:0/1029584902 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f66940b2990 0x7f66940b4e50 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:05.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:05.777+0000 7f66b4800700 1 --2- 172.21.15.60:0/1029584902 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66ac10fb60 0x7f66ac1a4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:05.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:05.777+0000 7f66b4800700 1 --2- 172.21.15.60:0/1029584902 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66ac109ad0 0x7f66ac1a02f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:05.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:05.777+0000 7f66b4800700 1 --2- 172.21.15.60:0/1029584902 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66ac10a490 0x7f66ac1a0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:05.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:05.777+0000 7f66b4800700 1 -- 172.21.15.60:0/1029584902 >> 172.21.15.60:0/1029584902 conn(0x7f66ac076c70 msgr2=0x7f66ac077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:48:05.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:05.777+0000 7f66b4800700 1 -- 172.21.15.60:0/1029584902 shutdown_connections 2024-05-03T14:48:05.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:05.777+0000 7f66b4800700 1 -- 172.21.15.60:0/1029584902 wait complete. 2024-05-03T14:48:05.782 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 29 2024-05-03T14:48:05.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:48:05 smithi149 ceph-mon[33332]: pgmap v439: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:48:06.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:48:05 smithi060 ceph-mon[28873]: pgmap v439: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:48:06.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:48:05 smithi090 ceph-mon[33568]: pgmap v439: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:48:06.640 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T14:48:06.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:48:06 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/1029584902' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T14:48:06.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:48:06 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/1029584902' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T14:48:07.098 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:48:07.107 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:48:06 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/1029584902' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T14:48:07.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:48:07 smithi060 ceph-mon[28873]: pgmap v440: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:48:07.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:48:07 smithi149 ceph-mon[33332]: pgmap v440: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:48:08.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:48:07 smithi090 ceph-mon[33568]: pgmap v440: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:48:09.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:09.200+0000 7f0ca4800700 1 -- 172.21.15.60:0/1649246993 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c9c104100 msgr2=0x7f0c9c108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:09.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:09.200+0000 7f0ca4800700 1 --2- 172.21.15.60:0/1649246993 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c9c104100 0x7f0c9c108120 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c0099c0 tx=0x7f0c8c009cd0 comp rx=0 tx=0).stop 2024-05-03T14:48:09.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:09.200+0000 7f0ca4800700 1 -- 172.21.15.60:0/1649246993 shutdown_connections 2024-05-03T14:48:09.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:09.200+0000 7f0ca4800700 1 --2- 172.21.15.60:0/1649246993 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c9c108850 0x7f0c9c1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:09.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:09.200+0000 7f0ca4800700 1 --2- 172.21.15.60:0/1649246993 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0c9c103750 0x7f0c9c103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:09.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:09.200+0000 7f0ca4800700 1 --2- 172.21.15.60:0/1649246993 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c9c104100 0x7f0c9c108120 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:09.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:09.200+0000 7f0ca4800700 1 -- 172.21.15.60:0/1649246993 >> 172.21.15.60:0/1649246993 conn(0x7f0c9c07a600 msgr2=0x7f0c9c07aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:48:09.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:09.200+0000 7f0ca4800700 1 -- 172.21.15.60:0/1649246993 shutdown_connections 2024-05-03T14:48:09.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:09.200+0000 7f0ca4800700 1 -- 172.21.15.60:0/1649246993 wait complete. 2024-05-03T14:48:09.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:09.201+0000 7f0ca4800700 1 Processor -- start 2024-05-03T14:48:09.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:09.201+0000 7f0ca4800700 1 -- start start 2024-05-03T14:48:09.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:09.202+0000 7f0ca4800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c9c103750 0x7f0c9c19bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:09.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:09.202+0000 7f0ca4800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c9c104100 0x7f0c9c19c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:09.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:09.202+0000 7f0ca4800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0c9c108850 0x7f0c9c1a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:09.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:09.202+0000 7f0ca4800700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f0c9c1138a0 con 0x7f0c9c103750 2024-05-03T14:48:09.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:09.202+0000 7f0ca4800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0c9c113720 con 0x7f0c9c108850 2024-05-03T14:48:09.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:09.202+0000 7f0ca4800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0c9c113a20 con 0x7f0c9c104100 2024-05-03T14:48:09.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:09.202+0000 7f0ca1e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c9c103750 0x7f0c9c19bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:09.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:09.202+0000 7f0ca2800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0c9c108850 0x7f0c9c1a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:09.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:09.202+0000 7f0ca1e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c9c103750 0x7f0c9c19bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37818/0 (socket says 172.21.15.60:37818) 2024-05-03T14:48:09.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:09.202+0000 7f0ca1e00700 1 -- 172.21.15.60:0/2557384329 learned_addr learned my addr 172.21.15.60:0/2557384329 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:48:09.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:09.202+0000 7f0ca2800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0c9c108850 0x7f0c9c1a0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.60:36466/0 (socket says 172.21.15.60:36466) 2024-05-03T14:48:09.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:09.202+0000 7f0ca1400700 1 --2- 172.21.15.60:0/2557384329 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c9c104100 0x7f0c9c19c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:09.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:09.203+0000 7f0ca2800700 1 -- 172.21.15.60:0/2557384329 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c9c104100 msgr2=0x7f0c9c19c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:09.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:09.203+0000 7f0ca2800700 1 --2- 172.21.15.60:0/2557384329 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c9c104100 0x7f0c9c19c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:09.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:09.203+0000 7f0ca2800700 1 -- 172.21.15.60:0/2557384329 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c9c103750 msgr2=0x7f0c9c19bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:09.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:09.203+0000 7f0ca2800700 1 --2- 172.21.15.60:0/2557384329 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c9c103750 0x7f0c9c19bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:09.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:09.203+0000 7f0ca2800700 1 -- 172.21.15.60:0/2557384329 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c9c1a0eb0 con 0x7f0c9c108850 2024-05-03T14:48:09.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:09.203+0000 7f0ca2800700 1 --2- 172.21.15.60:0/2557384329 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0c9c108850 0x7f0c9c1a0770 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0c9800efc0 tx=0x7f0c9800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:48:09.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:09.203+0000 7f0c96a00700 1 -- 172.21.15.60:0/2557384329 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c980092e0 con 0x7f0c9c108850 2024-05-03T14:48:09.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:09.203+0000 7f0c96a00700 1 -- 172.21.15.60:0/2557384329 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f0c98004bb0 con 0x7f0c9c108850 2024-05-03T14:48:09.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:09.203+0000 7f0c96a00700 1 -- 172.21.15.60:0/2557384329 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c98004280 con 0x7f0c9c108850 2024-05-03T14:48:09.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:09.204+0000 7f0ca4800700 1 -- 172.21.15.60:0/2557384329 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c9c1a1140 con 0x7f0c9c108850 2024-05-03T14:48:09.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:09.204+0000 7f0ca4800700 1 -- 172.21.15.60:0/2557384329 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c9c1a8a20 con 0x7f0c9c108850 2024-05-03T14:48:09.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:09.205+0000 7f0ca4800700 1 -- 172.21.15.60:0/2557384329 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c9c06a190 con 0x7f0c9c108850 2024-05-03T14:48:09.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:09.207+0000 7f0c96a00700 1 -- 172.21.15.60:0/2557384329 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f0c9801b030 con 0x7f0c9c108850 2024-05-03T14:48:09.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:09.209+0000 7f0c96a00700 1 --2- 172.21.15.60:0/2557384329 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f0c840b2990 0x7f0c840b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:09.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:09.209+0000 7f0c96a00700 1 -- 172.21.15.60:0/2557384329 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f0c980e2310 con 0x7f0c9c108850 2024-05-03T14:48:09.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:09.209+0000 7f0ca1e00700 1 --2- 172.21.15.60:0/2557384329 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f0c840b2990 0x7f0c840b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:09.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:09.210+0000 7f0ca1e00700 1 --2- 172.21.15.60:0/2557384329 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f0c840b2990 0x7f0c840b4e50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0c90009d20 tx=0x7f0c90009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:48:09.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:09.211+0000 7f0c96a00700 1 -- 172.21.15.60:0/2557384329 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f0c98022020 con 0x7f0c9c108850 2024-05-03T14:48:09.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:09.503+0000 7f0ca4800700 1 -- 172.21.15.60:0/2557384329 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0c9c1a8bf0 con 0x7f0c9c108850 2024-05-03T14:48:09.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:09.504+0000 7f0c96a00700 1 -- 172.21.15.60:0/2557384329 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11929 (secure 0 0 0) 0x7f0c98010040 con 0x7f0c9c108850 2024-05-03T14:48:09.507 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:48:09.507 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1849151522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1849151522},{"type":"v1","addr":"172.21.15.60:6835","nonce":1849151522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/523710346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":523710346},{"type":"v1","addr":"172.21.15.60:6837","nonce":523710346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T14:44:34.226052+0000","modified":"2024-05-03T14:46:01.367913+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24511,"mds_1":24509,"mds_2":24487,"mds_3":24475,"mds_4":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14397":{"gid":14397,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.60:6839/3846205424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3846205424},{"type":"v1","addr":"172.21.15.60:6839","nonce":3846205424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6833/3033424631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":3033424631},{"type":"v1","addr":"172.21.15.90:6833","nonce":3033424631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6833/3454021291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3454021291},{"type":"v1","addr":"172.21.15.149:6833","nonce":3454021291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6835/1057049585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1057049585},{"type":"v1","addr":"172.21.15.90:6835","nonce":1057049585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/882239924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":882239924},{"type":"v1","addr":"172.21.15.149:6835","nonce":882239924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":31,"addr":"172.21.15.90:6837/2243702844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":2243702844},{"type":"v1","addr":"172.21.15.90:6837","nonce":2243702844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":32,"addr":"172.21.15.149:6837/2611278690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2611278690},{"type":"v1","addr":"172.21.15.149:6837","nonce":2611278690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.149:6839/2527622212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2527622212},{"type":"v1","addr":"172.21.15.149:6839","nonce":2527622212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.90:6839/3512381172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3512381172},{"type":"v1","addr":"172.21.15.90:6839","nonce":3512381172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.60:6841/924163182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":924163182},{"type":"v1","addr":"172.21.15.60:6841","nonce":924163182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24511,"qdb_cluster":[24470,24475,24487,24509,24511]},"id":1}]} 2024-05-03T14:48:09.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:09.508+0000 7f0ca4800700 1 -- 172.21.15.60:0/2557384329 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f0c840b2990 msgr2=0x7f0c840b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:09.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:09.508+0000 7f0ca4800700 1 --2- 172.21.15.60:0/2557384329 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f0c840b2990 0x7f0c840b4e50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0c90009d20 tx=0x7f0c90009260 comp rx=0 tx=0).stop 2024-05-03T14:48:09.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:09.508+0000 7f0ca4800700 1 -- 172.21.15.60:0/2557384329 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0c9c108850 msgr2=0x7f0c9c1a0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:09.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:09.508+0000 7f0ca4800700 1 --2- 172.21.15.60:0/2557384329 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0c9c108850 0x7f0c9c1a0770 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0c9800efc0 tx=0x7f0c9800c3a0 comp rx=0 tx=0).stop 2024-05-03T14:48:09.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:09.508+0000 7f0ca4800700 1 -- 172.21.15.60:0/2557384329 shutdown_connections 2024-05-03T14:48:09.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:09.508+0000 7f0ca4800700 1 --2- 172.21.15.60:0/2557384329 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f0c840b2990 0x7f0c840b4e50 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:09.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:09.508+0000 7f0ca4800700 1 --2- 172.21.15.60:0/2557384329 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0c9c108850 0x7f0c9c1a0770 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:09.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:09.508+0000 7f0ca4800700 1 --2- 172.21.15.60:0/2557384329 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c9c103750 0x7f0c9c19bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:09.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:09.508+0000 7f0ca4800700 1 --2- 172.21.15.60:0/2557384329 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c9c104100 0x7f0c9c19c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:09.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:09.508+0000 7f0ca4800700 1 -- 172.21.15.60:0/2557384329 >> 172.21.15.60:0/2557384329 conn(0x7f0c9c07a600 msgr2=0x7f0c9c076870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:48:09.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:09.508+0000 7f0ca4800700 1 -- 172.21.15.60:0/2557384329 shutdown_connections 2024-05-03T14:48:09.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:09.508+0000 7f0ca4800700 1 -- 172.21.15.60:0/2557384329 wait complete. 2024-05-03T14:48:09.512 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 29 2024-05-03T14:48:09.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:48:09 smithi060 ceph-mon[28873]: pgmap v441: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:48:09.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:48:09 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/2557384329' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T14:48:09.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:48:09 smithi149 ceph-mon[33332]: pgmap v441: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:48:09.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:48:09 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/2557384329' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T14:48:10.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:48:09 smithi090 ceph-mon[33568]: pgmap v441: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:48:10.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:48:09 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/2557384329' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T14:48:10.323 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-03T14:48:10.787 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:48:11.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:48:11 smithi149 ceph-mon[33332]: pgmap v442: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:48:12.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:48:11 smithi060 ceph-mon[28873]: pgmap v442: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:48:12.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:48:11 smithi090 ceph-mon[33568]: pgmap v442: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:48:13.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:13.036+0000 7fe972a00700 1 -- 172.21.15.60:0/3075755456 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe96c06b180 msgr2=0x7fe96c06b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:13.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:13.036+0000 7fe972a00700 1 --2- 172.21.15.60:0/3075755456 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe96c06b180 0x7fe96c06b600 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe9580098f0 tx=0x7fe958009c00 comp rx=0 tx=0).stop 2024-05-03T14:48:13.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:13.036+0000 7fe972a00700 1 -- 172.21.15.60:0/3075755456 shutdown_connections 2024-05-03T14:48:13.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:13.036+0000 7fe972a00700 1 --2- 172.21.15.60:0/3075755456 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe96c06bb40 0x7fe96c110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:13.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:13.036+0000 7fe972a00700 1 --2- 172.21.15.60:0/3075755456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe96c109510 0x7fe96c1098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:13.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:13.036+0000 7fe972a00700 1 --2- 172.21.15.60:0/3075755456 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe96c06b180 0x7fe96c06b600 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:13.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:13.036+0000 7fe972a00700 1 -- 172.21.15.60:0/3075755456 >> 172.21.15.60:0/3075755456 conn(0x7fe96c079350 msgr2=0x7fe96c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:48:13.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:13.036+0000 7fe972a00700 1 -- 172.21.15.60:0/3075755456 shutdown_connections 2024-05-03T14:48:13.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:13.036+0000 7fe972a00700 1 -- 172.21.15.60:0/3075755456 wait complete. 2024-05-03T14:48:13.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:13.037+0000 7fe972a00700 1 Processor -- start 2024-05-03T14:48:13.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:13.037+0000 7fe972a00700 1 -- start start 2024-05-03T14:48:13.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:13.038+0000 7fe972a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe96c06b180 0x7fe96c19bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:13.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:13.038+0000 7fe972a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe96c06bb40 0x7fe96c19c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:13.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:13.038+0000 7fe972a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe96c109510 0x7fe96c1a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:13.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:13.038+0000 7fe972a00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe96c113900 con 0x7fe96c06bb40 2024-05-03T14:48:13.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:13.038+0000 7fe972a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe96c113780 con 0x7fe96c06b180 2024-05-03T14:48:13.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:13.038+0000 7fe972a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe96c113a80 con 0x7fe96c109510 2024-05-03T14:48:13.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:13.038+0000 7fe96b400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe96c06bb40 0x7fe96c19c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:13.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:13.038+0000 7fe96b400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe96c06bb40 0x7fe96c19c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51714/0 (socket says 172.21.15.60:51714) 2024-05-03T14:48:13.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:13.038+0000 7fe96b400700 1 -- 172.21.15.60:0/4273345384 learned_addr learned my addr 172.21.15.60:0/4273345384 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:48:13.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:13.038+0000 7fe970a00700 1 --2- 172.21.15.60:0/4273345384 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe96c109510 0x7fe96c1a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:13.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:13.038+0000 7fe96be00700 1 --2- 172.21.15.60:0/4273345384 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe96c06b180 0x7fe96c19bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:13.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:13.039+0000 7fe96b400700 1 -- 172.21.15.60:0/4273345384 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe96c109510 msgr2=0x7fe96c1a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:13.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:13.039+0000 7fe96b400700 1 --2- 172.21.15.60:0/4273345384 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe96c109510 0x7fe96c1a0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:13.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:13.039+0000 7fe96b400700 1 -- 172.21.15.60:0/4273345384 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe96c06b180 msgr2=0x7fe96c19bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:13.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:13.039+0000 7fe96b400700 1 --2- 172.21.15.60:0/4273345384 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe96c06b180 0x7fe96c19bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:13.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:13.039+0000 7fe96b400700 1 -- 172.21.15.60:0/4273345384 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe96c1a0fd0 con 0x7fe96c06bb40 2024-05-03T14:48:13.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:13.039+0000 7fe96b400700 1 --2- 172.21.15.60:0/4273345384 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe96c06bb40 0x7fe96c19c500 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe958010f80 tx=0x7fe958010fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:48:13.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:13.039+0000 7fe968c00700 1 -- 172.21.15.60:0/4273345384 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe958027070 con 0x7fe96c06bb40 2024-05-03T14:48:13.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:13.039+0000 7fe972a00700 1 -- 172.21.15.60:0/4273345384 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe96c1a1260 con 0x7fe96c06bb40 2024-05-03T14:48:13.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:13.039+0000 7fe968c00700 1 -- 172.21.15.60:0/4273345384 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7fe95800bb60 con 0x7fe96c06bb40 2024-05-03T14:48:13.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:13.039+0000 7fe968c00700 1 -- 172.21.15.60:0/4273345384 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9580049c0 con 0x7fe96c06bb40 2024-05-03T14:48:13.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:13.040+0000 7fe972a00700 1 -- 172.21.15.60:0/4273345384 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe96c1a8b40 con 0x7fe96c06bb40 2024-05-03T14:48:13.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:13.041+0000 7fe972a00700 1 -- 172.21.15.60:0/4273345384 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe96c06a190 con 0x7fe96c06bb40 2024-05-03T14:48:13.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:13.044+0000 7fe968c00700 1 -- 172.21.15.60:0/4273345384 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fe95802a070 con 0x7fe96c06bb40 2024-05-03T14:48:13.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:13.046+0000 7fe968c00700 1 --2- 172.21.15.60:0/4273345384 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fe9500b2910 0x7fe9500b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:13.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:13.046+0000 7fe968c00700 1 -- 172.21.15.60:0/4273345384 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7fe9580e35f0 con 0x7fe96c06bb40 2024-05-03T14:48:13.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:13.046+0000 7fe96be00700 1 --2- 172.21.15.60:0/4273345384 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fe9500b2910 0x7fe9500b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:13.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:13.047+0000 7fe96be00700 1 --2- 172.21.15.60:0/4273345384 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fe9500b2910 0x7fe9500b4dd0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe95c009be0 tx=0x7fe95c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:48:13.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:13.047+0000 7fe968c00700 1 -- 172.21.15.60:0/4273345384 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fe958031080 con 0x7fe96c06bb40 2024-05-03T14:48:13.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:13.286+0000 7fe972a00700 1 -- 172.21.15.60:0/4273345384 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe96c1a8d10 con 0x7fe96c06bb40 2024-05-03T14:48:13.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:13.290+0000 7fe968c00700 1 -- 172.21.15.60:0/4273345384 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24674 (secure 0 0 0) 0x7fe958013030 con 0x7fe96c06bb40 2024-05-03T14:48:13.293 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:48:13.293 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":89,"fsid":"285d64b2-095a-11ef-bc97-c7b262605968","created":"2024-05-03T14:34:53.560460+0000","modified":"2024-05-03T14:46:05.582395+0000","last_up_change":"2024-05-03T14:43:14.877043+0000","last_in_change":"2024-05-03T14:42:55.085959+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-03T14:38:54.333714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-03T14:44:25.608599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-03T14:44:30.154009+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-03T14:44:42.549502+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f0797730-c4c1-47ea-9dce-9270d5ebb4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2952930162},{"type":"v1","addr":"172.21.15.60:6803","nonce":2952930162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2952930162},{"type":"v1","addr":"172.21.15.60:6805","nonce":2952930162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":2952930162},{"type":"v1","addr":"172.21.15.60:6809","nonce":2952930162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2952930162},{"type":"v1","addr":"172.21.15.60:6807","nonce":2952930162}]},"public_addr":"172.21.15.60:6803/2952930162","cluster_addr":"172.21.15.60:6805/2952930162","heartbeat_back_addr":"172.21.15.60:6809/2952930162","heartbeat_front_addr":"172.21.15.60:6807/2952930162","state":["exists","up"]},{"osd":1,"uuid":"96d0ad08-cda4-4824-bad4-e64973508c0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":720063242},{"type":"v1","addr":"172.21.15.90:6801","nonce":720063242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":720063242},{"type":"v1","addr":"172.21.15.90:6803","nonce":720063242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":720063242},{"type":"v1","addr":"172.21.15.90:6807","nonce":720063242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":720063242},{"type":"v1","addr":"172.21.15.90:6805","nonce":720063242}]},"public_addr":"172.21.15.90:6801/720063242","cluster_addr":"172.21.15.90:6803/720063242","heartbeat_back_addr":"172.21.15.90:6807/720063242","heartbeat_front_addr":"172.21.15.90:6805/720063242","state":["exists","up"]},{"osd":2,"uuid":"ce95fb7c-1611-4dd1-aee7-97c420d6a466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1359662077},{"type":"v1","addr":"172.21.15.149:6801","nonce":1359662077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1359662077},{"type":"v1","addr":"172.21.15.149:6803","nonce":1359662077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1359662077},{"type":"v1","addr":"172.21.15.149:6807","nonce":1359662077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1359662077},{"type":"v1","addr":"172.21.15.149:6805","nonce":1359662077}]},"public_addr":"172.21.15.149:6801/1359662077","cluster_addr":"172.21.15.149:6803/1359662077","heartbeat_back_addr":"172.21.15.149:6807/1359662077","heartbeat_front_addr":"172.21.15.149:6805/1359662077","state":["exists","up"]},{"osd":3,"uuid":"ef31136a-9b36-4b87-8389-f53d311bc5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3226133334},{"type":"v1","addr":"172.21.15.60:6811","nonce":3226133334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3226133334},{"type":"v1","addr":"172.21.15.60:6813","nonce":3226133334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3226133334},{"type":"v1","addr":"172.21.15.60:6817","nonce":3226133334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3226133334},{"type":"v1","addr":"172.21.15.60:6815","nonce":3226133334}]},"public_addr":"172.21.15.60:6811/3226133334","cluster_addr":"172.21.15.60:6813/3226133334","heartbeat_back_addr":"172.21.15.60:6817/3226133334","heartbeat_front_addr":"172.21.15.60:6815/3226133334","state":["exists","up"]},{"osd":4,"uuid":"73db7408-a2ff-47e9-b13d-d31688a93610","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2201391883},{"type":"v1","addr":"172.21.15.90:6809","nonce":2201391883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":2201391883},{"type":"v1","addr":"172.21.15.90:6811","nonce":2201391883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":2201391883},{"type":"v1","addr":"172.21.15.90:6815","nonce":2201391883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":2201391883},{"type":"v1","addr":"172.21.15.90:6813","nonce":2201391883}]},"public_addr":"172.21.15.90:6809/2201391883","cluster_addr":"172.21.15.90:6811/2201391883","heartbeat_back_addr":"172.21.15.90:6815/2201391883","heartbeat_front_addr":"172.21.15.90:6813/2201391883","state":["exists","up"]},{"osd":5,"uuid":"aefb78a2-522a-4b35-92b8-e5c04fa47b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2139732836},{"type":"v1","addr":"172.21.15.149:6809","nonce":2139732836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":2139732836},{"type":"v1","addr":"172.21.15.149:6811","nonce":2139732836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":2139732836},{"type":"v1","addr":"172.21.15.149:6815","nonce":2139732836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":2139732836},{"type":"v1","addr":"172.21.15.149:6813","nonce":2139732836}]},"public_addr":"172.21.15.149:6809/2139732836","cluster_addr":"172.21.15.149:6811/2139732836","heartbeat_back_addr":"172.21.15.149:6815/2139732836","heartbeat_front_addr":"172.21.15.149:6813/2139732836","state":["exists","up"]},{"osd":6,"uuid":"7c887819-6182-4636-8cfd-d845c30bd13c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3738521002},{"type":"v1","addr":"172.21.15.60:6819","nonce":3738521002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3738521002},{"type":"v1","addr":"172.21.15.60:6821","nonce":3738521002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3738521002},{"type":"v1","addr":"172.21.15.60:6825","nonce":3738521002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3738521002},{"type":"v1","addr":"172.21.15.60:6823","nonce":3738521002}]},"public_addr":"172.21.15.60:6819/3738521002","cluster_addr":"172.21.15.60:6821/3738521002","heartbeat_back_addr":"172.21.15.60:6825/3738521002","heartbeat_front_addr":"172.21.15.60:6823/3738521002","state":["exists","up"]},{"osd":7,"uuid":"ebce6acf-8873-4b35-bd5f-80de2163a307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":1686298661},{"type":"v1","addr":"172.21.15.90:6817","nonce":1686298661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":1686298661},{"type":"v1","addr":"172.21.15.90:6819","nonce":1686298661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":1686298661},{"type":"v1","addr":"172.21.15.90:6823","nonce":1686298661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1686298661},{"type":"v1","addr":"172.21.15.90:6821","nonce":1686298661}]},"public_addr":"172.21.15.90:6817/1686298661","cluster_addr":"172.21.15.90:6819/1686298661","heartbeat_back_addr":"172.21.15.90:6823/1686298661","heartbeat_front_addr":"172.21.15.90:6821/1686298661","state":["exists","up"]},{"osd":8,"uuid":"4ae04b98-e9b2-4956-83d9-17e5ac3c963f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":2608348248},{"type":"v1","addr":"172.21.15.149:6817","nonce":2608348248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2608348248},{"type":"v1","addr":"172.21.15.149:6819","nonce":2608348248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2608348248},{"type":"v1","addr":"172.21.15.149:6823","nonce":2608348248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2608348248},{"type":"v1","addr":"172.21.15.149:6821","nonce":2608348248}]},"public_addr":"172.21.15.149:6817/2608348248","cluster_addr":"172.21.15.149:6819/2608348248","heartbeat_back_addr":"172.21.15.149:6823/2608348248","heartbeat_front_addr":"172.21.15.149:6821/2608348248","state":["exists","up"]},{"osd":9,"uuid":"d0c0a70d-aa76-47e4-91e8-c343c8c41197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":2156130999},{"type":"v1","addr":"172.21.15.60:6827","nonce":2156130999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":2156130999},{"type":"v1","addr":"172.21.15.60:6829","nonce":2156130999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":2156130999},{"type":"v1","addr":"172.21.15.60:6833","nonce":2156130999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":2156130999},{"type":"v1","addr":"172.21.15.60:6831","nonce":2156130999}]},"public_addr":"172.21.15.60:6827/2156130999","cluster_addr":"172.21.15.60:6829/2156130999","heartbeat_back_addr":"172.21.15.60:6833/2156130999","heartbeat_front_addr":"172.21.15.60:6831/2156130999","state":["exists","up"]},{"osd":10,"uuid":"cdc3d7b8-33c7-4cc4-be4e-6887d15a3d8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":2492593590},{"type":"v1","addr":"172.21.15.90:6825","nonce":2492593590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":2492593590},{"type":"v1","addr":"172.21.15.90:6827","nonce":2492593590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":2492593590},{"type":"v1","addr":"172.21.15.90:6831","nonce":2492593590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":2492593590},{"type":"v1","addr":"172.21.15.90:6829","nonce":2492593590}]},"public_addr":"172.21.15.90:6825/2492593590","cluster_addr":"172.21.15.90:6827/2492593590","heartbeat_back_addr":"172.21.15.90:6831/2492593590","heartbeat_front_addr":"172.21.15.90:6829/2492593590","state":["exists","up"]},{"osd":11,"uuid":"8d17ec1a-bdd5-41f1-9b59-c2b6a527bfbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":1063245126},{"type":"v1","addr":"172.21.15.149:6825","nonce":1063245126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":1063245126},{"type":"v1","addr":"172.21.15.149:6827","nonce":1063245126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":1063245126},{"type":"v1","addr":"172.21.15.149:6831","nonce":1063245126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":1063245126},{"type":"v1","addr":"172.21.15.149:6829","nonce":1063245126}]},"public_addr":"172.21.15.149:6825/1063245126","cluster_addr":"172.21.15.149:6827/1063245126","heartbeat_back_addr":"172.21.15.149:6831/1063245126","heartbeat_front_addr":"172.21.15.149:6829/1063245126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:38:02.056563+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:38:27.140677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:38:51.287070+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:39:25.484410+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:39:51.483133+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:40:17.200423+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:40:51.979804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:41:18.742542+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:41:44.922541+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:42:19.778451+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:42:47.438440+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:43:13.429419+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6801/424391417":"2024-05-04T14:36:04.154333+0000","172.21.15.60:6800/424391417":"2024-05-04T14:36:04.154333+0000","172.21.15.60:0/1426532561":"2024-05-04T14:36:04.154333+0000","172.21.15.60:0/3230079926":"2024-05-04T14:35:28.066961+0000","172.21.15.60:0/2334895610":"2024-05-04T14:35:28.066961+0000","172.21.15.60:6800/888447185":"2024-05-04T14:35:28.066961+0000","172.21.15.60:0/3996899937":"2024-05-04T14:35:28.066961+0000","172.21.15.60:0/2779644018":"2024-05-04T14:36:04.154333+0000","172.21.15.60:0/71440838":"2024-05-04T14:36:04.154333+0000","172.21.15.60:6801/888447185":"2024-05-04T14:35:28.066961+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-03T14:48:13.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:13.293+0000 7fe972a00700 1 -- 172.21.15.60:0/4273345384 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fe9500b2910 msgr2=0x7fe9500b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:13.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:13.293+0000 7fe972a00700 1 --2- 172.21.15.60:0/4273345384 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fe9500b2910 0x7fe9500b4dd0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe95c009be0 tx=0x7fe95c009260 comp rx=0 tx=0).stop 2024-05-03T14:48:13.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:13.293+0000 7fe972a00700 1 -- 172.21.15.60:0/4273345384 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe96c06bb40 msgr2=0x7fe96c19c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:13.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:13.293+0000 7fe972a00700 1 --2- 172.21.15.60:0/4273345384 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe96c06bb40 0x7fe96c19c500 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe958010f80 tx=0x7fe958010fb0 comp rx=0 tx=0).stop 2024-05-03T14:48:13.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:13.293+0000 7fe972a00700 1 -- 172.21.15.60:0/4273345384 shutdown_connections 2024-05-03T14:48:13.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:13.293+0000 7fe972a00700 1 --2- 172.21.15.60:0/4273345384 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fe9500b2910 0x7fe9500b4dd0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:13.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:13.293+0000 7fe972a00700 1 --2- 172.21.15.60:0/4273345384 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe96c109510 0x7fe96c1a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:13.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:13.293+0000 7fe972a00700 1 --2- 172.21.15.60:0/4273345384 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe96c06b180 0x7fe96c19bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:13.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:13.293+0000 7fe972a00700 1 --2- 172.21.15.60:0/4273345384 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe96c06bb40 0x7fe96c19c500 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:13.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:13.293+0000 7fe972a00700 1 -- 172.21.15.60:0/4273345384 >> 172.21.15.60:0/4273345384 conn(0x7fe96c079350 msgr2=0x7fe96c10e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:48:13.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:13.294+0000 7fe972a00700 1 -- 172.21.15.60:0/4273345384 shutdown_connections 2024-05-03T14:48:13.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:13.294+0000 7fe972a00700 1 -- 172.21.15.60:0/4273345384 wait complete. 2024-05-03T14:48:13.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:48:13 smithi149 ceph-mon[33332]: pgmap v443: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T14:48:13.924 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:48:13 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/4273345384' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T14:48:13.988 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:48:13 smithi060 ceph-mon[28873]: pgmap v443: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T14:48:13.988 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:48:13 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/4273345384' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T14:48:14.074 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T14:48:14.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:48:13 smithi090 ceph-mon[33568]: pgmap v443: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T14:48:14.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:48:13 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/4273345384' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T14:48:14.567 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:48:15.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:48:15 smithi060 ceph-mon[28873]: pgmap v444: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:48:15.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:48:15 smithi149 ceph-mon[33332]: pgmap v444: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:48:16.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:48:15 smithi090 ceph-mon[33568]: pgmap v444: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:48:16.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:16.781+0000 7f42e9600700 1 -- 172.21.15.60:0/2665647023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42e4104590 msgr2=0x7f42e4104970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:16.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:16.781+0000 7f42e9600700 1 --2- 172.21.15.60:0/2665647023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42e4104590 0x7f42e4104970 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f42d4009990 tx=0x7f42d4009ca0 comp rx=0 tx=0).stop 2024-05-03T14:48:16.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:16.781+0000 7f42e9600700 1 -- 172.21.15.60:0/2665647023 shutdown_connections 2024-05-03T14:48:16.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:16.781+0000 7f42e9600700 1 --2- 172.21.15.60:0/2665647023 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f42e410d9c0 0x7f42e410fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:16.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:16.781+0000 7f42e9600700 1 --2- 172.21.15.60:0/2665647023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42e4104590 0x7f42e4104970 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:16.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:16.781+0000 7f42e9600700 1 --2- 172.21.15.60:0/2665647023 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f42e4104eb0 0x7f42e410d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:16.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:16.781+0000 7f42e9600700 1 -- 172.21.15.60:0/2665647023 >> 172.21.15.60:0/2665647023 conn(0x7f42e40fddc0 msgr2=0x7f42e41001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:48:16.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:16.781+0000 7f42e9600700 1 -- 172.21.15.60:0/2665647023 shutdown_connections 2024-05-03T14:48:16.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:16.781+0000 7f42e9600700 1 -- 172.21.15.60:0/2665647023 wait complete. 2024-05-03T14:48:16.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:16.783+0000 7f42e9600700 1 Processor -- start 2024-05-03T14:48:16.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:16.783+0000 7f42e9600700 1 -- start start 2024-05-03T14:48:16.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:16.783+0000 7f42e9600700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42e4104590 0x7f42e4199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:16.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:16.783+0000 7f42e9600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f42e4104eb0 0x7f42e419a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:16.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:16.783+0000 7f42e9600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f42e410d9c0 0x7f42e419e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:16.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:16.783+0000 7f42e9600700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f42e4113870 con 0x7f42e4104590 2024-05-03T14:48:16.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:16.783+0000 7f42e9600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f42e41136f0 con 0x7f42e410d9c0 2024-05-03T14:48:16.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:16.783+0000 7f42e9600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f42e41139f0 con 0x7f42e4104eb0 2024-05-03T14:48:16.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:16.784+0000 7f42e3e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42e4104590 0x7f42e4199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:16.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:16.784+0000 7f42e8c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f42e410d9c0 0x7f42e419e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:16.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:16.784+0000 7f42e3e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42e4104590 0x7f42e4199d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51720/0 (socket says 172.21.15.60:51720) 2024-05-03T14:48:16.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:16.784+0000 7f42e8c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f42e410d9c0 0x7f42e419e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.60:41280/0 (socket says 172.21.15.60:41280) 2024-05-03T14:48:16.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:16.784+0000 7f42e3e00700 1 -- 172.21.15.60:0/3782537757 learned_addr learned my addr 172.21.15.60:0/3782537757 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:48:16.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:16.784+0000 7f42db400700 1 --2- 172.21.15.60:0/3782537757 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f42e4104eb0 0x7f42e419a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:16.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:16.784+0000 7f42e8c00700 1 -- 172.21.15.60:0/3782537757 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f42e4104eb0 msgr2=0x7f42e419a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:16.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:16.784+0000 7f42e8c00700 1 --2- 172.21.15.60:0/3782537757 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f42e4104eb0 0x7f42e419a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:16.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:16.784+0000 7f42e8c00700 1 -- 172.21.15.60:0/3782537757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42e4104590 msgr2=0x7f42e4199d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:16.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:16.784+0000 7f42e8c00700 1 --2- 172.21.15.60:0/3782537757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42e4104590 0x7f42e4199d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:16.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:16.784+0000 7f42e8c00700 1 -- 172.21.15.60:0/3782537757 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42e419ecd0 con 0x7f42e410d9c0 2024-05-03T14:48:16.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:16.784+0000 7f42e3e00700 1 --2- 172.21.15.60:0/3782537757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42e4104590 0x7f42e4199d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:48:16.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:16.785+0000 7f42e8c00700 1 --2- 172.21.15.60:0/3782537757 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f42e410d9c0 0x7f42e419e590 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f42dc00efb0 tx=0x7f42dc00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:48:16.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:16.785+0000 7f42e1600700 1 -- 172.21.15.60:0/3782537757 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42dc003a00 con 0x7f42e410d9c0 2024-05-03T14:48:16.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:16.785+0000 7f42e1600700 1 -- 172.21.15.60:0/3782537757 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f42dc003ba0 con 0x7f42e410d9c0 2024-05-03T14:48:16.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:16.785+0000 7f42e9600700 1 -- 172.21.15.60:0/3782537757 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42e419efc0 con 0x7f42e410d9c0 2024-05-03T14:48:16.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:16.785+0000 7f42e1600700 1 -- 172.21.15.60:0/3782537757 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42dc007500 con 0x7f42e410d9c0 2024-05-03T14:48:16.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:16.785+0000 7f42e9600700 1 -- 172.21.15.60:0/3782537757 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42e41a6840 con 0x7f42e410d9c0 2024-05-03T14:48:16.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:16.787+0000 7f42e9600700 1 -- 172.21.15.60:0/3782537757 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42e406a190 con 0x7f42e410d9c0 2024-05-03T14:48:16.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:16.788+0000 7f42e1600700 1 -- 172.21.15.60:0/3782537757 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f42dc012050 con 0x7f42e410d9c0 2024-05-03T14:48:16.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:16.790+0000 7f42e1600700 1 --2- 172.21.15.60:0/3782537757 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f42c80b2990 0x7f42c80b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:16.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:16.790+0000 7f42e1600700 1 -- 172.21.15.60:0/3782537757 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f42dc0e1eb0 con 0x7f42e410d9c0 2024-05-03T14:48:16.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:16.790+0000 7f42e3e00700 1 --2- 172.21.15.60:0/3782537757 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f42c80b2990 0x7f42c80b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:16.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:16.791+0000 7f42e3e00700 1 --2- 172.21.15.60:0/3782537757 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f42c80b2990 0x7f42c80b4e50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f42d4006010 tx=0x7f42d4009590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:48:16.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:16.794+0000 7f42e1600700 1 -- 172.21.15.60:0/3782537757 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f42dc022060 con 0x7f42e410d9c0 2024-05-03T14:48:17.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:17.082+0000 7f42e9600700 1 -- 172.21.15.60:0/3782537757 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f42e402e510 con 0x7f42e410d9c0 2024-05-03T14:48:17.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:17.083+0000 7f42e1600700 1 -- 172.21.15.60:0/3782537757 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11929 (secure 0 0 0) 0x7f42dc010040 con 0x7f42e410d9c0 2024-05-03T14:48:17.086 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:48:17.086 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1849151522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1849151522},{"type":"v1","addr":"172.21.15.60:6835","nonce":1849151522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/523710346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":523710346},{"type":"v1","addr":"172.21.15.60:6837","nonce":523710346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T14:44:34.226052+0000","modified":"2024-05-03T14:46:01.367913+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24511,"mds_1":24509,"mds_2":24487,"mds_3":24475,"mds_4":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14397":{"gid":14397,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.60:6839/3846205424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3846205424},{"type":"v1","addr":"172.21.15.60:6839","nonce":3846205424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6833/3033424631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":3033424631},{"type":"v1","addr":"172.21.15.90:6833","nonce":3033424631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6833/3454021291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3454021291},{"type":"v1","addr":"172.21.15.149:6833","nonce":3454021291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6835/1057049585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1057049585},{"type":"v1","addr":"172.21.15.90:6835","nonce":1057049585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/882239924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":882239924},{"type":"v1","addr":"172.21.15.149:6835","nonce":882239924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":31,"addr":"172.21.15.90:6837/2243702844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":2243702844},{"type":"v1","addr":"172.21.15.90:6837","nonce":2243702844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":32,"addr":"172.21.15.149:6837/2611278690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2611278690},{"type":"v1","addr":"172.21.15.149:6837","nonce":2611278690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.149:6839/2527622212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2527622212},{"type":"v1","addr":"172.21.15.149:6839","nonce":2527622212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.90:6839/3512381172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3512381172},{"type":"v1","addr":"172.21.15.90:6839","nonce":3512381172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.60:6841/924163182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":924163182},{"type":"v1","addr":"172.21.15.60:6841","nonce":924163182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24511,"qdb_cluster":[24470,24475,24487,24509,24511]},"id":1}]} 2024-05-03T14:48:17.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:17.086+0000 7f42e9600700 1 -- 172.21.15.60:0/3782537757 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f42c80b2990 msgr2=0x7f42c80b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:17.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:17.086+0000 7f42e9600700 1 --2- 172.21.15.60:0/3782537757 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f42c80b2990 0x7f42c80b4e50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f42d4006010 tx=0x7f42d4009590 comp rx=0 tx=0).stop 2024-05-03T14:48:17.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:17.087+0000 7f42e9600700 1 -- 172.21.15.60:0/3782537757 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f42e410d9c0 msgr2=0x7f42e419e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:17.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:17.087+0000 7f42e9600700 1 --2- 172.21.15.60:0/3782537757 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f42e410d9c0 0x7f42e419e590 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f42dc00efb0 tx=0x7f42dc00c3a0 comp rx=0 tx=0).stop 2024-05-03T14:48:17.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:17.087+0000 7f42e9600700 1 -- 172.21.15.60:0/3782537757 shutdown_connections 2024-05-03T14:48:17.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:17.087+0000 7f42e9600700 1 --2- 172.21.15.60:0/3782537757 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f42c80b2990 0x7f42c80b4e50 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:17.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:17.087+0000 7f42e9600700 1 --2- 172.21.15.60:0/3782537757 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f42e410d9c0 0x7f42e419e590 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:17.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:17.087+0000 7f42e9600700 1 --2- 172.21.15.60:0/3782537757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42e4104590 0x7f42e4199d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:17.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:17.087+0000 7f42e9600700 1 --2- 172.21.15.60:0/3782537757 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f42e4104eb0 0x7f42e419a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:17.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:17.087+0000 7f42e9600700 1 -- 172.21.15.60:0/3782537757 >> 172.21.15.60:0/3782537757 conn(0x7f42e40fddc0 msgr2=0x7f42e40fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:48:17.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:17.087+0000 7f42e9600700 1 -- 172.21.15.60:0/3782537757 shutdown_connections 2024-05-03T14:48:17.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:17.087+0000 7f42e9600700 1 -- 172.21.15.60:0/3782537757 wait complete. 2024-05-03T14:48:17.091 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 29 2024-05-03T14:48:18.022 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:48:17 smithi060 ceph-mon[28873]: pgmap v445: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:48:18.022 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:48:17 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/3782537757' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T14:48:18.100 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-03T14:44:34.226052+0000', 'modified': '2024-05-03T14:46:01.367913+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24511, 'mds_1': 24509, 'mds_2': 24487, 'mds_3': 24475, 'mds_4': 24470}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14397': {'gid': 14397, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.60:6839/3846205424', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6838', 'nonce': 3846205424}, {'type': 'v1', 'addr': '172.21.15.60:6839', 'nonce': 3846205424}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24425': {'gid': 24425, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.90:6833/3033424631', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.90:6832', 'nonce': 3033424631}, {'type': 'v1', 'addr': '172.21.15.90:6833', 'nonce': 3033424631}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24427': {'gid': 24427, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.149:6833/3454021291', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6832', 'nonce': 3454021291}, {'type': 'v1', 'addr': '172.21.15.149:6833', 'nonce': 3454021291}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24455': {'gid': 24455, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.90:6835/1057049585', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.90:6834', 'nonce': 1057049585}, {'type': 'v1', 'addr': '172.21.15.90:6835', 'nonce': 1057049585}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.149:6835/882239924', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6834', 'nonce': 882239924}, {'type': 'v1', 'addr': '172.21.15.149:6835', 'nonce': 882239924}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24470': {'gid': 24470, 'name': 'h', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 31, 'addr': '172.21.15.90:6837/2243702844', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.90:6836', 'nonce': 2243702844}, {'type': 'v1', 'addr': '172.21.15.90:6837', 'nonce': 2243702844}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24475': {'gid': 24475, 'name': 'c', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 32, 'addr': '172.21.15.149:6837/2611278690', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6836', 'nonce': 2611278690}, {'type': 'v1', 'addr': '172.21.15.149:6837', 'nonce': 2611278690}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 30, 'addr': '172.21.15.149:6839/2527622212', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6838', 'nonce': 2527622212}, {'type': 'v1', 'addr': '172.21.15.149:6839', 'nonce': 2527622212}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24509': {'gid': 24509, 'name': 'b', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 26, 'addr': '172.21.15.90:6839/3512381172', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.90:6838', 'nonce': 3512381172}, {'type': 'v1', 'addr': '172.21.15.90:6839', 'nonce': 3512381172}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24511': {'gid': 24511, 'name': 'a', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.60:6841/924163182', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6840', 'nonce': 924163182}, {'type': 'v1', 'addr': '172.21.15.60:6841', 'nonce': 924163182}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24511, 'qdb_cluster': [24470, 24475, 24487, 24509, 24511]} 2024-05-03T14:48:18.101 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-03T14:48:18.101 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph tell mds.h status 2024-05-03T14:48:18.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:48:17 smithi090 ceph-mon[33568]: pgmap v445: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:48:18.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:48:17 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/3782537757' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T14:48:18.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:48:17 smithi149 ceph-mon[33332]: pgmap v445: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:48:18.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:48:17 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/3782537757' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T14:48:18.561 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:48:19.740 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:48:19 smithi060 ceph-mon[28873]: pgmap v446: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:48:20.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:48:19 smithi090 ceph-mon[33568]: pgmap v446: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:48:20.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:48:19 smithi149 ceph-mon[33332]: pgmap v446: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:48:20.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.803+0000 7f30b1200700 1 -- 172.21.15.60:0/1624881172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30ac06b180 msgr2=0x7f30ac06b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:20.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.803+0000 7f30b1200700 1 --2- 172.21.15.60:0/1624881172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30ac06b180 0x7f30ac06b600 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3094009990 tx=0x7f3094009ca0 comp rx=0 tx=0).stop 2024-05-03T14:48:20.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.803+0000 7f30b1200700 1 -- 172.21.15.60:0/1624881172 shutdown_connections 2024-05-03T14:48:20.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.803+0000 7f30b1200700 1 --2- 172.21.15.60:0/1624881172 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f30ac06bb40 0x7f30ac110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:20.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.803+0000 7f30b1200700 1 --2- 172.21.15.60:0/1624881172 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30ac109510 0x7f30ac1098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:20.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.803+0000 7f30b1200700 1 --2- 172.21.15.60:0/1624881172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30ac06b180 0x7f30ac06b600 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:20.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.803+0000 7f30b1200700 1 -- 172.21.15.60:0/1624881172 >> 172.21.15.60:0/1624881172 conn(0x7f30ac079350 msgr2=0x7f30ac079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:48:20.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.804+0000 7f30b1200700 1 -- 172.21.15.60:0/1624881172 shutdown_connections 2024-05-03T14:48:20.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.804+0000 7f30b1200700 1 -- 172.21.15.60:0/1624881172 wait complete. 2024-05-03T14:48:20.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.805+0000 7f30b1200700 1 Processor -- start 2024-05-03T14:48:20.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.805+0000 7f30b1200700 1 -- start start 2024-05-03T14:48:20.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.806+0000 7f30b1200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30ac06b180 0x7f30ac19beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:20.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.806+0000 7f30b1200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f30ac06bb40 0x7f30ac19c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:20.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.806+0000 7f30b1200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30ac109510 0x7f30ac1a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:20.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.806+0000 7f30b1200700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f30ac1138b0 con 0x7f30ac109510 2024-05-03T14:48:20.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.806+0000 7f30b1200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f30ac113730 con 0x7f30ac06bb40 2024-05-03T14:48:20.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.806+0000 7f30b1200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f30ac113a30 con 0x7f30ac06b180 2024-05-03T14:48:20.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.806+0000 7f30aa800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30ac109510 0x7f30ac1a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:20.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.806+0000 7f30a9e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30ac06b180 0x7f30ac19beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:20.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.806+0000 7f30a9e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30ac06b180 0x7f30ac19beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.60:55014/0 (socket says 172.21.15.60:55014) 2024-05-03T14:48:20.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.806+0000 7f30aa800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30ac109510 0x7f30ac1a0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51734/0 (socket says 172.21.15.60:51734) 2024-05-03T14:48:20.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.806+0000 7f30a9e00700 1 -- 172.21.15.60:0/2595160863 learned_addr learned my addr 172.21.15.60:0/2595160863 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:48:20.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.806+0000 7f30a3e00700 1 --2- 172.21.15.60:0/2595160863 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f30ac06bb40 0x7f30ac19c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:20.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.806+0000 7f30a9e00700 1 -- 172.21.15.60:0/2595160863 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f30ac06bb40 msgr2=0x7f30ac19c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:20.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.807+0000 7f30a9e00700 1 --2- 172.21.15.60:0/2595160863 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f30ac06bb40 0x7f30ac19c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:20.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.807+0000 7f30a9e00700 1 -- 172.21.15.60:0/2595160863 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30ac109510 msgr2=0x7f30ac1a0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:20.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.807+0000 7f30a9e00700 1 --2- 172.21.15.60:0/2595160863 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30ac109510 0x7f30ac1a0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:20.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.807+0000 7f30a9e00700 1 -- 172.21.15.60:0/2595160863 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30ac1a0ec0 con 0x7f30ac06b180 2024-05-03T14:48:20.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.807+0000 7f30aa800700 1 --2- 172.21.15.60:0/2595160863 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30ac109510 0x7f30ac1a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:48:20.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.807+0000 7f30a9e00700 1 --2- 172.21.15.60:0/2595160863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30ac06b180 0x7f30ac19beb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f309c00e9a0 tx=0x7f309c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:48:20.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.807+0000 7f30a2a00700 1 -- 172.21.15.60:0/2595160863 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309c00ca10 con 0x7f30ac06b180 2024-05-03T14:48:20.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.807+0000 7f30b1200700 1 -- 172.21.15.60:0/2595160863 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30ac1a11b0 con 0x7f30ac06b180 2024-05-03T14:48:20.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.807+0000 7f30a2a00700 1 -- 172.21.15.60:0/2595160863 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f309c00cbb0 con 0x7f30ac06b180 2024-05-03T14:48:20.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.807+0000 7f30a2a00700 1 -- 172.21.15.60:0/2595160863 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309c0187a0 con 0x7f30ac06b180 2024-05-03T14:48:20.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.807+0000 7f30b1200700 1 -- 172.21.15.60:0/2595160863 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30ac1a8a30 con 0x7f30ac06b180 2024-05-03T14:48:20.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.811+0000 7f30a2a00700 1 -- 172.21.15.60:0/2595160863 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f309c010430 con 0x7f30ac06b180 2024-05-03T14:48:20.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.812+0000 7f30bbb91b80 1 Processor -- start 2024-05-03T14:48:20.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.812+0000 7f30bbb91b80 1 -- start start 2024-05-03T14:48:20.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.812+0000 7f30a2a00700 1 --2- 172.21.15.60:0/2595160863 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f30900b2990 0x7f30900b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:20.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.812+0000 7f30a2a00700 1 -- 172.21.15.60:0/2595160863 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f309c0e2830 con 0x7f30ac06b180 2024-05-03T14:48:20.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.812+0000 7f30bbb91b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e437838e0 0x561e43783cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:20.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.812+0000 7f30bbb91b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561e43784200 0x561e4377dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:20.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.813+0000 7f30aa800700 1 --2- 172.21.15.60:0/2595160863 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f30900b2990 0x7f30900b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:20.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.813+0000 7f30bbb91b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e4377e500 0x561e4377e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:20.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.813+0000 7f30bbb91b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x561e436d09d0 con 0x561e43784200 2024-05-03T14:48:20.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.813+0000 7f30bbb91b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561e4366e710 con 0x561e437838e0 2024-05-03T14:48:20.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.813+0000 7f30bbb91b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x561e436cc990 con 0x561e4377e500 2024-05-03T14:48:20.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.813+0000 7f30a3e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561e43784200 0x561e4377dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:20.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.813+0000 7f30a9e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e437838e0 0x561e43783cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:20.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.813+0000 7f30a9e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e437838e0 0x561e43783cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.60:41306/0 (socket says 172.21.15.60:41306) 2024-05-03T14:48:20.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.813+0000 7f30a9e00700 1 -- 172.21.15.60:0/2275450668 learned_addr learned my addr 172.21.15.60:0/2275450668 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:48:20.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.813+0000 7f30aa800700 1 --2- 172.21.15.60:0/2275450668 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e4377e500 0x561e4377e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:20.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.813+0000 7f30a9e00700 1 -- 172.21.15.60:0/2275450668 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e4377e500 msgr2=0x561e4377e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:48:20.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.813+0000 7f30a9e00700 1 --2- 172.21.15.60:0/2275450668 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e4377e500 0x561e4377e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:20.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.813+0000 7f30a9e00700 1 -- 172.21.15.60:0/2275450668 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561e43784200 msgr2=0x561e4377dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:48:20.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.813+0000 7f30a9e00700 1 --2- 172.21.15.60:0/2275450668 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561e43784200 0x561e4377dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:20.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.813+0000 7f30a9e00700 1 -- 172.21.15.60:0/2275450668 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e4361d880 con 0x561e437838e0 2024-05-03T14:48:20.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.813+0000 7f30aa800700 1 --2- 172.21.15.60:0/2275450668 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e4377e500 0x561e4377e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T14:48:20.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.814+0000 7f30aa800700 1 --2- 172.21.15.60:0/2595160863 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f30900b2990 0x7f30900b4e50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f30a4009ec0 tx=0x7f30a4004570 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:48:20.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.814+0000 7f30a9e00700 1 --2- 172.21.15.60:0/2275450668 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e437838e0 0x561e43783cc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f309c00c510 tx=0x7f309c0a9390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6032f709e34ff7f server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:48:20.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.814+0000 7f309b400700 1 -- 172.21.15.60:0/2275450668 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309c0aa010 con 0x561e437838e0 2024-05-03T14:48:20.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.814+0000 7f309b400700 1 -- 172.21.15.60:0/2275450668 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f309c0a9910 con 0x561e437838e0 2024-05-03T14:48:20.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.814+0000 7f309b400700 1 -- 172.21.15.60:0/2275450668 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309c0cc0b0 con 0x561e437838e0 2024-05-03T14:48:20.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.815+0000 7f30bbb91b80 1 -- 172.21.15.60:0/2275450668 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e437838e0 msgr2=0x561e43783cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:20.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.815+0000 7f30bbb91b80 1 --2- 172.21.15.60:0/2275450668 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e437838e0 0x561e43783cc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f309c00c510 tx=0x7f309c0a9390 comp rx=0 tx=0).stop 2024-05-03T14:48:20.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.815+0000 7f30bbb91b80 1 -- 172.21.15.60:0/2275450668 shutdown_connections 2024-05-03T14:48:20.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.815+0000 7f30bbb91b80 1 --2- 172.21.15.60:0/2275450668 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e4377e500 0x561e4377e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:20.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.815+0000 7f30bbb91b80 1 --2- 172.21.15.60:0/2275450668 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e437838e0 0x561e43783cc0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:20.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.815+0000 7f30bbb91b80 1 --2- 172.21.15.60:0/2275450668 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561e43784200 0x561e4377dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:20.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.815+0000 7f30bbb91b80 1 -- 172.21.15.60:0/2275450668 >> 172.21.15.60:0/2275450668 conn(0x561e436f6680 msgr2=0x561e437788b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:48:20.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.815+0000 7f30bbb91b80 1 -- 172.21.15.60:0/2275450668 shutdown_connections 2024-05-03T14:48:20.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.815+0000 7f30bbb91b80 1 -- 172.21.15.60:0/2275450668 wait complete. 2024-05-03T14:48:20.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.815+0000 7f30bbb91b80 1 -- start start 2024-05-03T14:48:20.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.816+0000 7f30bbb91b80 1 Processor -- start 2024-05-03T14:48:20.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.816+0000 7f30bbb91b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561e4377e500 0x561e43789df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:20.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.816+0000 7f30bbb91b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e437838e0 0x561e43787190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:20.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.816+0000 7f30bbb91b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e43784200 0x561e437876d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:20.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.816+0000 7f30bbb91b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x561e4378ac50 con 0x561e4377e500 2024-05-03T14:48:20.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.816+0000 7f30bbb91b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561e43636ad0 con 0x561e43784200 2024-05-03T14:48:20.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.816+0000 7f30bbb91b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x561e4378add0 con 0x561e437838e0 2024-05-03T14:48:20.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.816+0000 7f30bbb91b80 10 client.0 _ll_register_callbacks cb 0x561e43677c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T14:48:20.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.816+0000 7f30a9e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561e4377e500 0x561e43789df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:20.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.816+0000 7f30a9e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561e4377e500 0x561e43789df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51754/0 (socket says 172.21.15.60:51754) 2024-05-03T14:48:20.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.817+0000 7f30a9e00700 1 -- 172.21.15.60:0/3386561023 learned_addr learned my addr 172.21.15.60:0/3386561023 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:48:20.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.817+0000 7f30a3e00700 1 --2- 172.21.15.60:0/3386561023 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e437838e0 0x561e43787190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:20.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.817+0000 7f30aa800700 1 --2- 172.21.15.60:0/3386561023 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e43784200 0x561e437876d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:20.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.817+0000 7f30aa800700 1 -- 172.21.15.60:0/3386561023 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e437838e0 msgr2=0x561e43787190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:48:20.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.817+0000 7f30aa800700 1 --2- 172.21.15.60:0/3386561023 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e437838e0 0x561e43787190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:20.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.817+0000 7f30aa800700 1 -- 172.21.15.60:0/3386561023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561e4377e500 msgr2=0x561e43789df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:48:20.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.817+0000 7f30aa800700 1 --2- 172.21.15.60:0/3386561023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561e4377e500 0x561e43789df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:20.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.817+0000 7f30aa800700 1 -- 172.21.15.60:0/3386561023 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e436963d0 con 0x561e43784200 2024-05-03T14:48:20.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.817+0000 7f30aa800700 1 --2- 172.21.15.60:0/3386561023 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e43784200 0x561e437876d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f30a4009fd0 tx=0x7f30a4004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d65b48c32ca27f61 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:48:20.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.817+0000 7f308fe00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-03T14:48:20.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.818+0000 7f308fe00700 1 -- 172.21.15.60:0/3386561023 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a4029020 con 0x561e43784200 2024-05-03T14:48:20.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.818+0000 7f30bbb91b80 1 -- 172.21.15.60:0/3386561023 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f309c0a9910 con 0x561e43784200 2024-05-03T14:48:20.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.818+0000 7f308fe00700 1 -- 172.21.15.60:0/3386561023 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f30a4025430 con 0x561e43784200 2024-05-03T14:48:20.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.818+0000 7f308fe00700 1 -- 172.21.15.60:0/3386561023 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f30a4034050 con 0x561e43784200 2024-05-03T14:48:20.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.818+0000 7f30bbb91b80 10 client.24950 fetch_fsmap learned FSMap version 29 2024-05-03T14:48:20.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.818+0000 7f30bbb91b80 1 -- 172.21.15.60:0/3386561023 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561e437953a0 con 0x561e43784200 2024-05-03T14:48:20.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.818+0000 7f308fe00700 1 -- 172.21.15.60:0/3386561023 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f309c0a9910 con 0x561e43784200 2024-05-03T14:48:20.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.819+0000 7f308fe00700 1 -- 172.21.15.60:0/3386561023 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f30a4018600 con 0x561e43784200 2024-05-03T14:48:20.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.819+0000 7f30bbb91b80 10 client.24950 fetch_fsmap finished waiting for FSMap version 29 2024-05-03T14:48:20.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.819+0000 7f30bbb91b80 10 client.24950 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-03T14:48:20.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.819+0000 7f30bbb91b80 20 client.24950 populate_metadata read hostname 'smithi060' 2024-05-03T14:48:20.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.819+0000 7f30bbb91b80 1 --2- 172.21.15.60:0/3386561023 >> [v2:172.21.15.90:6836/2243702844,v1:172.21.15.90:6837/2243702844] conn(0x561e43797a20 0x561e43798e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:20.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.819+0000 7f30bbb91b80 4 client.24950 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T14:48:20.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.819+0000 7f30bbb91b80 1 -- 172.21.15.60:0/3386561023 --> [v2:172.21.15.90:6836/2243702844,v1:172.21.15.90:6837/2243702844] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e4374afa0 con 0x561e43797a20 2024-05-03T14:48:20.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.819+0000 7f30a9e00700 1 --2- 172.21.15.60:0/3386561023 >> [v2:172.21.15.90:6836/2243702844,v1:172.21.15.90:6837/2243702844] conn(0x561e43797a20 0x561e43798e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:20.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.820+0000 7f30a9e00700 1 --2- 172.21.15.60:0/3386561023 >> [v2:172.21.15.90:6836/2243702844,v1:172.21.15.90:6837/2243702844] conn(0x561e43797a20 0x561e43798e90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1abde0a1e4d631e9 server_cookie=6c20c08883ff381d in_seq=0 out_seq=0 2024-05-03T14:48:20.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.820+0000 7f308fe00700 10 client.24950 ms_handle_connect on v2:172.21.15.90:6836/2243702844 2024-05-03T14:48:20.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.821+0000 7f308fe00700 1 -- 172.21.15.60:0/3386561023 <== mds.4 v2:172.21.15.90:6836/2243702844 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561e4374afa0 con 0x561e43797a20 2024-05-03T14:48:20.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.821+0000 7f308fe00700 10 client.24950 handle_command_reply: tid=0 2024-05-03T14:48:20.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.821+0000 7f30bbb91b80 1 client.24950 shutdown 2024-05-03T14:48:20.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.821+0000 7f30bbb91b80 1 -- 172.21.15.60:0/3386561023 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e43784200 msgr2=0x561e437876d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:20.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.821+0000 7f30bbb91b80 1 --2- 172.21.15.60:0/3386561023 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e43784200 0x561e437876d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f30a4009fd0 tx=0x7f30a4004b40 comp rx=0 tx=0).stop 2024-05-03T14:48:20.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.821+0000 7f30bbb91b80 1 -- 172.21.15.60:0/3386561023 shutdown_connections 2024-05-03T14:48:20.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.821+0000 7f30bbb91b80 1 --2- 172.21.15.60:0/3386561023 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e43784200 0x561e437876d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:20.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.821+0000 7f30bbb91b80 1 --2- 172.21.15.60:0/3386561023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561e4377e500 0x561e43789df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:20.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.821+0000 7f30bbb91b80 1 --2- 172.21.15.60:0/3386561023 >> [v2:172.21.15.90:6836/2243702844,v1:172.21.15.90:6837/2243702844] conn(0x561e43797a20 0x561e43798e90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:20.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.821+0000 7f30bbb91b80 1 --2- 172.21.15.60:0/3386561023 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e437838e0 0x561e43787190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:20.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.821+0000 7f308fe00700 0 client.24950 ms_handle_reset on v2:172.21.15.90:6836/2243702844 2024-05-03T14:48:20.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.821+0000 7f30bbb91b80 1 -- 172.21.15.60:0/3386561023 >> 172.21.15.60:0/3386561023 conn(0x561e436f6680 msgr2=0x561e43773360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:48:20.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.821+0000 7f30bbb91b80 1 -- 172.21.15.60:0/3386561023 shutdown_connections 2024-05-03T14:48:20.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.821+0000 7f30bbb91b80 1 -- 172.21.15.60:0/3386561023 wait complete. 2024-05-03T14:48:20.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.821+0000 7f30bbb91b80 20 client.24950 trim_cache size 0 max 16384 2024-05-03T14:48:20.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.844+0000 7f30bbb91b80 1 Processor -- start 2024-05-03T14:48:20.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.844+0000 7f30bbb91b80 1 -- start start 2024-05-03T14:48:20.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.845+0000 7f30bbb91b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e4377e500 0x561e4377dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:20.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.845+0000 7f30bbb91b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e437838e0 0x561e43778800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:20.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.845+0000 7f30bbb91b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561e43784200 0x561e43787150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:20.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.845+0000 7f30bbb91b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x561e437971d0 con 0x561e43784200 2024-05-03T14:48:20.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.845+0000 7f30bbb91b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561e437978a0 con 0x561e437838e0 2024-05-03T14:48:20.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.845+0000 7f30bbb91b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x561e43797f70 con 0x561e4377e500 2024-05-03T14:48:20.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.845+0000 7f30aa800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561e43784200 0x561e43787150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:20.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.845+0000 7f30aa800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561e43784200 0x561e43787150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51770/0 (socket says 172.21.15.60:51770) 2024-05-03T14:48:20.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.845+0000 7f30aa800700 1 -- 172.21.15.60:0/1300081862 learned_addr learned my addr 172.21.15.60:0/1300081862 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:48:20.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.845+0000 7f30a9e00700 1 --2- 172.21.15.60:0/1300081862 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e4377e500 0x561e4377dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:20.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.845+0000 7f30a3e00700 1 --2- 172.21.15.60:0/1300081862 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e437838e0 0x561e43778800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:20.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.846+0000 7f30aa800700 1 -- 172.21.15.60:0/1300081862 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e4377e500 msgr2=0x561e4377dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:48:20.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.846+0000 7f30aa800700 1 --2- 172.21.15.60:0/1300081862 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e4377e500 0x561e4377dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:20.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.846+0000 7f30aa800700 1 -- 172.21.15.60:0/1300081862 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e437838e0 msgr2=0x561e43778800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:48:20.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.846+0000 7f30aa800700 1 --2- 172.21.15.60:0/1300081862 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e437838e0 0x561e43778800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:20.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.846+0000 7f30aa800700 1 -- 172.21.15.60:0/1300081862 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e43785700 con 0x561e43784200 2024-05-03T14:48:20.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.846+0000 7f30aa800700 1 --2- 172.21.15.60:0/1300081862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561e43784200 0x561e43787150 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f30a4000f80 tx=0x7f30a40060e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82e5901f4f1dd7c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:48:20.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.846+0000 7f309aa00700 1 -- 172.21.15.60:0/1300081862 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a403e020 con 0x561e43784200 2024-05-03T14:48:20.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.846+0000 7f309aa00700 1 -- 172.21.15.60:0/1300081862 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f30a4018c90 con 0x561e43784200 2024-05-03T14:48:20.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.846+0000 7f309aa00700 1 -- 172.21.15.60:0/1300081862 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a4029040 con 0x561e43784200 2024-05-03T14:48:20.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.847+0000 7f30bbb91b80 1 -- 172.21.15.60:0/1300081862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561e43784200 msgr2=0x561e43787150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:20.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.847+0000 7f30bbb91b80 1 --2- 172.21.15.60:0/1300081862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561e43784200 0x561e43787150 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f30a4000f80 tx=0x7f30a40060e0 comp rx=0 tx=0).stop 2024-05-03T14:48:20.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.847+0000 7f30bbb91b80 1 -- 172.21.15.60:0/1300081862 shutdown_connections 2024-05-03T14:48:20.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.847+0000 7f30bbb91b80 1 --2- 172.21.15.60:0/1300081862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561e43784200 0x561e43787150 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:20.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.847+0000 7f30bbb91b80 1 --2- 172.21.15.60:0/1300081862 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e4377e500 0x561e4377dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:20.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.847+0000 7f30bbb91b80 1 --2- 172.21.15.60:0/1300081862 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e437838e0 0x561e43778800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:20.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.847+0000 7f30bbb91b80 1 -- 172.21.15.60:0/1300081862 >> 172.21.15.60:0/1300081862 conn(0x561e436f6680 msgr2=0x561e43772300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:48:20.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.847+0000 7f30bbb91b80 1 -- 172.21.15.60:0/1300081862 shutdown_connections 2024-05-03T14:48:20.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.847+0000 7f30bbb91b80 1 -- 172.21.15.60:0/1300081862 wait complete. 2024-05-03T14:48:20.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.848+0000 7f30bbb91b80 1 -- start start 2024-05-03T14:48:20.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.848+0000 7f30bbb91b80 1 Processor -- start 2024-05-03T14:48:20.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.848+0000 7f30bbb91b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561e4377e500 0x561e437a2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:20.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.848+0000 7f30bbb91b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e437838e0 0x561e4379d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:20.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.848+0000 7f30bbb91b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e43784200 0x561e4379d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:20.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.848+0000 7f30bbb91b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x561e43798d10 con 0x561e4377e500 2024-05-03T14:48:20.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.848+0000 7f30bbb91b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561e43798640 con 0x561e437838e0 2024-05-03T14:48:20.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.848+0000 7f30bbb91b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x561e437993e0 con 0x561e43784200 2024-05-03T14:48:20.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.848+0000 7f30bbb91b80 10 client.0 _ll_register_callbacks cb 0x561e43677c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T14:48:20.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.849+0000 7f30a9e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561e4377e500 0x561e437a2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:20.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.849+0000 7f30a9e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561e4377e500 0x561e437a2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51776/0 (socket says 172.21.15.60:51776) 2024-05-03T14:48:20.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.849+0000 7f30a9e00700 1 -- 172.21.15.60:0/3517479894 learned_addr learned my addr 172.21.15.60:0/3517479894 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:48:20.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.849+0000 7f30a3e00700 1 --2- 172.21.15.60:0/3517479894 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e437838e0 0x561e4379d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:20.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.849+0000 7f30aa800700 1 --2- 172.21.15.60:0/3517479894 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e43784200 0x561e4379d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:20.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.849+0000 7f30a9e00700 1 -- 172.21.15.60:0/3517479894 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e43784200 msgr2=0x561e4379d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:48:20.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.849+0000 7f30a9e00700 1 --2- 172.21.15.60:0/3517479894 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e43784200 0x561e4379d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:20.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.849+0000 7f30a9e00700 1 -- 172.21.15.60:0/3517479894 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e437838e0 msgr2=0x561e4379d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:48:20.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.849+0000 7f30a9e00700 1 --2- 172.21.15.60:0/3517479894 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e437838e0 0x561e4379d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:20.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.849+0000 7f30a9e00700 1 -- 172.21.15.60:0/3517479894 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e43802e90 con 0x561e4377e500 2024-05-03T14:48:20.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.849+0000 7f30a9e00700 1 --2- 172.21.15.60:0/3517479894 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561e4377e500 0x561e437a2960 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f309c0a98a0 tx=0x7f309c0a9260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f5981491d09c69f server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:48:20.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.849+0000 7f308fe00700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-05-03T14:48:20.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.850+0000 7f308fe00700 1 -- 172.21.15.60:0/3517479894 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309c0a9910 con 0x561e4377e500 2024-05-03T14:48:20.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.850+0000 7f308fe00700 1 -- 172.21.15.60:0/3517479894 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f309c0109c0 con 0x561e4377e500 2024-05-03T14:48:20.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.850+0000 7f30bbb91b80 1 -- 172.21.15.60:0/3517479894 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30a4018c90 con 0x561e4377e500 2024-05-03T14:48:20.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.850+0000 7f308fe00700 1 -- 172.21.15.60:0/3517479894 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f309c0aa010 con 0x561e4377e500 2024-05-03T14:48:20.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.850+0000 7f30bbb91b80 10 client.14856 fetch_fsmap learned FSMap version 29 2024-05-03T14:48:20.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.850+0000 7f30bbb91b80 1 -- 172.21.15.60:0/3517479894 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561e43804040 con 0x561e4377e500 2024-05-03T14:48:20.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.850+0000 7f308fe00700 1 -- 172.21.15.60:0/3517479894 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f30a4018c90 con 0x561e4377e500 2024-05-03T14:48:20.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.850+0000 7f308fe00700 1 -- 172.21.15.60:0/3517479894 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f309c0cc0b0 con 0x561e4377e500 2024-05-03T14:48:20.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.850+0000 7f30bbb91b80 10 client.14856 fetch_fsmap finished waiting for FSMap version 29 2024-05-03T14:48:20.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.850+0000 7f30bbb91b80 10 client.14856 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-03T14:48:20.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.850+0000 7f30bbb91b80 20 client.14856 populate_metadata read hostname 'smithi060' 2024-05-03T14:48:20.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.850+0000 7f30bbb91b80 1 --2- 172.21.15.60:0/3517479894 >> [v2:172.21.15.90:6836/2243702844,v1:172.21.15.90:6837/2243702844] conn(0x561e438087a0 0x561e438074b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:20.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.851+0000 7f30bbb91b80 4 client.14856 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-03T14:48:20.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.851+0000 7f30bbb91b80 1 -- 172.21.15.60:0/3517479894 --> [v2:172.21.15.90:6836/2243702844,v1:172.21.15.90:6837/2243702844] -- command(tid 0: {"prefix": "status"}) -- 0x561e436de3b0 con 0x561e438087a0 2024-05-03T14:48:20.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.851+0000 7f30a3e00700 1 --2- 172.21.15.60:0/3517479894 >> [v2:172.21.15.90:6836/2243702844,v1:172.21.15.90:6837/2243702844] conn(0x561e438087a0 0x561e438074b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:20.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.851+0000 7f30a3e00700 1 --2- 172.21.15.60:0/3517479894 >> [v2:172.21.15.90:6836/2243702844,v1:172.21.15.90:6837/2243702844] conn(0x561e438087a0 0x561e438074b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b2d14fa4cc45394f server_cookie=fe31530ab7c4790e in_seq=0 out_seq=0 2024-05-03T14:48:20.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.851+0000 7f308fe00700 10 client.14856 ms_handle_connect on v2:172.21.15.90:6836/2243702844 2024-05-03T14:48:20.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.852+0000 7f308fe00700 1 -- 172.21.15.60:0/3517479894 <== mds.4 v2:172.21.15.90:6836/2243702844 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x561e436de3b0 con 0x561e438087a0 2024-05-03T14:48:20.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.852+0000 7f308fe00700 10 client.14856 handle_command_reply: tid=0 2024-05-03T14:48:20.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.852+0000 7f30bbb91b80 1 client.14856 shutdown 2024-05-03T14:48:20.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.852+0000 7f30bbb91b80 1 -- 172.21.15.60:0/3517479894 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561e4377e500 msgr2=0x561e437a2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:20.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.852+0000 7f30bbb91b80 1 --2- 172.21.15.60:0/3517479894 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561e4377e500 0x561e437a2960 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f309c0a98a0 tx=0x7f309c0a9260 comp rx=0 tx=0).stop 2024-05-03T14:48:20.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.852+0000 7f30bbb91b80 1 -- 172.21.15.60:0/3517479894 shutdown_connections 2024-05-03T14:48:20.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.852+0000 7f30bbb91b80 1 --2- 172.21.15.60:0/3517479894 >> [v2:172.21.15.90:6836/2243702844,v1:172.21.15.90:6837/2243702844] conn(0x561e438087a0 0x561e438074b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:20.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.852+0000 7f30bbb91b80 1 --2- 172.21.15.60:0/3517479894 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e43784200 0x561e4379d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:20.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.852+0000 7f30bbb91b80 1 --2- 172.21.15.60:0/3517479894 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561e4377e500 0x561e437a2960 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:20.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.852+0000 7f308fe00700 0 client.14856 ms_handle_reset on v2:172.21.15.90:6836/2243702844 2024-05-03T14:48:20.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.852+0000 7f30bbb91b80 1 --2- 172.21.15.60:0/3517479894 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e437838e0 0x561e4379d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:20.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.852+0000 7f30bbb91b80 1 -- 172.21.15.60:0/3517479894 >> 172.21.15.60:0/3517479894 conn(0x561e436f6680 msgr2=0x561e437813f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:48:20.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.852+0000 7f30bbb91b80 1 -- 172.21.15.60:0/3517479894 shutdown_connections 2024-05-03T14:48:20.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.852+0000 7f30bbb91b80 1 -- 172.21.15.60:0/3517479894 wait complete. 2024-05-03T14:48:20.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.853+0000 7f30bbb91b80 20 client.14856 trim_cache size 0 max 16384 2024-05-03T14:48:20.858 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-05-03T14:48:20.858 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "285d64b2-095a-11ef-bc97-c7b262605968", 2024-05-03T14:48:20.859 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 4, 2024-05-03T14:48:20.859 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 24470, 2024-05-03T14:48:20.859 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-05-03T14:48:20.859 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-05-03T14:48:20.859 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-05-03T14:48:20.859 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 144.32212376699999, 2024-05-03T14:48:20.859 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 29, 2024-05-03T14:48:20.859 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 89, 2024-05-03T14:48:20.859 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 87, 2024-05-03T14:48:20.859 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 263.80259298099998 2024-05-03T14:48:20.859 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-05-03T14:48:20.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.853+0000 7f30b1200700 1 -- 172.21.15.60:0/2595160863 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f30900b2990 msgr2=0x7f30900b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:20.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.853+0000 7f30b1200700 1 --2- 172.21.15.60:0/2595160863 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f30900b2990 0x7f30900b4e50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f30a4009ec0 tx=0x7f30a4004570 comp rx=0 tx=0).stop 2024-05-03T14:48:20.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.853+0000 7f30b1200700 1 -- 172.21.15.60:0/2595160863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30ac06b180 msgr2=0x7f30ac19beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:20.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.853+0000 7f30b1200700 1 --2- 172.21.15.60:0/2595160863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30ac06b180 0x7f30ac19beb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f309c00e9a0 tx=0x7f309c00ecb0 comp rx=0 tx=0).stop 2024-05-03T14:48:20.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.853+0000 7f30b1200700 1 -- 172.21.15.60:0/2595160863 shutdown_connections 2024-05-03T14:48:20.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.853+0000 7f30b1200700 1 --2- 172.21.15.60:0/2595160863 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f30900b2990 0x7f30900b4e50 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:20.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.853+0000 7f30b1200700 1 --2- 172.21.15.60:0/2595160863 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30ac109510 0x7f30ac1a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:20.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.853+0000 7f30b1200700 1 --2- 172.21.15.60:0/2595160863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30ac06b180 0x7f30ac19beb0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:20.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.853+0000 7f30b1200700 1 --2- 172.21.15.60:0/2595160863 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f30ac06bb40 0x7f30ac19c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:20.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.853+0000 7f30b1200700 1 -- 172.21.15.60:0/2595160863 >> 172.21.15.60:0/2595160863 conn(0x7f30ac079350 msgr2=0x7f30ac10e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:48:20.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.853+0000 7f30b1200700 1 -- 172.21.15.60:0/2595160863 shutdown_connections 2024-05-03T14:48:20.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:20.854+0000 7f30b1200700 1 -- 172.21.15.60:0/2595160863 wait complete. 2024-05-03T14:48:21.827 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph tell mds.c status 2024-05-03T14:48:22.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:48:21 smithi060 ceph-mon[28873]: pgmap v447: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-03T14:48:22.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:48:21 smithi090 ceph-mon[33568]: pgmap v447: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-03T14:48:22.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:48:21 smithi149 ceph-mon[33332]: pgmap v447: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-03T14:48:22.314 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:48:24.027 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:48:23 smithi060 ceph-mon[28873]: pgmap v448: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-03T14:48:24.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:48:23 smithi090 ceph-mon[33568]: pgmap v448: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-03T14:48:24.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:48:23 smithi149 ceph-mon[33332]: pgmap v448: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-03T14:48:24.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.534+0000 7f92d3200700 1 -- 172.21.15.60:0/4233080313 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92d41033d0 msgr2=0x7f92d4103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:24.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.534+0000 7f92d3200700 1 --2- 172.21.15.60:0/4233080313 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92d41033d0 0x7f92d4103850 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f92bc0098f0 tx=0x7f92bc009c00 comp rx=0 tx=0).stop 2024-05-03T14:48:24.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.534+0000 7f92d3200700 1 -- 172.21.15.60:0/4233080313 shutdown_connections 2024-05-03T14:48:24.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.534+0000 7f92d3200700 1 --2- 172.21.15.60:0/4233080313 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f92d4103d90 0x7f92d41102a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:24.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.534+0000 7f92d3200700 1 --2- 172.21.15.60:0/4233080313 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92d4109460 0x7f92d4109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:24.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.534+0000 7f92d3200700 1 --2- 172.21.15.60:0/4233080313 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92d41033d0 0x7f92d4103850 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:24.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.534+0000 7f92d3200700 1 -- 172.21.15.60:0/4233080313 >> 172.21.15.60:0/4233080313 conn(0x7f92d40fef20 msgr2=0x7f92d4101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:48:24.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.534+0000 7f92d3200700 1 -- 172.21.15.60:0/4233080313 shutdown_connections 2024-05-03T14:48:24.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.534+0000 7f92d3200700 1 -- 172.21.15.60:0/4233080313 wait complete. 2024-05-03T14:48:24.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.535+0000 7f92d3200700 1 Processor -- start 2024-05-03T14:48:24.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.535+0000 7f92d3200700 1 -- start start 2024-05-03T14:48:24.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.536+0000 7f92d3200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92d41033d0 0x7f92d40757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:24.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.536+0000 7f92d3200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f92d4103d90 0x7f92d4075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:24.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.536+0000 7f92d3200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92d4109460 0x7f92d4076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:24.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.536+0000 7f92d3200700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f92d4113870 con 0x7f92d4109460 2024-05-03T14:48:24.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.536+0000 7f92d3200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f92d41136f0 con 0x7f92d4103d90 2024-05-03T14:48:24.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.536+0000 7f92d3200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f92d41139f0 con 0x7f92d41033d0 2024-05-03T14:48:24.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.536+0000 7f92d1e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92d41033d0 0x7f92d40757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:24.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.536+0000 7f92d2800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92d4109460 0x7f92d4076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:24.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.536+0000 7f92d1e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92d41033d0 0x7f92d40757a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.60:48380/0 (socket says 172.21.15.60:48380) 2024-05-03T14:48:24.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.536+0000 7f92d1e00700 1 -- 172.21.15.60:0/961508470 learned_addr learned my addr 172.21.15.60:0/961508470 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:48:24.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.536+0000 7f92c9400700 1 --2- 172.21.15.60:0/961508470 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f92d4103d90 0x7f92d4075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:24.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.536+0000 7f92d1e00700 1 -- 172.21.15.60:0/961508470 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f92d4103d90 msgr2=0x7f92d4075ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:24.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.536+0000 7f92d1e00700 1 --2- 172.21.15.60:0/961508470 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f92d4103d90 0x7f92d4075ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:24.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.536+0000 7f92d1e00700 1 -- 172.21.15.60:0/961508470 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92d4109460 msgr2=0x7f92d4076220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:24.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.536+0000 7f92d1e00700 1 --2- 172.21.15.60:0/961508470 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92d4109460 0x7f92d4076220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:24.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.536+0000 7f92d1e00700 1 -- 172.21.15.60:0/961508470 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92d41a8210 con 0x7f92d41033d0 2024-05-03T14:48:24.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.537+0000 7f92d1e00700 1 --2- 172.21.15.60:0/961508470 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92d41033d0 0x7f92d40757a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f92c400e9f0 tx=0x7f92c400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:48:24.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.537+0000 7f92cb400700 1 -- 172.21.15.60:0/961508470 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c4004a30 con 0x7f92d41033d0 2024-05-03T14:48:24.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.537+0000 7f92d3200700 1 -- 172.21.15.60:0/961508470 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92d41a8500 con 0x7f92d41033d0 2024-05-03T14:48:24.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.537+0000 7f92cb400700 1 -- 172.21.15.60:0/961508470 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f92c400ccc0 con 0x7f92d41033d0 2024-05-03T14:48:24.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.537+0000 7f92cb400700 1 -- 172.21.15.60:0/961508470 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c40056c0 con 0x7f92d41033d0 2024-05-03T14:48:24.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.537+0000 7f92d3200700 1 -- 172.21.15.60:0/961508470 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92d41a89c0 con 0x7f92d41033d0 2024-05-03T14:48:24.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.541+0000 7f92cb400700 1 -- 172.21.15.60:0/961508470 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f92c400c7c0 con 0x7f92d41033d0 2024-05-03T14:48:24.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.542+0000 7f92e360ab80 1 Processor -- start 2024-05-03T14:48:24.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.542+0000 7f92e360ab80 1 -- start start 2024-05-03T14:48:24.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.542+0000 7f92cb400700 1 --2- 172.21.15.60:0/961508470 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f92b80b2910 0x7f92b80b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:24.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.543+0000 7f92cb400700 1 -- 172.21.15.60:0/961508470 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f92c40e2a80 con 0x7f92d41033d0 2024-05-03T14:48:24.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.543+0000 7f92e360ab80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5623632c99f0 0x5623632c9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:24.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.543+0000 7f92d2800700 1 --2- 172.21.15.60:0/961508470 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f92b80b2910 0x7f92b80b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:24.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.543+0000 7f92e360ab80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5623632ca310 0x5623632c3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:24.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.543+0000 7f92e360ab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623632c4580 0x5623632c49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:24.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.543+0000 7f92e360ab80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5623632169d0 con 0x5623632ca310 2024-05-03T14:48:24.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.543+0000 7f92e360ab80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5623631b4710 con 0x5623632c99f0 2024-05-03T14:48:24.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.543+0000 7f92e360ab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x562363212990 con 0x5623632c4580 2024-05-03T14:48:24.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.543+0000 7f92d2800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623632c4580 0x5623632c49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:24.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.543+0000 7f92d2800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623632c4580 0x5623632c49e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.60:48384/0 (socket says 172.21.15.60:48384) 2024-05-03T14:48:24.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.543+0000 7f92c9400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5623632ca310 0x5623632c3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:24.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.543+0000 7f92d2800700 1 -- 172.21.15.60:0/2623871027 learned_addr learned my addr 172.21.15.60:0/2623871027 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:48:24.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.543+0000 7f92d1e00700 1 --2- 172.21.15.60:0/2623871027 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5623632c99f0 0x5623632c9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:24.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.544+0000 7f92c9400700 1 -- 172.21.15.60:0/2623871027 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623632c4580 msgr2=0x5623632c49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:48:24.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.544+0000 7f92c9400700 1 --2- 172.21.15.60:0/2623871027 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623632c4580 0x5623632c49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:24.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.544+0000 7f92c9400700 1 -- 172.21.15.60:0/2623871027 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5623632c99f0 msgr2=0x5623632c9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:48:24.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.544+0000 7f92c9400700 1 --2- 172.21.15.60:0/2623871027 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5623632c99f0 0x5623632c9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:24.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.544+0000 7f92c9400700 1 -- 172.21.15.60:0/2623871027 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562363163880 con 0x5623632ca310 2024-05-03T14:48:24.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.544+0000 7f92d1e00700 1 --2- 172.21.15.60:0/2623871027 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5623632c99f0 0x5623632c9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:48:24.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.544+0000 7f92d2800700 1 --2- 172.21.15.60:0/961508470 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f92b80b2910 0x7f92b80b4dd0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f92cc009da0 tx=0x7f92cc013560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:48:24.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.544+0000 7f92c9400700 1 --2- 172.21.15.60:0/2623871027 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5623632ca310 0x5623632c3f60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f92bc000c00 tx=0x7f92bc009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee36b9a85cd6aab1 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:48:24.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.545+0000 7f92c3400700 1 -- 172.21.15.60:0/2623871027 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92bc026070 con 0x5623632ca310 2024-05-03T14:48:24.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.545+0000 7f92c3400700 1 -- 172.21.15.60:0/2623871027 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f92bc00f460 con 0x5623632ca310 2024-05-03T14:48:24.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.545+0000 7f92c3400700 1 -- 172.21.15.60:0/2623871027 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92bc0039e0 con 0x5623632ca310 2024-05-03T14:48:24.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.545+0000 7f92e360ab80 1 -- 172.21.15.60:0/2623871027 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5623632ca310 msgr2=0x5623632c3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:24.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.545+0000 7f92e360ab80 1 --2- 172.21.15.60:0/2623871027 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5623632ca310 0x5623632c3f60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f92bc000c00 tx=0x7f92bc009f70 comp rx=0 tx=0).stop 2024-05-03T14:48:24.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.546+0000 7f92e360ab80 1 -- 172.21.15.60:0/2623871027 shutdown_connections 2024-05-03T14:48:24.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.546+0000 7f92e360ab80 1 --2- 172.21.15.60:0/2623871027 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623632c4580 0x5623632c49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:24.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.546+0000 7f92e360ab80 1 --2- 172.21.15.60:0/2623871027 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5623632c99f0 0x5623632c9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:24.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.546+0000 7f92e360ab80 1 --2- 172.21.15.60:0/2623871027 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5623632ca310 0x5623632c3f60 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:24.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.546+0000 7f92e360ab80 1 -- 172.21.15.60:0/2623871027 >> 172.21.15.60:0/2623871027 conn(0x5623631f2b10 msgr2=0x5623632be980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:48:24.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.546+0000 7f92e360ab80 1 -- 172.21.15.60:0/2623871027 shutdown_connections 2024-05-03T14:48:24.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.546+0000 7f92e360ab80 1 -- 172.21.15.60:0/2623871027 wait complete. 2024-05-03T14:48:24.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.546+0000 7f92e360ab80 1 -- start start 2024-05-03T14:48:24.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.547+0000 7f92e360ab80 1 Processor -- start 2024-05-03T14:48:24.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.547+0000 7f92e360ab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623632c4580 0x5623632cfff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:24.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.547+0000 7f92e360ab80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5623632c99f0 0x5623632cd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:24.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.547+0000 7f92e360ab80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5623632ca310 0x5623632cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:24.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.547+0000 7f92e360ab80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5623632d0d20 con 0x5623632c99f0 2024-05-03T14:48:24.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.547+0000 7f92e360ab80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56236317cad0 con 0x5623632ca310 2024-05-03T14:48:24.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.547+0000 7f92e360ab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5623632d0ea0 con 0x5623632c4580 2024-05-03T14:48:24.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.547+0000 7f92e360ab80 10 client.0 _ll_register_callbacks cb 0x5623631bdc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T14:48:24.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.547+0000 7f92c9400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5623632c99f0 0x5623632cd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:24.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.548+0000 7f92d2800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5623632ca310 0x5623632cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:24.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.548+0000 7f92d1e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623632c4580 0x5623632cfff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:24.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.548+0000 7f92c9400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5623632c99f0 0x5623632cd3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36436/0 (socket says 172.21.15.60:36436) 2024-05-03T14:48:24.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.548+0000 7f92c9400700 1 -- 172.21.15.60:0/999246646 learned_addr learned my addr 172.21.15.60:0/999246646 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:48:24.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.548+0000 7f92d2800700 1 -- 172.21.15.60:0/999246646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623632c4580 msgr2=0x5623632cfff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:48:24.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.548+0000 7f92d2800700 1 --2- 172.21.15.60:0/999246646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623632c4580 0x5623632cfff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:24.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.548+0000 7f92d2800700 1 -- 172.21.15.60:0/999246646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5623632c99f0 msgr2=0x5623632cd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:48:24.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.548+0000 7f92d2800700 1 --2- 172.21.15.60:0/999246646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5623632c99f0 0x5623632cd3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:24.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.548+0000 7f92d2800700 1 -- 172.21.15.60:0/999246646 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623631dc3d0 con 0x5623632ca310 2024-05-03T14:48:24.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.548+0000 7f92d2800700 1 --2- 172.21.15.60:0/999246646 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5623632ca310 0x5623632cd920 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f92cc0137c0 tx=0x7f92cc004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a303dc2218d72c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:48:24.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.548+0000 7f92b7e00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-03T14:48:24.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.549+0000 7f92b7e00700 1 -- 172.21.15.60:0/999246646 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92cc029030 con 0x5623632ca310 2024-05-03T14:48:24.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.549+0000 7f92b7e00700 1 -- 172.21.15.60:0/999246646 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f92cc013850 con 0x5623632ca310 2024-05-03T14:48:24.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.549+0000 7f92e360ab80 1 -- 172.21.15.60:0/999246646 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92bc00f460 con 0x5623632ca310 2024-05-03T14:48:24.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.549+0000 7f92b7e00700 1 -- 172.21.15.60:0/999246646 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f92cc033050 con 0x5623632ca310 2024-05-03T14:48:24.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.549+0000 7f92e360ab80 10 client.24968 fetch_fsmap learned FSMap version 29 2024-05-03T14:48:24.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.549+0000 7f92e360ab80 1 -- 172.21.15.60:0/999246646 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5623632db520 con 0x5623632ca310 2024-05-03T14:48:24.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.549+0000 7f92b7e00700 1 -- 172.21.15.60:0/999246646 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f92bc00f460 con 0x5623632ca310 2024-05-03T14:48:24.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.550+0000 7f92b7e00700 1 -- 172.21.15.60:0/999246646 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f92cc008a00 con 0x5623632ca310 2024-05-03T14:48:24.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.550+0000 7f92e360ab80 10 client.24968 fetch_fsmap finished waiting for FSMap version 29 2024-05-03T14:48:24.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.550+0000 7f92e360ab80 10 client.24968 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-03T14:48:24.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.550+0000 7f92e360ab80 20 client.24968 populate_metadata read hostname 'smithi060' 2024-05-03T14:48:24.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.550+0000 7f92e360ab80 1 --2- 172.21.15.60:0/999246646 >> [v2:172.21.15.149:6836/2611278690,v1:172.21.15.149:6837/2611278690] conn(0x5623632ddbf0 0x5623632df060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:24.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.550+0000 7f92e360ab80 4 client.24968 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T14:48:24.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.550+0000 7f92e360ab80 1 -- 172.21.15.60:0/999246646 --> [v2:172.21.15.149:6836/2611278690,v1:172.21.15.149:6837/2611278690] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562363291180 con 0x5623632ddbf0 2024-05-03T14:48:24.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.550+0000 7f92d1e00700 1 --2- 172.21.15.60:0/999246646 >> [v2:172.21.15.149:6836/2611278690,v1:172.21.15.149:6837/2611278690] conn(0x5623632ddbf0 0x5623632df060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:24.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.551+0000 7f92d1e00700 1 --2- 172.21.15.60:0/999246646 >> [v2:172.21.15.149:6836/2611278690,v1:172.21.15.149:6837/2611278690] conn(0x5623632ddbf0 0x5623632df060 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=56e53f1c6d913378 server_cookie=ba512f27e69be5de in_seq=0 out_seq=0 2024-05-03T14:48:24.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.551+0000 7f92b7e00700 10 client.24968 ms_handle_connect on v2:172.21.15.149:6836/2611278690 2024-05-03T14:48:24.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.552+0000 7f92b7e00700 1 -- 172.21.15.60:0/999246646 <== mds.3 v2:172.21.15.149:6836/2611278690 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562363291180 con 0x5623632ddbf0 2024-05-03T14:48:24.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.552+0000 7f92b7e00700 10 client.24968 handle_command_reply: tid=0 2024-05-03T14:48:24.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.552+0000 7f92e360ab80 1 client.24968 shutdown 2024-05-03T14:48:24.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.552+0000 7f92e360ab80 1 -- 172.21.15.60:0/999246646 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5623632ca310 msgr2=0x5623632cd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:24.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.552+0000 7f92e360ab80 1 --2- 172.21.15.60:0/999246646 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5623632ca310 0x5623632cd920 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f92cc0137c0 tx=0x7f92cc004b40 comp rx=0 tx=0).stop 2024-05-03T14:48:24.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.552+0000 7f92e360ab80 1 -- 172.21.15.60:0/999246646 shutdown_connections 2024-05-03T14:48:24.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.552+0000 7f92e360ab80 1 --2- 172.21.15.60:0/999246646 >> [v2:172.21.15.149:6836/2611278690,v1:172.21.15.149:6837/2611278690] conn(0x5623632ddbf0 0x5623632df060 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:24.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.552+0000 7f92e360ab80 1 --2- 172.21.15.60:0/999246646 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5623632ca310 0x5623632cd920 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:24.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.552+0000 7f92e360ab80 1 --2- 172.21.15.60:0/999246646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623632c4580 0x5623632cfff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:24.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.552+0000 7f92e360ab80 1 --2- 172.21.15.60:0/999246646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5623632c99f0 0x5623632cd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:24.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.552+0000 7f92b7e00700 0 client.24968 ms_handle_reset on v2:172.21.15.149:6836/2611278690 2024-05-03T14:48:24.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.552+0000 7f92e360ab80 1 -- 172.21.15.60:0/999246646 >> 172.21.15.60:0/999246646 conn(0x5623631f2b10 msgr2=0x5623632be030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:48:24.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.553+0000 7f92e360ab80 1 -- 172.21.15.60:0/999246646 shutdown_connections 2024-05-03T14:48:24.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.553+0000 7f92e360ab80 1 -- 172.21.15.60:0/999246646 wait complete. 2024-05-03T14:48:24.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.553+0000 7f92e360ab80 20 client.24968 trim_cache size 0 max 16384 2024-05-03T14:48:24.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.577+0000 7f92e360ab80 1 Processor -- start 2024-05-03T14:48:24.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.577+0000 7f92e360ab80 1 -- start start 2024-05-03T14:48:24.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.577+0000 7f92e360ab80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5623632c4580 0x5623632c3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:24.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.577+0000 7f92e360ab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623632c99f0 0x5623632c7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:24.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.577+0000 7f92e360ab80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5623632ca310 0x5623632c79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:24.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.577+0000 7f92e360ab80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5623632dd3a0 con 0x5623632c4580 2024-05-03T14:48:24.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.577+0000 7f92e360ab80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5623632dda70 con 0x5623632ca310 2024-05-03T14:48:24.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.577+0000 7f92e360ab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5623632de140 con 0x5623632c99f0 2024-05-03T14:48:24.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.577+0000 7f92d1e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5623632c4580 0x5623632c3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:24.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.578+0000 7f92c9400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623632c99f0 0x5623632c7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:24.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.578+0000 7f92d2800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5623632ca310 0x5623632c79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:24.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.578+0000 7f92d2800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5623632ca310 0x5623632c79d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.60:53942/0 (socket says 172.21.15.60:53942) 2024-05-03T14:48:24.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.578+0000 7f92c9400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623632c99f0 0x5623632c7490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.60:48394/0 (socket says 172.21.15.60:48394) 2024-05-03T14:48:24.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.578+0000 7f92d2800700 1 -- 172.21.15.60:0/3369523130 learned_addr learned my addr 172.21.15.60:0/3369523130 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:48:24.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.578+0000 7f92d1e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5623632c4580 0x5623632c3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36450/0 (socket says 172.21.15.60:36450) 2024-05-03T14:48:24.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.578+0000 7f92d2800700 1 -- 172.21.15.60:0/3369523130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623632c99f0 msgr2=0x5623632c7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:48:24.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.578+0000 7f92d2800700 1 --2- 172.21.15.60:0/3369523130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623632c99f0 0x5623632c7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:24.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.578+0000 7f92d2800700 1 -- 172.21.15.60:0/3369523130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5623632c4580 msgr2=0x5623632c3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:48:24.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.578+0000 7f92d2800700 1 --2- 172.21.15.60:0/3369523130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5623632c4580 0x5623632c3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:24.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.578+0000 7f92d2800700 1 -- 172.21.15.60:0/3369523130 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623632bf5e0 con 0x5623632ca310 2024-05-03T14:48:24.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.578+0000 7f92d2800700 1 --2- 172.21.15.60:0/3369523130 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5623632ca310 0x5623632c79d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f92cc009960 tx=0x7f92cc004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd1fafac6211e324 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:48:24.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.579+0000 7f92c2a00700 1 -- 172.21.15.60:0/3369523130 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92cc029070 con 0x5623632ca310 2024-05-03T14:48:24.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.579+0000 7f92c2a00700 1 -- 172.21.15.60:0/3369523130 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f92cc03c070 con 0x5623632ca310 2024-05-03T14:48:24.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.579+0000 7f92c2a00700 1 -- 172.21.15.60:0/3369523130 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92cc021070 con 0x5623632ca310 2024-05-03T14:48:24.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.580+0000 7f92e360ab80 1 -- 172.21.15.60:0/3369523130 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5623632ca310 msgr2=0x5623632c79d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:24.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.580+0000 7f92e360ab80 1 --2- 172.21.15.60:0/3369523130 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5623632ca310 0x5623632c79d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f92cc009960 tx=0x7f92cc004ab0 comp rx=0 tx=0).stop 2024-05-03T14:48:24.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.580+0000 7f92e360ab80 1 -- 172.21.15.60:0/3369523130 shutdown_connections 2024-05-03T14:48:24.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.580+0000 7f92e360ab80 1 --2- 172.21.15.60:0/3369523130 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5623632ca310 0x5623632c79d0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:24.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.580+0000 7f92e360ab80 1 --2- 172.21.15.60:0/3369523130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5623632c4580 0x5623632c3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:24.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.580+0000 7f92e360ab80 1 --2- 172.21.15.60:0/3369523130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623632c99f0 0x5623632c7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:24.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.580+0000 7f92e360ab80 1 -- 172.21.15.60:0/3369523130 >> 172.21.15.60:0/3369523130 conn(0x5623631f2b10 msgr2=0x5623632cd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:48:24.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.580+0000 7f92e360ab80 1 -- 172.21.15.60:0/3369523130 shutdown_connections 2024-05-03T14:48:24.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.580+0000 7f92e360ab80 1 -- 172.21.15.60:0/3369523130 wait complete. 2024-05-03T14:48:24.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.580+0000 7f92e360ab80 1 -- start start 2024-05-03T14:48:24.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.580+0000 7f92e360ab80 1 Processor -- start 2024-05-03T14:48:24.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.581+0000 7f92e360ab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623632c4580 0x5623632e84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:24.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.581+0000 7f92e360ab80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5623632c99f0 0x5623632e8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:24.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.581+0000 7f92e360ab80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5623632ca310 0x5623632e3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:24.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.581+0000 7f92e360ab80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5623632deee0 con 0x5623632ca310 2024-05-03T14:48:24.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.581+0000 7f92e360ab80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5623632de810 con 0x5623632c99f0 2024-05-03T14:48:24.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.581+0000 7f92e360ab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5623632df5b0 con 0x5623632c4580 2024-05-03T14:48:24.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.581+0000 7f92e360ab80 10 client.0 _ll_register_callbacks cb 0x5623631bdc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T14:48:24.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.581+0000 7f92d2800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5623632ca310 0x5623632e3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:24.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.581+0000 7f92d2800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5623632ca310 0x5623632e3490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36462/0 (socket says 172.21.15.60:36462) 2024-05-03T14:48:24.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.581+0000 7f92d2800700 1 -- 172.21.15.60:0/568751483 learned_addr learned my addr 172.21.15.60:0/568751483 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:48:24.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.581+0000 7f92c9400700 1 --2- 172.21.15.60:0/568751483 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5623632c99f0 0x5623632e8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:24.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.581+0000 7f92d1e00700 1 --2- 172.21.15.60:0/568751483 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623632c4580 0x5623632e84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:24.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.582+0000 7f92d2800700 1 -- 172.21.15.60:0/568751483 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623632c4580 msgr2=0x5623632e84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:48:24.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.582+0000 7f92d2800700 1 --2- 172.21.15.60:0/568751483 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623632c4580 0x5623632e84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:24.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.582+0000 7f92d2800700 1 -- 172.21.15.60:0/568751483 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5623632c99f0 msgr2=0x5623632e8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:48:24.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.582+0000 7f92d2800700 1 --2- 172.21.15.60:0/568751483 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5623632c99f0 0x5623632e8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:24.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.582+0000 7f92d2800700 1 -- 172.21.15.60:0/568751483 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623632e3bb0 con 0x5623632ca310 2024-05-03T14:48:24.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.582+0000 7f92d1e00700 1 --2- 172.21.15.60:0/568751483 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623632c4580 0x5623632e84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:48:24.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.582+0000 7f92c9400700 1 --2- 172.21.15.60:0/568751483 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5623632c99f0 0x5623632e8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:48:24.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.582+0000 7f92d2800700 1 --2- 172.21.15.60:0/568751483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5623632ca310 0x5623632e3490 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f92cc009fd0 tx=0x7f92cc01b680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89f7539c714999cd server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:48:24.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.582+0000 7f92b7e00700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-05-03T14:48:24.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.583+0000 7f92b7e00700 1 -- 172.21.15.60:0/568751483 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92cc045060 con 0x5623632ca310 2024-05-03T14:48:24.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.583+0000 7f92b7e00700 1 -- 172.21.15.60:0/568751483 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f92cc005e00 con 0x5623632ca310 2024-05-03T14:48:24.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.583+0000 7f92e360ab80 1 -- 172.21.15.60:0/568751483 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92cc03c070 con 0x5623632ca310 2024-05-03T14:48:24.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.583+0000 7f92b7e00700 1 -- 172.21.15.60:0/568751483 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f92cc04c050 con 0x5623632ca310 2024-05-03T14:48:24.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.583+0000 7f92e360ab80 10 client.14874 fetch_fsmap learned FSMap version 29 2024-05-03T14:48:24.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.583+0000 7f92e360ab80 1 -- 172.21.15.60:0/568751483 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562363349a80 con 0x5623632ca310 2024-05-03T14:48:24.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.583+0000 7f92b7e00700 1 -- 172.21.15.60:0/568751483 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f92cc03c070 con 0x5623632ca310 2024-05-03T14:48:24.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.584+0000 7f92b7e00700 1 -- 172.21.15.60:0/568751483 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f92cc01b870 con 0x5623632ca310 2024-05-03T14:48:24.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.584+0000 7f92e360ab80 10 client.14874 fetch_fsmap finished waiting for FSMap version 29 2024-05-03T14:48:24.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.584+0000 7f92e360ab80 10 client.14874 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-03T14:48:24.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.584+0000 7f92e360ab80 20 client.14874 populate_metadata read hostname 'smithi060' 2024-05-03T14:48:24.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.584+0000 7f92e360ab80 1 --2- 172.21.15.60:0/568751483 >> [v2:172.21.15.149:6836/2611278690,v1:172.21.15.149:6837/2611278690] conn(0x56236334e2f0 0x56236334d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:24.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.584+0000 7f92e360ab80 4 client.14874 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-03T14:48:24.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.584+0000 7f92e360ab80 1 -- 172.21.15.60:0/568751483 --> [v2:172.21.15.149:6836/2611278690,v1:172.21.15.149:6837/2611278690] -- command(tid 0: {"prefix": "status"}) -- 0x5623632243b0 con 0x56236334e2f0 2024-05-03T14:48:24.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.584+0000 7f92d1e00700 1 --2- 172.21.15.60:0/568751483 >> [v2:172.21.15.149:6836/2611278690,v1:172.21.15.149:6837/2611278690] conn(0x56236334e2f0 0x56236334d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:24.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.585+0000 7f92d1e00700 1 --2- 172.21.15.60:0/568751483 >> [v2:172.21.15.149:6836/2611278690,v1:172.21.15.149:6837/2611278690] conn(0x56236334e2f0 0x56236334d000 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9a09af3f7b7c9067 server_cookie=7fbecb548f3510da in_seq=0 out_seq=0 2024-05-03T14:48:24.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.585+0000 7f92b7e00700 10 client.14874 ms_handle_connect on v2:172.21.15.149:6836/2611278690 2024-05-03T14:48:24.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.585+0000 7f92b7e00700 1 -- 172.21.15.60:0/568751483 <== mds.3 v2:172.21.15.149:6836/2611278690 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5623632243b0 con 0x56236334e2f0 2024-05-03T14:48:24.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.585+0000 7f92b7e00700 10 client.14874 handle_command_reply: tid=0 2024-05-03T14:48:24.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.585+0000 7f92e360ab80 1 client.14874 shutdown 2024-05-03T14:48:24.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.585+0000 7f92e360ab80 1 -- 172.21.15.60:0/568751483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5623632ca310 msgr2=0x5623632e3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:24.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.585+0000 7f92e360ab80 1 --2- 172.21.15.60:0/568751483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5623632ca310 0x5623632e3490 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f92cc009fd0 tx=0x7f92cc01b680 comp rx=0 tx=0).stop 2024-05-03T14:48:24.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.585+0000 7f92e360ab80 1 -- 172.21.15.60:0/568751483 shutdown_connections 2024-05-03T14:48:24.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.585+0000 7f92e360ab80 1 --2- 172.21.15.60:0/568751483 >> [v2:172.21.15.149:6836/2611278690,v1:172.21.15.149:6837/2611278690] conn(0x56236334e2f0 0x56236334d000 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:24.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.585+0000 7f92e360ab80 1 --2- 172.21.15.60:0/568751483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5623632ca310 0x5623632e3490 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:24.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.585+0000 7f92e360ab80 1 --2- 172.21.15.60:0/568751483 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623632c4580 0x5623632e84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:24.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.585+0000 7f92e360ab80 1 --2- 172.21.15.60:0/568751483 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5623632c99f0 0x5623632e8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:24.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.585+0000 7f92b7e00700 0 client.14874 ms_handle_reset on v2:172.21.15.149:6836/2611278690 2024-05-03T14:48:24.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.585+0000 7f92e360ab80 1 -- 172.21.15.60:0/568751483 >> 172.21.15.60:0/568751483 conn(0x5623631f2b10 msgr2=0x5623632cd360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:48:24.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.586+0000 7f92e360ab80 1 -- 172.21.15.60:0/568751483 shutdown_connections 2024-05-03T14:48:24.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.586+0000 7f92e360ab80 1 -- 172.21.15.60:0/568751483 wait complete. 2024-05-03T14:48:24.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.586+0000 7f92e360ab80 20 client.14874 trim_cache size 0 max 16384 2024-05-03T14:48:24.592 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-05-03T14:48:24.592 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "285d64b2-095a-11ef-bc97-c7b262605968", 2024-05-03T14:48:24.593 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 3, 2024-05-03T14:48:24.593 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 24475, 2024-05-03T14:48:24.593 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-05-03T14:48:24.593 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-05-03T14:48:24.593 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-05-03T14:48:24.593 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 149.075297178, 2024-05-03T14:48:24.593 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 29, 2024-05-03T14:48:24.593 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 89, 2024-05-03T14:48:24.593 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 87, 2024-05-03T14:48:24.593 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 270.36416037499998 2024-05-03T14:48:24.593 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-05-03T14:48:24.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.587+0000 7f92d3200700 1 -- 172.21.15.60:0/961508470 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f92b80b2910 msgr2=0x7f92b80b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:24.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.587+0000 7f92d3200700 1 --2- 172.21.15.60:0/961508470 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f92b80b2910 0x7f92b80b4dd0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f92cc009da0 tx=0x7f92cc013560 comp rx=0 tx=0).stop 2024-05-03T14:48:24.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.587+0000 7f92d3200700 1 -- 172.21.15.60:0/961508470 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92d41033d0 msgr2=0x7f92d40757a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:24.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.587+0000 7f92d3200700 1 --2- 172.21.15.60:0/961508470 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92d41033d0 0x7f92d40757a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f92c400e9f0 tx=0x7f92c400ed00 comp rx=0 tx=0).stop 2024-05-03T14:48:24.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.587+0000 7f92d3200700 1 -- 172.21.15.60:0/961508470 shutdown_connections 2024-05-03T14:48:24.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.587+0000 7f92d3200700 1 --2- 172.21.15.60:0/961508470 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f92b80b2910 0x7f92b80b4dd0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:24.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.587+0000 7f92d3200700 1 --2- 172.21.15.60:0/961508470 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92d4109460 0x7f92d4076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:24.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.587+0000 7f92d3200700 1 --2- 172.21.15.60:0/961508470 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92d41033d0 0x7f92d40757a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:24.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.587+0000 7f92d3200700 1 --2- 172.21.15.60:0/961508470 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f92d4103d90 0x7f92d4075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:24.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.587+0000 7f92d3200700 1 -- 172.21.15.60:0/961508470 >> 172.21.15.60:0/961508470 conn(0x7f92d40fef20 msgr2=0x7f92d40ff660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:48:24.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.587+0000 7f92d3200700 1 -- 172.21.15.60:0/961508470 shutdown_connections 2024-05-03T14:48:24.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:24.587+0000 7f92d3200700 1 -- 172.21.15.60:0/961508470 wait complete. 2024-05-03T14:48:25.634 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph tell mds.i status 2024-05-03T14:48:25.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:48:25 smithi060 ceph-mon[28873]: pgmap v449: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-03T14:48:26.104 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:48:26.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:48:25 smithi090 ceph-mon[33568]: pgmap v449: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-03T14:48:26.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:48:25 smithi149 ceph-mon[33332]: pgmap v449: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-03T14:48:27.806 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:48:27 smithi060 ceph-mon[28873]: pgmap v450: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-03T14:48:28.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:48:27 smithi090 ceph-mon[33568]: pgmap v450: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-03T14:48:28.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:48:27 smithi149 ceph-mon[33332]: pgmap v450: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-03T14:48:28.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.325+0000 7fe260200700 1 -- 172.21.15.60:0/2366061081 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe258114670 msgr2=0x7fe258116a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:28.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.325+0000 7fe260200700 1 --2- 172.21.15.60:0/2366061081 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe258114670 0x7fe258116a60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe25000b0a0 tx=0x7fe25000b3b0 comp rx=0 tx=0).stop 2024-05-03T14:48:28.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.325+0000 7fe260200700 1 -- 172.21.15.60:0/2366061081 shutdown_connections 2024-05-03T14:48:28.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.325+0000 7fe260200700 1 --2- 172.21.15.60:0/2366061081 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe258114670 0x7fe258116a60 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:28.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.325+0000 7fe260200700 1 --2- 172.21.15.60:0/2366061081 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2580756d0 0x7fe258075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:28.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.325+0000 7fe260200700 1 --2- 172.21.15.60:0/2366061081 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe258075ff0 0x7fe258114130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:28.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.325+0000 7fe260200700 1 -- 172.21.15.60:0/2366061081 >> 172.21.15.60:0/2366061081 conn(0x7fe25807a600 msgr2=0x7fe25807aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:48:28.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.325+0000 7fe260200700 1 -- 172.21.15.60:0/2366061081 shutdown_connections 2024-05-03T14:48:28.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.325+0000 7fe260200700 1 -- 172.21.15.60:0/2366061081 wait complete. 2024-05-03T14:48:28.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.326+0000 7fe260200700 1 Processor -- start 2024-05-03T14:48:28.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.326+0000 7fe260200700 1 -- start start 2024-05-03T14:48:28.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.327+0000 7fe260200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2580756d0 0x7fe2581a02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:28.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.327+0000 7fe260200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe258075ff0 0x7fe2581a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:28.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.327+0000 7fe260200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe258114670 0x7fe2581a4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:28.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.327+0000 7fe260200700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe258119ed0 con 0x7fe258114670 2024-05-03T14:48:28.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.327+0000 7fe260200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe258119d50 con 0x7fe2580756d0 2024-05-03T14:48:28.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.327+0000 7fe260200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe25811a050 con 0x7fe258075ff0 2024-05-03T14:48:28.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.327+0000 7fe25ce00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe258075ff0 0x7fe2581a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:28.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.327+0000 7fe25d800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2580756d0 0x7fe2581a02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:28.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.327+0000 7fe25ce00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe258075ff0 0x7fe2581a0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.60:48428/0 (socket says 172.21.15.60:48428) 2024-05-03T14:48:28.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.327+0000 7fe25ce00700 1 -- 172.21.15.60:0/2307512062 learned_addr learned my addr 172.21.15.60:0/2307512062 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:48:28.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.327+0000 7fe25e200700 1 --2- 172.21.15.60:0/2307512062 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe258114670 0x7fe2581a4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:28.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.328+0000 7fe25ce00700 1 -- 172.21.15.60:0/2307512062 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2580756d0 msgr2=0x7fe2581a02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:28.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.328+0000 7fe25ce00700 1 --2- 172.21.15.60:0/2307512062 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2580756d0 0x7fe2581a02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:28.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.328+0000 7fe25ce00700 1 -- 172.21.15.60:0/2307512062 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe258114670 msgr2=0x7fe2581a4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:28.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.328+0000 7fe25ce00700 1 --2- 172.21.15.60:0/2307512062 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe258114670 0x7fe2581a4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:28.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.328+0000 7fe25ce00700 1 -- 172.21.15.60:0/2307512062 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2581a52f0 con 0x7fe258075ff0 2024-05-03T14:48:28.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.328+0000 7fe25ce00700 1 --2- 172.21.15.60:0/2307512062 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe258075ff0 0x7fe2581a0820 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe24400e9a0 tx=0x7fe24400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:48:28.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.328+0000 7fe256000700 1 -- 172.21.15.60:0/2307512062 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24400ca10 con 0x7fe258075ff0 2024-05-03T14:48:28.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.328+0000 7fe256000700 1 -- 172.21.15.60:0/2307512062 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7fe24400cbb0 con 0x7fe258075ff0 2024-05-03T14:48:28.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.328+0000 7fe260200700 1 -- 172.21.15.60:0/2307512062 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2581a55e0 con 0x7fe258075ff0 2024-05-03T14:48:28.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.328+0000 7fe256000700 1 -- 172.21.15.60:0/2307512062 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2440187a0 con 0x7fe258075ff0 2024-05-03T14:48:28.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.328+0000 7fe260200700 1 -- 172.21.15.60:0/2307512062 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2581ace60 con 0x7fe258075ff0 2024-05-03T14:48:28.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.332+0000 7fe256000700 1 -- 172.21.15.60:0/2307512062 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fe244010430 con 0x7fe258075ff0 2024-05-03T14:48:28.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.333+0000 7fe26ac1eb80 1 Processor -- start 2024-05-03T14:48:28.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.333+0000 7fe26ac1eb80 1 -- start start 2024-05-03T14:48:28.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.334+0000 7fe256000700 1 --2- 172.21.15.60:0/2307512062 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fe2400b2910 0x7fe2400b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:28.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.334+0000 7fe26ac1eb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560c125db8e0 0x560c125dbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:28.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.334+0000 7fe26ac1eb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c125dc200 0x560c125d5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:28.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.334+0000 7fe26ac1eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560c125d6500 0x560c125d6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:28.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.334+0000 7fe26ac1eb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x560c125289d0 con 0x560c125dc200 2024-05-03T14:48:28.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.334+0000 7fe26ac1eb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x560c124c6710 con 0x560c125db8e0 2024-05-03T14:48:28.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.334+0000 7fe26ac1eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x560c12524990 con 0x560c125d6500 2024-05-03T14:48:28.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.334+0000 7fe256000700 1 -- 172.21.15.60:0/2307512062 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7fe2440e2870 con 0x7fe258075ff0 2024-05-03T14:48:28.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.334+0000 7fe25ce00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c125dc200 0x560c125d5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:28.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.334+0000 7fe25ce00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c125dc200 0x560c125d5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36484/0 (socket says 172.21.15.60:36484) 2024-05-03T14:48:28.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.334+0000 7fe25ce00700 1 -- 172.21.15.60:0/509614735 learned_addr learned my addr 172.21.15.60:0/509614735 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:48:28.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.334+0000 7fe25d800700 1 --2- 172.21.15.60:0/509614735 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560c125db8e0 0x560c125dbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:28.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.334+0000 7fe25e200700 1 --2- 172.21.15.60:0/2307512062 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fe2400b2910 0x7fe2400b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:28.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.335+0000 7fe25ce00700 1 -- 172.21.15.60:0/509614735 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560c125d6500 msgr2=0x560c125d6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:48:28.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.335+0000 7fe25ce00700 1 --2- 172.21.15.60:0/509614735 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560c125d6500 0x560c125d6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:28.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.335+0000 7fe25ce00700 1 -- 172.21.15.60:0/509614735 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560c125db8e0 msgr2=0x560c125dbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:48:28.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.335+0000 7fe25ce00700 1 --2- 172.21.15.60:0/509614735 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560c125db8e0 0x560c125dbcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:28.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.335+0000 7fe25ce00700 1 -- 172.21.15.60:0/509614735 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c12475880 con 0x560c125dc200 2024-05-03T14:48:28.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.335+0000 7fe25ce00700 1 --2- 172.21.15.60:0/509614735 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c125dc200 0x560c125d5ee0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe244000c00 tx=0x7fe2440d9240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34125408ee4d9e50 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:48:28.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.335+0000 7fe24aa00700 1 -- 172.21.15.60:0/509614735 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2440a50e0 con 0x560c125dc200 2024-05-03T14:48:28.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.335+0000 7fe24aa00700 1 -- 172.21.15.60:0/509614735 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7fe2440a9460 con 0x560c125dc200 2024-05-03T14:48:28.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.335+0000 7fe24aa00700 1 -- 172.21.15.60:0/509614735 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2440109c0 con 0x560c125dc200 2024-05-03T14:48:28.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.335+0000 7fe25e200700 1 --2- 172.21.15.60:0/2307512062 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fe2400b2910 0x7fe2400b4dd0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x560c125d6ec0 tx=0x7fe250009b80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:48:28.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.336+0000 7fe26ac1eb80 1 -- 172.21.15.60:0/509614735 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c125dc200 msgr2=0x560c125d5ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:28.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.336+0000 7fe26ac1eb80 1 --2- 172.21.15.60:0/509614735 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c125dc200 0x560c125d5ee0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe244000c00 tx=0x7fe2440d9240 comp rx=0 tx=0).stop 2024-05-03T14:48:28.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.336+0000 7fe26ac1eb80 1 -- 172.21.15.60:0/509614735 shutdown_connections 2024-05-03T14:48:28.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.336+0000 7fe26ac1eb80 1 --2- 172.21.15.60:0/509614735 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560c125d6500 0x560c125d6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:28.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.336+0000 7fe26ac1eb80 1 --2- 172.21.15.60:0/509614735 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560c125db8e0 0x560c125dbcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:28.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.336+0000 7fe26ac1eb80 1 --2- 172.21.15.60:0/509614735 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c125dc200 0x560c125d5ee0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:28.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.336+0000 7fe26ac1eb80 1 -- 172.21.15.60:0/509614735 >> 172.21.15.60:0/509614735 conn(0x560c1254e680 msgr2=0x560c125d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:48:28.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.336+0000 7fe26ac1eb80 1 -- 172.21.15.60:0/509614735 shutdown_connections 2024-05-03T14:48:28.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.337+0000 7fe26ac1eb80 1 -- 172.21.15.60:0/509614735 wait complete. 2024-05-03T14:48:28.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.337+0000 7fe26ac1eb80 1 -- start start 2024-05-03T14:48:28.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.337+0000 7fe26ac1eb80 1 Processor -- start 2024-05-03T14:48:28.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.338+0000 7fe26ac1eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560c125d6500 0x560c125e1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:28.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.338+0000 7fe26ac1eb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560c125db8e0 0x560c125df1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:28.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.338+0000 7fe26ac1eb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c125dc200 0x560c125df6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:28.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.338+0000 7fe26ac1eb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x560c125e2bf0 con 0x560c125dc200 2024-05-03T14:48:28.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.338+0000 7fe26ac1eb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x560c1248ead0 con 0x560c125db8e0 2024-05-03T14:48:28.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.338+0000 7fe26ac1eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x560c125e2d70 con 0x560c125d6500 2024-05-03T14:48:28.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.338+0000 7fe26ac1eb80 10 client.0 _ll_register_callbacks cb 0x560c124cfc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T14:48:28.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.338+0000 7fe25d800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560c125d6500 0x560c125e1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:28.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.338+0000 7fe25e200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c125dc200 0x560c125df6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:28.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.338+0000 7fe25d800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560c125d6500 0x560c125e1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.60:48442/0 (socket says 172.21.15.60:48442) 2024-05-03T14:48:28.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.338+0000 7fe25d800700 1 -- 172.21.15.60:0/600020148 learned_addr learned my addr 172.21.15.60:0/600020148 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:48:28.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.338+0000 7fe25ce00700 1 --2- 172.21.15.60:0/600020148 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560c125db8e0 0x560c125df1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:28.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.339+0000 7fe25d800700 1 -- 172.21.15.60:0/600020148 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560c125db8e0 msgr2=0x560c125df1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:48:28.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.339+0000 7fe25d800700 1 --2- 172.21.15.60:0/600020148 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560c125db8e0 0x560c125df1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:28.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.339+0000 7fe25d800700 1 -- 172.21.15.60:0/600020148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c125dc200 msgr2=0x560c125df6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:48:28.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.339+0000 7fe25d800700 1 --2- 172.21.15.60:0/600020148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c125dc200 0x560c125df6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:28.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.339+0000 7fe25d800700 1 -- 172.21.15.60:0/600020148 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c124ee3d0 con 0x560c125d6500 2024-05-03T14:48:28.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.339+0000 7fe25ce00700 1 --2- 172.21.15.60:0/600020148 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560c125db8e0 0x560c125df1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:48:28.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.339+0000 7fe25e200700 1 --2- 172.21.15.60:0/600020148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c125dc200 0x560c125df6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:48:28.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.339+0000 7fe25d800700 1 --2- 172.21.15.60:0/600020148 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560c125d6500 0x560c125e1e10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe24c012760 tx=0x7fe24c012a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=910921c898efbdc5 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:48:28.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.339+0000 7fe23f400700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-05-03T14:48:28.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.339+0000 7fe23f400700 1 -- 172.21.15.60:0/600020148 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24c00f480 con 0x560c125d6500 2024-05-03T14:48:28.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.340+0000 7fe26ac1eb80 1 -- 172.21.15.60:0/600020148 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2440a9460 con 0x560c125d6500 2024-05-03T14:48:28.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.340+0000 7fe23f400700 1 -- 172.21.15.60:0/600020148 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7fe24c00f620 con 0x560c125d6500 2024-05-03T14:48:28.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.340+0000 7fe23f400700 1 -- 172.21.15.60:0/600020148 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe24c00f820 con 0x560c125d6500 2024-05-03T14:48:28.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.340+0000 7fe26ac1eb80 10 client.24994 fetch_fsmap learned FSMap version 29 2024-05-03T14:48:28.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.340+0000 7fe26ac1eb80 1 -- 172.21.15.60:0/600020148 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560c125ed3c0 con 0x560c125d6500 2024-05-03T14:48:28.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.340+0000 7fe23f400700 1 -- 172.21.15.60:0/600020148 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7fe2440a9460 con 0x560c125d6500 2024-05-03T14:48:28.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.340+0000 7fe23f400700 1 -- 172.21.15.60:0/600020148 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe24c010ba0 con 0x560c125d6500 2024-05-03T14:48:28.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.341+0000 7fe26ac1eb80 10 client.24994 fetch_fsmap finished waiting for FSMap version 29 2024-05-03T14:48:28.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.341+0000 7fe26ac1eb80 10 client.24994 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-03T14:48:28.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.341+0000 7fe26ac1eb80 20 client.24994 populate_metadata read hostname 'smithi060' 2024-05-03T14:48:28.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.341+0000 7fe26ac1eb80 1 --2- 172.21.15.60:0/600020148 >> [v2:172.21.15.149:6838/2527622212,v1:172.21.15.149:6839/2527622212] conn(0x560c125ef9a0 0x560c125f0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:28.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.341+0000 7fe26ac1eb80 4 client.24994 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T14:48:28.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.341+0000 7fe26ac1eb80 1 -- 172.21.15.60:0/600020148 --> [v2:172.21.15.149:6838/2527622212,v1:172.21.15.149:6839/2527622212] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c125a2fa0 con 0x560c125ef9a0 2024-05-03T14:48:28.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.341+0000 7fe25ce00700 1 --2- 172.21.15.60:0/600020148 >> [v2:172.21.15.149:6838/2527622212,v1:172.21.15.149:6839/2527622212] conn(0x560c125ef9a0 0x560c125f0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:28.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.342+0000 7fe25ce00700 1 --2- 172.21.15.60:0/600020148 >> [v2:172.21.15.149:6838/2527622212,v1:172.21.15.149:6839/2527622212] conn(0x560c125ef9a0 0x560c125f0e10 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a584f809e35f4637 server_cookie=b307a21da82cb9db in_seq=0 out_seq=0 2024-05-03T14:48:28.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.342+0000 7fe23f400700 10 client.24994 ms_handle_connect on v2:172.21.15.149:6838/2527622212 2024-05-03T14:48:28.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.342+0000 7fe23f400700 1 -- 172.21.15.60:0/600020148 <== mds.2 v2:172.21.15.149:6838/2527622212 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560c125a2fa0 con 0x560c125ef9a0 2024-05-03T14:48:28.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.342+0000 7fe23f400700 10 client.24994 handle_command_reply: tid=0 2024-05-03T14:48:28.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.343+0000 7fe26ac1eb80 1 client.24994 shutdown 2024-05-03T14:48:28.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.343+0000 7fe26ac1eb80 1 -- 172.21.15.60:0/600020148 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560c125d6500 msgr2=0x560c125e1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:28.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.343+0000 7fe26ac1eb80 1 --2- 172.21.15.60:0/600020148 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560c125d6500 0x560c125e1e10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe24c012760 tx=0x7fe24c012a70 comp rx=0 tx=0).stop 2024-05-03T14:48:28.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.343+0000 7fe26ac1eb80 1 -- 172.21.15.60:0/600020148 shutdown_connections 2024-05-03T14:48:28.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.343+0000 7fe26ac1eb80 1 --2- 172.21.15.60:0/600020148 >> [v2:172.21.15.149:6838/2527622212,v1:172.21.15.149:6839/2527622212] conn(0x560c125ef9a0 0x560c125f0e10 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:28.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.343+0000 7fe26ac1eb80 1 --2- 172.21.15.60:0/600020148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c125dc200 0x560c125df6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:28.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.343+0000 7fe26ac1eb80 1 --2- 172.21.15.60:0/600020148 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560c125d6500 0x560c125e1e10 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:28.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.343+0000 7fe26ac1eb80 1 --2- 172.21.15.60:0/600020148 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560c125db8e0 0x560c125df1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:28.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.343+0000 7fe26ac1eb80 1 -- 172.21.15.60:0/600020148 >> 172.21.15.60:0/600020148 conn(0x560c1254e680 msgr2=0x560c125cb360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:48:28.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.343+0000 7fe23f400700 0 client.24994 ms_handle_reset on v2:172.21.15.149:6838/2527622212 2024-05-03T14:48:28.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.343+0000 7fe26ac1eb80 1 -- 172.21.15.60:0/600020148 shutdown_connections 2024-05-03T14:48:28.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.343+0000 7fe26ac1eb80 1 -- 172.21.15.60:0/600020148 wait complete. 2024-05-03T14:48:28.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.343+0000 7fe26ac1eb80 20 client.24994 trim_cache size 0 max 16384 2024-05-03T14:48:28.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.366+0000 7fe26ac1eb80 1 Processor -- start 2024-05-03T14:48:28.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.366+0000 7fe26ac1eb80 1 -- start start 2024-05-03T14:48:28.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.367+0000 7fe26ac1eb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c125d6500 0x560c125d93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:28.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.367+0000 7fe26ac1eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560c125db8e0 0x560c125d9930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:28.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.367+0000 7fe26ac1eb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560c125dc200 0x560c125df170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:28.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.367+0000 7fe26ac1eb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x560c125ef150 con 0x560c125d6500 2024-05-03T14:48:28.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.367+0000 7fe26ac1eb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x560c125ef820 con 0x560c125dc200 2024-05-03T14:48:28.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.367+0000 7fe26ac1eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x560c125efef0 con 0x560c125db8e0 2024-05-03T14:48:28.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.367+0000 7fe25d800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c125d6500 0x560c125d93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:28.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.367+0000 7fe25d800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c125d6500 0x560c125d93f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36510/0 (socket says 172.21.15.60:36510) 2024-05-03T14:48:28.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.367+0000 7fe25d800700 1 -- 172.21.15.60:0/3215741241 learned_addr learned my addr 172.21.15.60:0/3215741241 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:48:28.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.367+0000 7fe25e200700 1 --2- 172.21.15.60:0/3215741241 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560c125dc200 0x560c125df170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:28.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.367+0000 7fe25ce00700 1 --2- 172.21.15.60:0/3215741241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560c125db8e0 0x560c125d9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:28.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.367+0000 7fe25d800700 1 -- 172.21.15.60:0/3215741241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560c125db8e0 msgr2=0x560c125d9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:48:28.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.367+0000 7fe25d800700 1 --2- 172.21.15.60:0/3215741241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560c125db8e0 0x560c125d9930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:28.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.368+0000 7fe25d800700 1 -- 172.21.15.60:0/3215741241 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560c125dc200 msgr2=0x560c125df170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:48:28.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.368+0000 7fe25d800700 1 --2- 172.21.15.60:0/3215741241 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560c125dc200 0x560c125df170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:28.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.368+0000 7fe25d800700 1 -- 172.21.15.60:0/3215741241 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c125ef640 con 0x560c125d6500 2024-05-03T14:48:28.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.368+0000 7fe25ce00700 1 --2- 172.21.15.60:0/3215741241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560c125db8e0 0x560c125d9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T14:48:28.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.368+0000 7fe25e200700 1 --2- 172.21.15.60:0/3215741241 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560c125dc200 0x560c125df170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:48:28.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.368+0000 7fe25d800700 1 --2- 172.21.15.60:0/3215741241 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c125d6500 0x560c125d93f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe24c01c510 tx=0x7fe24c012470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9073e3140eb53f7f server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:48:28.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.368+0000 7fe24a000700 1 -- 172.21.15.60:0/3215741241 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24c00f480 con 0x560c125d6500 2024-05-03T14:48:28.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.368+0000 7fe24a000700 1 -- 172.21.15.60:0/3215741241 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7fe24c00f620 con 0x560c125d6500 2024-05-03T14:48:28.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.368+0000 7fe24a000700 1 -- 172.21.15.60:0/3215741241 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24c005320 con 0x560c125d6500 2024-05-03T14:48:28.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.369+0000 7fe26ac1eb80 1 -- 172.21.15.60:0/3215741241 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c125d6500 msgr2=0x560c125d93f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:28.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.369+0000 7fe26ac1eb80 1 --2- 172.21.15.60:0/3215741241 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c125d6500 0x560c125d93f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe24c01c510 tx=0x7fe24c012470 comp rx=0 tx=0).stop 2024-05-03T14:48:28.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.369+0000 7fe26ac1eb80 1 -- 172.21.15.60:0/3215741241 shutdown_connections 2024-05-03T14:48:28.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.369+0000 7fe26ac1eb80 1 --2- 172.21.15.60:0/3215741241 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560c125dc200 0x560c125df170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:28.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.369+0000 7fe26ac1eb80 1 --2- 172.21.15.60:0/3215741241 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c125d6500 0x560c125d93f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:28.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.369+0000 7fe26ac1eb80 1 --2- 172.21.15.60:0/3215741241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560c125db8e0 0x560c125d9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:28.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.369+0000 7fe26ac1eb80 1 -- 172.21.15.60:0/3215741241 >> 172.21.15.60:0/3215741241 conn(0x560c1254e680 msgr2=0x560c125de710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:48:28.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.369+0000 7fe26ac1eb80 1 -- 172.21.15.60:0/3215741241 shutdown_connections 2024-05-03T14:48:28.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.369+0000 7fe26ac1eb80 1 -- 172.21.15.60:0/3215741241 wait complete. 2024-05-03T14:48:28.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.370+0000 7fe26ac1eb80 1 -- start start 2024-05-03T14:48:28.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.370+0000 7fe26ac1eb80 1 Processor -- start 2024-05-03T14:48:28.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.371+0000 7fe26ac1eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560c125d6500 0x560c125d4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:28.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.371+0000 7fe26ac1eb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560c125db8e0 0x560c125d54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:28.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.371+0000 7fe26ac1eb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c125dc200 0x560c125d5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:28.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.371+0000 7fe26ac1eb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x560c125f0c90 con 0x560c125dc200 2024-05-03T14:48:28.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.371+0000 7fe26ac1eb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x560c125f05c0 con 0x560c125db8e0 2024-05-03T14:48:28.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.371+0000 7fe26ac1eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x560c125f1360 con 0x560c125d6500 2024-05-03T14:48:28.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.371+0000 7fe26ac1eb80 10 client.0 _ll_register_callbacks cb 0x560c124cfc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T14:48:28.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.371+0000 7fe25d800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560c125d6500 0x560c125d4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:28.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.371+0000 7fe25e200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c125dc200 0x560c125d5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:28.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.371+0000 7fe25ce00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560c125db8e0 0x560c125d54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:28.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.371+0000 7fe25d800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560c125d6500 0x560c125d4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.60:48462/0 (socket says 172.21.15.60:48462) 2024-05-03T14:48:28.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.371+0000 7fe25d800700 1 -- 172.21.15.60:0/2507361465 learned_addr learned my addr 172.21.15.60:0/2507361465 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:48:28.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.372+0000 7fe25ce00700 1 -- 172.21.15.60:0/2507361465 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560c125d6500 msgr2=0x560c125d4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:48:28.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.372+0000 7fe25ce00700 1 --2- 172.21.15.60:0/2507361465 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560c125d6500 0x560c125d4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:28.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.372+0000 7fe25ce00700 1 -- 172.21.15.60:0/2507361465 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c125dc200 msgr2=0x560c125d5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:48:28.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.372+0000 7fe25ce00700 1 --2- 172.21.15.60:0/2507361465 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c125dc200 0x560c125d5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:28.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.372+0000 7fe25ce00700 1 -- 172.21.15.60:0/2507361465 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c125f8b60 con 0x560c125db8e0 2024-05-03T14:48:28.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.372+0000 7fe25e200700 1 --2- 172.21.15.60:0/2507361465 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c125dc200 0x560c125d5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:48:28.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.372+0000 7fe25d800700 1 --2- 172.21.15.60:0/2507361465 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560c125d6500 0x560c125d4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:48:28.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.372+0000 7fe25ce00700 1 --2- 172.21.15.60:0/2507361465 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560c125db8e0 0x560c125d54f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe244009230 tx=0x7fe2440a9850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84179a5a42d0ff0d server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:48:28.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.372+0000 7fe23f400700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-03T14:48:28.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.372+0000 7fe23f400700 1 -- 172.21.15.60:0/2507361465 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2440a5810 con 0x560c125db8e0 2024-05-03T14:48:28.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.372+0000 7fe23f400700 1 -- 172.21.15.60:0/2507361465 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7fe2440109c0 con 0x560c125db8e0 2024-05-03T14:48:28.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.372+0000 7fe26ac1eb80 1 -- 172.21.15.60:0/2507361465 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe24c00f620 con 0x560c125db8e0 2024-05-03T14:48:28.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.372+0000 7fe23f400700 1 -- 172.21.15.60:0/2507361465 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe2440a9460 con 0x560c125db8e0 2024-05-03T14:48:28.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.372+0000 7fe26ac1eb80 10 client.24992 fetch_fsmap learned FSMap version 29 2024-05-03T14:48:28.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.372+0000 7fe26ac1eb80 1 -- 172.21.15.60:0/2507361465 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560c1265b3d0 con 0x560c125db8e0 2024-05-03T14:48:28.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.373+0000 7fe23f400700 1 -- 172.21.15.60:0/2507361465 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7fe24c00f620 con 0x560c125db8e0 2024-05-03T14:48:28.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.373+0000 7fe23f400700 1 -- 172.21.15.60:0/2507361465 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe2440d3ce0 con 0x560c125db8e0 2024-05-03T14:48:28.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.373+0000 7fe26ac1eb80 10 client.24992 fetch_fsmap finished waiting for FSMap version 29 2024-05-03T14:48:28.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.373+0000 7fe26ac1eb80 10 client.24992 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-03T14:48:28.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.373+0000 7fe26ac1eb80 20 client.24992 populate_metadata read hostname 'smithi060' 2024-05-03T14:48:28.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.373+0000 7fe26ac1eb80 1 --2- 172.21.15.60:0/2507361465 >> [v2:172.21.15.149:6838/2527622212,v1:172.21.15.149:6839/2527622212] conn(0x560c1265faa0 0x560c1265e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:28.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.373+0000 7fe26ac1eb80 4 client.24992 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-03T14:48:28.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.373+0000 7fe26ac1eb80 1 -- 172.21.15.60:0/2507361465 --> [v2:172.21.15.149:6838/2527622212,v1:172.21.15.149:6839/2527622212] -- command(tid 0: {"prefix": "status"}) -- 0x560c125363b0 con 0x560c1265faa0 2024-05-03T14:48:28.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.374+0000 7fe25d800700 1 --2- 172.21.15.60:0/2507361465 >> [v2:172.21.15.149:6838/2527622212,v1:172.21.15.149:6839/2527622212] conn(0x560c1265faa0 0x560c1265e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:28.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.374+0000 7fe25d800700 1 --2- 172.21.15.60:0/2507361465 >> [v2:172.21.15.149:6838/2527622212,v1:172.21.15.149:6839/2527622212] conn(0x560c1265faa0 0x560c1265e7b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d10cfbfa12b52b30 server_cookie=2c915f3b45c16911 in_seq=0 out_seq=0 2024-05-03T14:48:28.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.374+0000 7fe23f400700 10 client.24992 ms_handle_connect on v2:172.21.15.149:6838/2527622212 2024-05-03T14:48:28.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.374+0000 7fe23f400700 1 -- 172.21.15.60:0/2507361465 <== mds.2 v2:172.21.15.149:6838/2527622212 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x560c125363b0 con 0x560c1265faa0 2024-05-03T14:48:28.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.374+0000 7fe23f400700 10 client.24992 handle_command_reply: tid=0 2024-05-03T14:48:28.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.374+0000 7fe26ac1eb80 1 client.24992 shutdown 2024-05-03T14:48:28.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.375+0000 7fe26ac1eb80 1 -- 172.21.15.60:0/2507361465 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560c125db8e0 msgr2=0x560c125d54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:28.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.375+0000 7fe26ac1eb80 1 --2- 172.21.15.60:0/2507361465 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560c125db8e0 0x560c125d54f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe244009230 tx=0x7fe2440a9850 comp rx=0 tx=0).stop 2024-05-03T14:48:28.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.375+0000 7fe26ac1eb80 1 -- 172.21.15.60:0/2507361465 shutdown_connections 2024-05-03T14:48:28.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.375+0000 7fe26ac1eb80 1 --2- 172.21.15.60:0/2507361465 >> [v2:172.21.15.149:6838/2527622212,v1:172.21.15.149:6839/2527622212] conn(0x560c1265faa0 0x560c1265e7b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:28.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.375+0000 7fe26ac1eb80 1 --2- 172.21.15.60:0/2507361465 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c125dc200 0x560c125d5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:28.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.375+0000 7fe26ac1eb80 1 --2- 172.21.15.60:0/2507361465 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560c125d6500 0x560c125d4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:28.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.375+0000 7fe26ac1eb80 1 --2- 172.21.15.60:0/2507361465 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560c125db8e0 0x560c125d54f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:28.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.375+0000 7fe23f400700 0 client.24992 ms_handle_reset on v2:172.21.15.149:6838/2527622212 2024-05-03T14:48:28.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.375+0000 7fe26ac1eb80 1 -- 172.21.15.60:0/2507361465 >> 172.21.15.60:0/2507361465 conn(0x560c1254e680 msgr2=0x560c125d93f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:48:28.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.375+0000 7fe26ac1eb80 1 -- 172.21.15.60:0/2507361465 shutdown_connections 2024-05-03T14:48:28.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.375+0000 7fe26ac1eb80 1 -- 172.21.15.60:0/2507361465 wait complete. 2024-05-03T14:48:28.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.375+0000 7fe26ac1eb80 20 client.24992 trim_cache size 0 max 16384 2024-05-03T14:48:28.382 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-05-03T14:48:28.382 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "285d64b2-095a-11ef-bc97-c7b262605968", 2024-05-03T14:48:28.382 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 2, 2024-05-03T14:48:28.382 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 24487, 2024-05-03T14:48:28.382 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-05-03T14:48:28.382 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-05-03T14:48:28.382 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-05-03T14:48:28.383 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 153.87636950800001, 2024-05-03T14:48:28.383 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 29, 2024-05-03T14:48:28.383 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 89, 2024-05-03T14:48:28.383 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 87, 2024-05-03T14:48:28.383 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 268.49800750399999 2024-05-03T14:48:28.383 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-05-03T14:48:28.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.376+0000 7fe260200700 1 -- 172.21.15.60:0/2307512062 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fe2400b2910 msgr2=0x7fe2400b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:28.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.376+0000 7fe260200700 1 --2- 172.21.15.60:0/2307512062 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fe2400b2910 0x7fe2400b4dd0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x560c125d6ec0 tx=0x7fe250009b80 comp rx=0 tx=0).stop 2024-05-03T14:48:28.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.376+0000 7fe260200700 1 -- 172.21.15.60:0/2307512062 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe258075ff0 msgr2=0x7fe2581a0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:28.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.376+0000 7fe260200700 1 --2- 172.21.15.60:0/2307512062 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe258075ff0 0x7fe2581a0820 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe24400e9a0 tx=0x7fe24400ecb0 comp rx=0 tx=0).stop 2024-05-03T14:48:28.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.376+0000 7fe260200700 1 -- 172.21.15.60:0/2307512062 shutdown_connections 2024-05-03T14:48:28.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.376+0000 7fe260200700 1 --2- 172.21.15.60:0/2307512062 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fe2400b2910 0x7fe2400b4dd0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:28.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.376+0000 7fe260200700 1 --2- 172.21.15.60:0/2307512062 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe258114670 0x7fe2581a4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:28.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.376+0000 7fe260200700 1 --2- 172.21.15.60:0/2307512062 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2580756d0 0x7fe2581a02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:28.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.376+0000 7fe260200700 1 --2- 172.21.15.60:0/2307512062 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe258075ff0 0x7fe2581a0820 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:28.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.376+0000 7fe260200700 1 -- 172.21.15.60:0/2307512062 >> 172.21.15.60:0/2307512062 conn(0x7fe25807a600 msgr2=0x7fe258113360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:48:28.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.376+0000 7fe260200700 1 -- 172.21.15.60:0/2307512062 shutdown_connections 2024-05-03T14:48:28.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:28.376+0000 7fe260200700 1 -- 172.21.15.60:0/2307512062 wait complete. 2024-05-03T14:48:29.228 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph tell mds.b status 2024-05-03T14:48:29.701 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:48:30.008 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:48:29 smithi060 ceph-mon[28873]: pgmap v451: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-05-03T14:48:30.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:48:29 smithi090 ceph-mon[33568]: pgmap v451: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-05-03T14:48:30.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:48:29 smithi149 ceph-mon[33332]: pgmap v451: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-05-03T14:48:31.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.896+0000 7f8ebd600700 1 -- 172.21.15.60:0/539803080 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8eb81023a0 msgr2=0x7f8eb8102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:31.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.896+0000 7f8ebd600700 1 --2- 172.21.15.60:0/539803080 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8eb81023a0 0x7f8eb8102820 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0009990 tx=0x7f8ea0009ca0 comp rx=0 tx=0).stop 2024-05-03T14:48:31.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.897+0000 7f8ebd600700 1 -- 172.21.15.60:0/539803080 shutdown_connections 2024-05-03T14:48:31.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.897+0000 7f8ebd600700 1 --2- 172.21.15.60:0/539803080 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8eb8102d60 0x7f8eb810f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:31.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.897+0000 7f8ebd600700 1 --2- 172.21.15.60:0/539803080 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8eb8108410 0x7f8eb81087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:31.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.897+0000 7f8ebd600700 1 --2- 172.21.15.60:0/539803080 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8eb81023a0 0x7f8eb8102820 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:31.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.897+0000 7f8ebd600700 1 -- 172.21.15.60:0/539803080 >> 172.21.15.60:0/539803080 conn(0x7f8eb8079350 msgr2=0x7f8eb8079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:48:31.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.897+0000 7f8ebd600700 1 -- 172.21.15.60:0/539803080 shutdown_connections 2024-05-03T14:48:31.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.897+0000 7f8ebd600700 1 -- 172.21.15.60:0/539803080 wait complete. 2024-05-03T14:48:31.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.898+0000 7f8ebd600700 1 Processor -- start 2024-05-03T14:48:31.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.898+0000 7f8ebd600700 1 -- start start 2024-05-03T14:48:31.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.898+0000 7f8ebd600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8eb81023a0 0x7f8eb819bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:31.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.898+0000 7f8ebd600700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8eb8102d60 0x7f8eb819c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:31.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.898+0000 7f8ebd600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8eb8108410 0x7f8eb81a0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:31.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.898+0000 7f8ebd600700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb8112840 con 0x7f8eb8102d60 2024-05-03T14:48:31.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.898+0000 7f8ebd600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb81126c0 con 0x7f8eb8108410 2024-05-03T14:48:31.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.898+0000 7f8ebd600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb81129c0 con 0x7f8eb81023a0 2024-05-03T14:48:31.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.899+0000 7f8eafe00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8eb8102d60 0x7f8eb819c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:31.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.899+0000 7f8eafe00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8eb8102d60 0x7f8eb819c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41838/0 (socket says 172.21.15.60:41838) 2024-05-03T14:48:31.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.899+0000 7f8eafe00700 1 -- 172.21.15.60:0/1474616075 learned_addr learned my addr 172.21.15.60:0/1474616075 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:48:31.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.899+0000 7f8eb6a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8eb81023a0 0x7f8eb819bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:31.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.899+0000 7f8eb7400700 1 --2- 172.21.15.60:0/1474616075 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8eb8108410 0x7f8eb81a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:31.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.899+0000 7f8eafe00700 1 -- 172.21.15.60:0/1474616075 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8eb81023a0 msgr2=0x7f8eb819bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:31.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.899+0000 7f8eafe00700 1 --2- 172.21.15.60:0/1474616075 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8eb81023a0 0x7f8eb819bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:31.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.899+0000 7f8eafe00700 1 -- 172.21.15.60:0/1474616075 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8eb8108410 msgr2=0x7f8eb81a0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:31.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.899+0000 7f8eafe00700 1 --2- 172.21.15.60:0/1474616075 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8eb8108410 0x7f8eb81a0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:31.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.899+0000 7f8eafe00700 1 -- 172.21.15.60:0/1474616075 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eb81a0f80 con 0x7f8eb8102d60 2024-05-03T14:48:31.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.899+0000 7f8eb7400700 1 --2- 172.21.15.60:0/1474616075 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8eb8108410 0x7f8eb81a0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T14:48:31.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.900+0000 7f8eafe00700 1 --2- 172.21.15.60:0/1474616075 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8eb8102d60 0x7f8eb819c4b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0000c00 tx=0x7f8ea0005590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:48:31.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.900+0000 7f8eaf400700 1 -- 172.21.15.60:0/1474616075 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea0027070 con 0x7f8eb8102d60 2024-05-03T14:48:31.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.900+0000 7f8eaf400700 1 -- 172.21.15.60:0/1474616075 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f8ea00056b0 con 0x7f8eb8102d60 2024-05-03T14:48:31.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.900+0000 7f8ebd600700 1 -- 172.21.15.60:0/1474616075 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8eb81a1210 con 0x7f8eb8102d60 2024-05-03T14:48:31.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.900+0000 7f8eaf400700 1 -- 172.21.15.60:0/1474616075 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea0017650 con 0x7f8eb8102d60 2024-05-03T14:48:31.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.900+0000 7f8ebd600700 1 -- 172.21.15.60:0/1474616075 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8eb81a8af0 con 0x7f8eb8102d60 2024-05-03T14:48:31.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.906+0000 7f8eaf400700 1 -- 172.21.15.60:0/1474616075 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f8ea002a070 con 0x7f8eb8102d60 2024-05-03T14:48:31.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.906+0000 7f8ec7eaeb80 1 Processor -- start 2024-05-03T14:48:31.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.907+0000 7f8ec7eaeb80 1 -- start start 2024-05-03T14:48:31.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.907+0000 7f8ec7eaeb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5596addbb760 0x5596addbbb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:31.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.907+0000 7f8ec7eaeb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5596addbc080 0x5596addb6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:31.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.907+0000 7f8ec7eaeb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596addb7580 0x5596addb79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:31.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.907+0000 7f8ec7eaeb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5596add099d0 con 0x5596addbb760 2024-05-03T14:48:31.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.907+0000 7f8ec7eaeb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5596adca7710 con 0x5596addb7580 2024-05-03T14:48:31.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.907+0000 7f8ec7eaeb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5596add05990 con 0x5596addbc080 2024-05-03T14:48:31.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.907+0000 7f8eb7400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5596addbb760 0x5596addbbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:31.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.907+0000 7f8eb7400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5596addbb760 0x5596addbbb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41852/0 (socket says 172.21.15.60:41852) 2024-05-03T14:48:31.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.907+0000 7f8eaf400700 1 --2- 172.21.15.60:0/1474616075 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f8e9c0b2910 0x7f8e9c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:31.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.908+0000 7f8eb7400700 1 -- 172.21.15.60:0/2934010844 learned_addr learned my addr 172.21.15.60:0/2934010844 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:48:31.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.908+0000 7f8eafe00700 1 --2- 172.21.15.60:0/2934010844 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596addb7580 0x5596addb79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:31.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.908+0000 7f8eb6a00700 1 --2- 172.21.15.60:0/2934010844 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5596addbc080 0x5596addb6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:31.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.908+0000 7f8eaf400700 1 -- 172.21.15.60:0/1474616075 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f8ea00e36d0 con 0x7f8eb8102d60 2024-05-03T14:48:31.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.908+0000 7f8eb7400700 1 --2- 172.21.15.60:0/1474616075 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f8e9c0b2910 0x7f8e9c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:31.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.908+0000 7f8eafe00700 1 -- 172.21.15.60:0/2934010844 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5596addbc080 msgr2=0x5596addb6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:48:31.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.908+0000 7f8eafe00700 1 --2- 172.21.15.60:0/2934010844 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5596addbc080 0x5596addb6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:31.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.908+0000 7f8eafe00700 1 -- 172.21.15.60:0/2934010844 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5596addbb760 msgr2=0x5596addbbb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:48:31.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.908+0000 7f8eafe00700 1 --2- 172.21.15.60:0/2934010844 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5596addbb760 0x5596addbbb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:31.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.908+0000 7f8eafe00700 1 -- 172.21.15.60:0/2934010844 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596adc56880 con 0x5596addb7580 2024-05-03T14:48:31.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.908+0000 7f8eb6a00700 1 --2- 172.21.15.60:0/2934010844 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5596addbc080 0x5596addb6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:48:31.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.909+0000 7f8eafe00700 1 --2- 172.21.15.60:0/2934010844 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596addb7580 0x5596addb79e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8ea00e0ca0 tx=0x7f8ea0092590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2802b2bef276721 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:48:31.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.909+0000 7f8ea7e00700 1 -- 172.21.15.60:0/2934010844 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea00a9fb0 con 0x5596addb7580 2024-05-03T14:48:31.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.909+0000 7f8ea7e00700 1 -- 172.21.15.60:0/2934010844 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f8ea00aa350 con 0x5596addb7580 2024-05-03T14:48:31.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.909+0000 7f8ea7e00700 1 -- 172.21.15.60:0/2934010844 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea00ae0d0 con 0x5596addb7580 2024-05-03T14:48:31.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.909+0000 7f8eb7400700 1 --2- 172.21.15.60:0/1474616075 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f8e9c0b2910 0x7f8e9c0b4dd0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8eb0007860 tx=0x7f8eb0004b40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:48:31.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.910+0000 7f8ec7eaeb80 1 -- 172.21.15.60:0/2934010844 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596addb7580 msgr2=0x5596addb79e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:31.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.910+0000 7f8ec7eaeb80 1 --2- 172.21.15.60:0/2934010844 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596addb7580 0x5596addb79e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8ea00e0ca0 tx=0x7f8ea0092590 comp rx=0 tx=0).stop 2024-05-03T14:48:31.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.910+0000 7f8ec7eaeb80 1 -- 172.21.15.60:0/2934010844 shutdown_connections 2024-05-03T14:48:31.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.910+0000 7f8ec7eaeb80 1 --2- 172.21.15.60:0/2934010844 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596addb7580 0x5596addb79e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:31.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.910+0000 7f8ec7eaeb80 1 --2- 172.21.15.60:0/2934010844 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5596addbb760 0x5596addbbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:31.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.910+0000 7f8ec7eaeb80 1 --2- 172.21.15.60:0/2934010844 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5596addbc080 0x5596addb6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:31.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.910+0000 7f8ec7eaeb80 1 -- 172.21.15.60:0/2934010844 >> 172.21.15.60:0/2934010844 conn(0x5596adce5b10 msgr2=0x5596addb17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:48:31.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.910+0000 7f8ec7eaeb80 1 -- 172.21.15.60:0/2934010844 shutdown_connections 2024-05-03T14:48:31.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.910+0000 7f8ec7eaeb80 1 -- 172.21.15.60:0/2934010844 wait complete. 2024-05-03T14:48:31.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.911+0000 7f8ec7eaeb80 1 -- start start 2024-05-03T14:48:31.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.911+0000 7f8ec7eaeb80 1 Processor -- start 2024-05-03T14:48:31.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.911+0000 7f8ec7eaeb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5596addb7580 0x5596addc2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:31.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.911+0000 7f8ec7eaeb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596addbb760 0x5596addc0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:31.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.911+0000 7f8ec7eaeb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5596addbc080 0x5596addc06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:31.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.911+0000 7f8ec7eaeb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5596adca7710 con 0x5596addbc080 2024-05-03T14:48:31.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.911+0000 7f8ec7eaeb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5596adc6fad0 con 0x5596addbb760 2024-05-03T14:48:31.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.912+0000 7f8ec7eaeb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5596addc3a10 con 0x5596addb7580 2024-05-03T14:48:31.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.912+0000 7f8ec7eaeb80 10 client.0 _ll_register_callbacks cb 0x5596adcb0c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T14:48:31.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.912+0000 7f8eb7400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5596addbc080 0x5596addc06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:31.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.912+0000 7f8eafe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596addbb760 0x5596addc0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:31.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.912+0000 7f8eafe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596addbb760 0x5596addc0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.60:45846/0 (socket says 172.21.15.60:45846) 2024-05-03T14:48:31.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.912+0000 7f8eb6a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5596addb7580 0x5596addc2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:31.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.912+0000 7f8eafe00700 1 -- 172.21.15.60:0/1702273411 learned_addr learned my addr 172.21.15.60:0/1702273411 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:48:31.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.912+0000 7f8eb7400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5596addbc080 0x5596addc06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41854/0 (socket says 172.21.15.60:41854) 2024-05-03T14:48:31.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.912+0000 7f8eafe00700 1 -- 172.21.15.60:0/1702273411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5596addb7580 msgr2=0x5596addc2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:48:31.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.912+0000 7f8eafe00700 1 --2- 172.21.15.60:0/1702273411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5596addb7580 0x5596addc2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:31.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.912+0000 7f8eafe00700 1 -- 172.21.15.60:0/1702273411 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5596addbc080 msgr2=0x5596addc06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:48:31.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.912+0000 7f8eafe00700 1 --2- 172.21.15.60:0/1702273411 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5596addbc080 0x5596addc06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:31.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.912+0000 7f8eafe00700 1 -- 172.21.15.60:0/1702273411 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596adccf3d0 con 0x5596addbb760 2024-05-03T14:48:31.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.913+0000 7f8eafe00700 1 --2- 172.21.15.60:0/1702273411 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596addbb760 0x5596addc0170 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8ea00aa7d0 tx=0x7f8ea0009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=885ea52bd9e3587f server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:48:31.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.913+0000 7f8ea4c00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-03T14:48:31.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.913+0000 7f8ea4c00700 1 -- 172.21.15.60:0/1702273411 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea00aaf40 con 0x5596addbb760 2024-05-03T14:48:31.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.913+0000 7f8ea4c00700 1 -- 172.21.15.60:0/1702273411 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f8ea00e33e0 con 0x5596addbb760 2024-05-03T14:48:31.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.913+0000 7f8ec7eaeb80 1 -- 172.21.15.60:0/1702273411 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ea00aa350 con 0x5596addbb760 2024-05-03T14:48:31.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.913+0000 7f8ea4c00700 1 -- 172.21.15.60:0/1702273411 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8ea00cd0d0 con 0x5596addbb760 2024-05-03T14:48:31.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.913+0000 7f8ec7eaeb80 10 client.25004 fetch_fsmap learned FSMap version 29 2024-05-03T14:48:31.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.913+0000 7f8ec7eaeb80 1 -- 172.21.15.60:0/1702273411 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5596addce000 con 0x5596addbb760 2024-05-03T14:48:31.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.914+0000 7f8ea4c00700 1 -- 172.21.15.60:0/1702273411 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f8ea00aa350 con 0x5596addbb760 2024-05-03T14:48:31.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.914+0000 7f8ea4c00700 1 -- 172.21.15.60:0/1702273411 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8ea00920e0 con 0x5596addbb760 2024-05-03T14:48:31.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.914+0000 7f8ec7eaeb80 10 client.25004 fetch_fsmap finished waiting for FSMap version 29 2024-05-03T14:48:31.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.914+0000 7f8ec7eaeb80 10 client.25004 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-03T14:48:31.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.914+0000 7f8ec7eaeb80 20 client.25004 populate_metadata read hostname 'smithi060' 2024-05-03T14:48:31.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.914+0000 7f8ec7eaeb80 1 --2- 172.21.15.60:0/1702273411 >> [v2:172.21.15.90:6838/3512381172,v1:172.21.15.90:6839/3512381172] conn(0x5596addd03d0 0x5596addd1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:31.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.914+0000 7f8ec7eaeb80 4 client.25004 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T14:48:31.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.914+0000 7f8ec7eaeb80 1 -- 172.21.15.60:0/1702273411 --> [v2:172.21.15.90:6838/3512381172,v1:172.21.15.90:6839/3512381172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596add84180 con 0x5596addd03d0 2024-05-03T14:48:31.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.914+0000 7f8eb6a00700 1 --2- 172.21.15.60:0/1702273411 >> [v2:172.21.15.90:6838/3512381172,v1:172.21.15.90:6839/3512381172] conn(0x5596addd03d0 0x5596addd1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:31.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.915+0000 7f8eb6a00700 1 --2- 172.21.15.60:0/1702273411 >> [v2:172.21.15.90:6838/3512381172,v1:172.21.15.90:6839/3512381172] conn(0x5596addd03d0 0x5596addd1840 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cdc0a92e1046d4d6 server_cookie=ab1a8ddb8c4710a6 in_seq=0 out_seq=0 2024-05-03T14:48:31.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.915+0000 7f8ea4c00700 10 client.25004 ms_handle_connect on v2:172.21.15.90:6838/3512381172 2024-05-03T14:48:31.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.916+0000 7f8ea4c00700 1 -- 172.21.15.60:0/1702273411 <== mds.1 v2:172.21.15.90:6838/3512381172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5596add84180 con 0x5596addd03d0 2024-05-03T14:48:31.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.916+0000 7f8ea4c00700 10 client.25004 handle_command_reply: tid=0 2024-05-03T14:48:31.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.916+0000 7f8ec7eaeb80 1 client.25004 shutdown 2024-05-03T14:48:31.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.916+0000 7f8ec7eaeb80 1 -- 172.21.15.60:0/1702273411 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596addbb760 msgr2=0x5596addc0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:31.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.916+0000 7f8ec7eaeb80 1 --2- 172.21.15.60:0/1702273411 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596addbb760 0x5596addc0170 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8ea00aa7d0 tx=0x7f8ea0009310 comp rx=0 tx=0).stop 2024-05-03T14:48:31.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.917+0000 7f8ec7eaeb80 1 -- 172.21.15.60:0/1702273411 shutdown_connections 2024-05-03T14:48:31.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.917+0000 7f8ec7eaeb80 1 --2- 172.21.15.60:0/1702273411 >> [v2:172.21.15.90:6838/3512381172,v1:172.21.15.90:6839/3512381172] conn(0x5596addd03d0 0x5596addd1840 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:31.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.917+0000 7f8ec7eaeb80 1 --2- 172.21.15.60:0/1702273411 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5596addbc080 0x5596addc06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:31.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.917+0000 7f8ec7eaeb80 1 --2- 172.21.15.60:0/1702273411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5596addb7580 0x5596addc2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:31.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.917+0000 7f8ec7eaeb80 1 --2- 172.21.15.60:0/1702273411 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596addbb760 0x5596addc0170 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:31.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.917+0000 7f8ea4c00700 0 client.25004 ms_handle_reset on v2:172.21.15.90:6838/3512381172 2024-05-03T14:48:31.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.917+0000 7f8ec7eaeb80 1 -- 172.21.15.60:0/1702273411 >> 172.21.15.60:0/1702273411 conn(0x5596adce5b10 msgr2=0x5596addb0600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:48:31.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.917+0000 7f8ec7eaeb80 1 -- 172.21.15.60:0/1702273411 shutdown_connections 2024-05-03T14:48:31.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.917+0000 7f8ec7eaeb80 1 -- 172.21.15.60:0/1702273411 wait complete. 2024-05-03T14:48:31.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.917+0000 7f8ec7eaeb80 20 client.25004 trim_cache size 0 max 16384 2024-05-03T14:48:31.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.940+0000 7f8ec7eaeb80 1 Processor -- start 2024-05-03T14:48:31.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.940+0000 7f8ec7eaeb80 1 -- start start 2024-05-03T14:48:31.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.941+0000 7f8ec7eaeb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5596addb7580 0x5596addb34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:31.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.941+0000 7f8ec7eaeb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596addbb760 0x5596addc1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:31.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.941+0000 7f8ec7eaeb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5596addbc080 0x5596addc1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:31.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.941+0000 7f8ec7eaeb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5596addcfb80 con 0x5596addbc080 2024-05-03T14:48:31.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.941+0000 7f8ec7eaeb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5596addd0250 con 0x5596addbb760 2024-05-03T14:48:31.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.941+0000 7f8ec7eaeb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5596addd0920 con 0x5596addb7580 2024-05-03T14:48:31.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.941+0000 7f8eb7400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5596addbc080 0x5596addc1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:31.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.941+0000 7f8eb6a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5596addb7580 0x5596addb34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:31.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.941+0000 7f8eb7400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5596addbc080 0x5596addc1cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41856/0 (socket says 172.21.15.60:41856) 2024-05-03T14:48:31.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.941+0000 7f8eb7400700 1 -- 172.21.15.60:0/1441997857 learned_addr learned my addr 172.21.15.60:0/1441997857 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:48:31.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.941+0000 7f8eafe00700 1 --2- 172.21.15.60:0/1441997857 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596addbb760 0x5596addc1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:31.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.942+0000 7f8eb6a00700 1 -- 172.21.15.60:0/1441997857 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596addbb760 msgr2=0x5596addc1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:48:31.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.942+0000 7f8eb6a00700 1 --2- 172.21.15.60:0/1441997857 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596addbb760 0x5596addc1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:31.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.942+0000 7f8eb6a00700 1 -- 172.21.15.60:0/1441997857 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5596addbc080 msgr2=0x5596addc1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:48:31.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.942+0000 7f8eb6a00700 1 --2- 172.21.15.60:0/1441997857 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5596addbc080 0x5596addc1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:31.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.942+0000 7f8eb6a00700 1 -- 172.21.15.60:0/1441997857 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596addbb3b0 con 0x5596addb7580 2024-05-03T14:48:31.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.942+0000 7f8eb6a00700 1 --2- 172.21.15.60:0/1441997857 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5596addb7580 0x5596addb34a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x5596addd2200 tx=0x7f8ea8009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46d8c78e68c8e9f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:48:31.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.942+0000 7f8ea7400700 1 -- 172.21.15.60:0/1441997857 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea8003c40 con 0x5596addb7580 2024-05-03T14:48:31.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.942+0000 7f8ea7400700 1 -- 172.21.15.60:0/1441997857 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f8ea8003de0 con 0x5596addb7580 2024-05-03T14:48:31.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.942+0000 7f8ea7400700 1 -- 172.21.15.60:0/1441997857 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea801d450 con 0x5596addb7580 2024-05-03T14:48:31.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.943+0000 7f8ec7eaeb80 1 -- 172.21.15.60:0/1441997857 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5596addb7580 msgr2=0x5596addb34a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:31.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.943+0000 7f8ec7eaeb80 1 --2- 172.21.15.60:0/1441997857 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5596addb7580 0x5596addb34a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x5596addd2200 tx=0x7f8ea8009260 comp rx=0 tx=0).stop 2024-05-03T14:48:31.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.943+0000 7f8ec7eaeb80 1 -- 172.21.15.60:0/1441997857 shutdown_connections 2024-05-03T14:48:31.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.943+0000 7f8ec7eaeb80 1 --2- 172.21.15.60:0/1441997857 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5596addbc080 0x5596addc1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:31.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.943+0000 7f8ec7eaeb80 1 --2- 172.21.15.60:0/1441997857 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5596addb7580 0x5596addb34a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:31.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.943+0000 7f8ec7eaeb80 1 --2- 172.21.15.60:0/1441997857 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596addbb760 0x5596addc1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:31.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.943+0000 7f8ec7eaeb80 1 -- 172.21.15.60:0/1441997857 >> 172.21.15.60:0/1441997857 conn(0x5596adce5b10 msgr2=0x5596addcf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:48:31.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.943+0000 7f8ec7eaeb80 1 -- 172.21.15.60:0/1441997857 shutdown_connections 2024-05-03T14:48:31.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.943+0000 7f8ec7eaeb80 1 -- 172.21.15.60:0/1441997857 wait complete. 2024-05-03T14:48:31.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.944+0000 7f8ec7eaeb80 1 -- start start 2024-05-03T14:48:31.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.944+0000 7f8ec7eaeb80 1 Processor -- start 2024-05-03T14:48:31.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.944+0000 7f8ec7eaeb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5596addb7580 0x5596adddacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:31.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.944+0000 7f8ec7eaeb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596addbb760 0x5596adddb200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:31.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.944+0000 7f8ec7eaeb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5596addbc080 0x5596addd3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:31.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.944+0000 7f8ec7eaeb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5596addd16c0 con 0x5596addb7580 2024-05-03T14:48:31.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.944+0000 7f8ec7eaeb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5596addd0ff0 con 0x5596addbb760 2024-05-03T14:48:31.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.944+0000 7f8ec7eaeb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5596addd1d90 con 0x5596addbc080 2024-05-03T14:48:31.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.944+0000 7f8ec7eaeb80 10 client.0 _ll_register_callbacks cb 0x5596adcb0c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T14:48:31.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.945+0000 7f8eb6a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5596addb7580 0x5596adddacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:31.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.945+0000 7f8eb6a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5596addb7580 0x5596adddacc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41860/0 (socket says 172.21.15.60:41860) 2024-05-03T14:48:31.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.945+0000 7f8eafe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596addbb760 0x5596adddb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:31.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.945+0000 7f8eb6a00700 1 -- 172.21.15.60:0/3312675101 learned_addr learned my addr 172.21.15.60:0/3312675101 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:48:31.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.945+0000 7f8eb7400700 1 --2- 172.21.15.60:0/3312675101 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5596addbc080 0x5596addd3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:31.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.945+0000 7f8eb6a00700 1 -- 172.21.15.60:0/3312675101 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5596addbc080 msgr2=0x5596addd3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:48:31.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.945+0000 7f8eb6a00700 1 --2- 172.21.15.60:0/3312675101 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5596addbc080 0x5596addd3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:31.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.945+0000 7f8eb6a00700 1 -- 172.21.15.60:0/3312675101 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596addbb760 msgr2=0x5596adddb200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:48:31.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.945+0000 7f8eb6a00700 1 --2- 172.21.15.60:0/3312675101 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596addbb760 0x5596adddb200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:31.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.945+0000 7f8eb6a00700 1 -- 172.21.15.60:0/3312675101 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596addd42a0 con 0x5596addb7580 2024-05-03T14:48:31.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.946+0000 7f8eb6a00700 1 --2- 172.21.15.60:0/3312675101 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5596addb7580 0x5596adddacc0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8ea80046e0 tx=0x7f8ea8009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e48aab4005bafd10 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:48:31.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.946+0000 7f8ea4c00700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-05-03T14:48:31.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.946+0000 7f8ea4c00700 1 -- 172.21.15.60:0/3312675101 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea8003a40 con 0x5596addb7580 2024-05-03T14:48:31.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.946+0000 7f8ea4c00700 1 -- 172.21.15.60:0/3312675101 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f8ea8003be0 con 0x5596addb7580 2024-05-03T14:48:31.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.946+0000 7f8ec7eaeb80 1 -- 172.21.15.60:0/3312675101 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ea8003de0 con 0x5596addb7580 2024-05-03T14:48:31.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.946+0000 7f8ea4c00700 1 -- 172.21.15.60:0/3312675101 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8ea8035050 con 0x5596addb7580 2024-05-03T14:48:31.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.946+0000 7f8ec7eaeb80 10 client.14910 fetch_fsmap learned FSMap version 29 2024-05-03T14:48:31.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.946+0000 7f8ec7eaeb80 1 -- 172.21.15.60:0/3312675101 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5596ade44950 con 0x5596addb7580 2024-05-03T14:48:31.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.947+0000 7f8ea4c00700 1 -- 172.21.15.60:0/3312675101 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f8ea8003de0 con 0x5596addb7580 2024-05-03T14:48:31.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.947+0000 7f8ea4c00700 1 -- 172.21.15.60:0/3312675101 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8ea801d570 con 0x5596addb7580 2024-05-03T14:48:31.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.947+0000 7f8ec7eaeb80 10 client.14910 fetch_fsmap finished waiting for FSMap version 29 2024-05-03T14:48:31.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.947+0000 7f8ec7eaeb80 10 client.14910 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-03T14:48:31.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.947+0000 7f8ec7eaeb80 20 client.14910 populate_metadata read hostname 'smithi060' 2024-05-03T14:48:31.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.947+0000 7f8ec7eaeb80 1 --2- 172.21.15.60:0/3312675101 >> [v2:172.21.15.90:6838/3512381172,v1:172.21.15.90:6839/3512381172] conn(0x5596ade49260 0x5596ade47f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:31.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.947+0000 7f8ec7eaeb80 4 client.14910 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-03T14:48:31.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.947+0000 7f8ec7eaeb80 1 -- 172.21.15.60:0/3312675101 --> [v2:172.21.15.90:6838/3512381172,v1:172.21.15.90:6839/3512381172] -- command(tid 0: {"prefix": "status"}) -- 0x5596add173b0 con 0x5596ade49260 2024-05-03T14:48:31.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.947+0000 7f8eafe00700 1 --2- 172.21.15.60:0/3312675101 >> [v2:172.21.15.90:6838/3512381172,v1:172.21.15.90:6839/3512381172] conn(0x5596ade49260 0x5596ade47f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:31.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.948+0000 7f8eafe00700 1 --2- 172.21.15.60:0/3312675101 >> [v2:172.21.15.90:6838/3512381172,v1:172.21.15.90:6839/3512381172] conn(0x5596ade49260 0x5596ade47f70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3588950c5dc27da server_cookie=7fd6e0d2e92ac15e in_seq=0 out_seq=0 2024-05-03T14:48:31.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.948+0000 7f8ea4c00700 10 client.14910 ms_handle_connect on v2:172.21.15.90:6838/3512381172 2024-05-03T14:48:31.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.948+0000 7f8ea4c00700 1 -- 172.21.15.60:0/3312675101 <== mds.1 v2:172.21.15.90:6838/3512381172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5596add173b0 con 0x5596ade49260 2024-05-03T14:48:31.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.948+0000 7f8ea4c00700 10 client.14910 handle_command_reply: tid=0 2024-05-03T14:48:31.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.948+0000 7f8ec7eaeb80 1 client.14910 shutdown 2024-05-03T14:48:31.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.948+0000 7f8ec7eaeb80 1 -- 172.21.15.60:0/3312675101 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5596addb7580 msgr2=0x5596adddacc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:31.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.948+0000 7f8ec7eaeb80 1 --2- 172.21.15.60:0/3312675101 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5596addb7580 0x5596adddacc0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8ea80046e0 tx=0x7f8ea8009bf0 comp rx=0 tx=0).stop 2024-05-03T14:48:31.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.949+0000 7f8ec7eaeb80 1 -- 172.21.15.60:0/3312675101 shutdown_connections 2024-05-03T14:48:31.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.949+0000 7f8ec7eaeb80 1 --2- 172.21.15.60:0/3312675101 >> [v2:172.21.15.90:6838/3512381172,v1:172.21.15.90:6839/3512381172] conn(0x5596ade49260 0x5596ade47f70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:31.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.949+0000 7f8ec7eaeb80 1 --2- 172.21.15.60:0/3312675101 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5596addbc080 0x5596addd3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:31.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.949+0000 7f8ec7eaeb80 1 --2- 172.21.15.60:0/3312675101 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5596addb7580 0x5596adddacc0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:31.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.949+0000 7f8ec7eaeb80 1 --2- 172.21.15.60:0/3312675101 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596addbb760 0x5596adddb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:31.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.949+0000 7f8ec7eaeb80 1 -- 172.21.15.60:0/3312675101 >> 172.21.15.60:0/3312675101 conn(0x5596adce5b10 msgr2=0x5596addb5f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:48:31.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.949+0000 7f8ea4c00700 0 client.14910 ms_handle_reset on v2:172.21.15.90:6838/3512381172 2024-05-03T14:48:31.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.949+0000 7f8ec7eaeb80 1 -- 172.21.15.60:0/3312675101 shutdown_connections 2024-05-03T14:48:31.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.949+0000 7f8ec7eaeb80 1 -- 172.21.15.60:0/3312675101 wait complete. 2024-05-03T14:48:31.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.949+0000 7f8ec7eaeb80 20 client.14910 trim_cache size 0 max 16384 2024-05-03T14:48:31.954 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-05-03T14:48:31.954 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "285d64b2-095a-11ef-bc97-c7b262605968", 2024-05-03T14:48:31.954 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 1, 2024-05-03T14:48:31.954 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 24509, 2024-05-03T14:48:31.954 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-05-03T14:48:31.954 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-05-03T14:48:31.954 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-05-03T14:48:31.955 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 158.47380753499999, 2024-05-03T14:48:31.955 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 29, 2024-05-03T14:48:31.955 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 89, 2024-05-03T14:48:31.955 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 87, 2024-05-03T14:48:31.955 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 257.61026232799998 2024-05-03T14:48:31.955 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-05-03T14:48:31.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.950+0000 7f8ebd600700 1 -- 172.21.15.60:0/1474616075 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f8e9c0b2910 msgr2=0x7f8e9c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:31.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.950+0000 7f8ebd600700 1 --2- 172.21.15.60:0/1474616075 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f8e9c0b2910 0x7f8e9c0b4dd0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8eb0007860 tx=0x7f8eb0004b40 comp rx=0 tx=0).stop 2024-05-03T14:48:31.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.950+0000 7f8ebd600700 1 -- 172.21.15.60:0/1474616075 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8eb8102d60 msgr2=0x7f8eb819c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:31.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.950+0000 7f8ebd600700 1 --2- 172.21.15.60:0/1474616075 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8eb8102d60 0x7f8eb819c4b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0000c00 tx=0x7f8ea0005590 comp rx=0 tx=0).stop 2024-05-03T14:48:31.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.950+0000 7f8ebd600700 1 -- 172.21.15.60:0/1474616075 shutdown_connections 2024-05-03T14:48:31.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.950+0000 7f8ebd600700 1 --2- 172.21.15.60:0/1474616075 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f8e9c0b2910 0x7f8e9c0b4dd0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:31.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.950+0000 7f8ebd600700 1 --2- 172.21.15.60:0/1474616075 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8eb8108410 0x7f8eb81a0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:31.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.950+0000 7f8ebd600700 1 --2- 172.21.15.60:0/1474616075 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8eb81023a0 0x7f8eb819bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:31.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.950+0000 7f8ebd600700 1 --2- 172.21.15.60:0/1474616075 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8eb8102d60 0x7f8eb819c4b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:31.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.950+0000 7f8ebd600700 1 -- 172.21.15.60:0/1474616075 >> 172.21.15.60:0/1474616075 conn(0x7f8eb8079350 msgr2=0x7f8eb806b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:48:31.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.950+0000 7f8ebd600700 1 -- 172.21.15.60:0/1474616075 shutdown_connections 2024-05-03T14:48:31.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:31.950+0000 7f8ebd600700 1 -- 172.21.15.60:0/1474616075 wait complete. 2024-05-03T14:48:31.969 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:48:31 smithi060 ceph-mon[28873]: pgmap v452: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-03T14:48:32.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:48:31 smithi090 ceph-mon[33568]: pgmap v452: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-03T14:48:32.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:48:31 smithi149 ceph-mon[33332]: pgmap v452: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-03T14:48:32.897 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph tell mds.a status 2024-05-03T14:48:33.358 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:48:33.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:48:33 smithi060 ceph-mon[28873]: pgmap v453: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T14:48:34.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:48:33 smithi090 ceph-mon[33568]: pgmap v453: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T14:48:34.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:48:33 smithi149 ceph-mon[33332]: pgmap v453: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T14:48:35.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.581+0000 7f636d400700 1 -- 172.21.15.60:0/1283744414 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f636806b180 msgr2=0x7f636806b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:35.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.581+0000 7f636d400700 1 --2- 172.21.15.60:0/1283744414 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f636806b180 0x7f636806b600 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6354009990 tx=0x7f6354009ca0 comp rx=0 tx=0).stop 2024-05-03T14:48:35.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.581+0000 7f636d400700 1 -- 172.21.15.60:0/1283744414 shutdown_connections 2024-05-03T14:48:35.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.581+0000 7f636d400700 1 --2- 172.21.15.60:0/1283744414 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f636806bb40 0x7f6368110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:35.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.581+0000 7f636d400700 1 --2- 172.21.15.60:0/1283744414 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6368109510 0x7f63681098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:35.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.581+0000 7f636d400700 1 --2- 172.21.15.60:0/1283744414 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f636806b180 0x7f636806b600 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:35.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.581+0000 7f636d400700 1 -- 172.21.15.60:0/1283744414 >> 172.21.15.60:0/1283744414 conn(0x7f6368079350 msgr2=0x7f6368079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:48:35.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.582+0000 7f636d400700 1 -- 172.21.15.60:0/1283744414 shutdown_connections 2024-05-03T14:48:35.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.582+0000 7f636d400700 1 -- 172.21.15.60:0/1283744414 wait complete. 2024-05-03T14:48:35.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.583+0000 7f636d400700 1 Processor -- start 2024-05-03T14:48:35.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.583+0000 7f636d400700 1 -- start start 2024-05-03T14:48:35.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.583+0000 7f636d400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f636806b180 0x7f636819bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:35.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.583+0000 7f636d400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f636806bb40 0x7f636819c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:35.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.583+0000 7f636d400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6368109510 0x7f63681a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:35.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.583+0000 7f636d400700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f6368113900 con 0x7f636806b180 2024-05-03T14:48:35.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.583+0000 7f636d400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6368113780 con 0x7f6368109510 2024-05-03T14:48:35.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.583+0000 7f636d400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6368113a80 con 0x7f636806bb40 2024-05-03T14:48:35.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.584+0000 7f6366800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f636806b180 0x7f636819bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:35.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.584+0000 7f6366800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f636806b180 0x7f636819bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41880/0 (socket says 172.21.15.60:41880) 2024-05-03T14:48:35.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.584+0000 7f6366800700 1 -- 172.21.15.60:0/1671853642 learned_addr learned my addr 172.21.15.60:0/1671853642 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:48:35.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.584+0000 7f6365e00700 1 --2- 172.21.15.60:0/1671853642 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f636806bb40 0x7f636819c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:35.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.584+0000 7f6367200700 1 --2- 172.21.15.60:0/1671853642 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6368109510 0x7f63681a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:35.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.584+0000 7f6366800700 1 -- 172.21.15.60:0/1671853642 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f636806bb40 msgr2=0x7f636819c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:35.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.584+0000 7f6366800700 1 --2- 172.21.15.60:0/1671853642 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f636806bb40 0x7f636819c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:35.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.584+0000 7f6366800700 1 -- 172.21.15.60:0/1671853642 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6368109510 msgr2=0x7f63681a07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:35.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.584+0000 7f6366800700 1 --2- 172.21.15.60:0/1671853642 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6368109510 0x7f63681a07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:35.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.584+0000 7f6366800700 1 -- 172.21.15.60:0/1671853642 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63681a0f30 con 0x7f636806b180 2024-05-03T14:48:35.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.584+0000 7f6367200700 1 --2- 172.21.15.60:0/1671853642 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6368109510 0x7f63681a07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:48:35.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.584+0000 7f6365e00700 1 --2- 172.21.15.60:0/1671853642 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f636806bb40 0x7f636819c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T14:48:35.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.584+0000 7f6366800700 1 --2- 172.21.15.60:0/1671853642 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f636806b180 0x7f636819bf20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f635800e9a0 tx=0x7f635800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:48:35.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.585+0000 7f635f400700 1 -- 172.21.15.60:0/1671853642 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f635800cbd0 con 0x7f636806b180 2024-05-03T14:48:35.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.585+0000 7f636d400700 1 -- 172.21.15.60:0/1671853642 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63681a1220 con 0x7f636806b180 2024-05-03T14:48:35.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.585+0000 7f635f400700 1 -- 172.21.15.60:0/1671853642 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f635800cd70 con 0x7f636806b180 2024-05-03T14:48:35.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.585+0000 7f635f400700 1 -- 172.21.15.60:0/1671853642 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6358018820 con 0x7f636806b180 2024-05-03T14:48:35.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.585+0000 7f636d400700 1 -- 172.21.15.60:0/1671853642 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63681a8aa0 con 0x7f636806b180 2024-05-03T14:48:35.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.590+0000 7f6377d22b80 1 Processor -- start 2024-05-03T14:48:35.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.590+0000 7f6377d22b80 1 -- start start 2024-05-03T14:48:35.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.590+0000 7f6377d22b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561a12e798e0 0x561a12e79cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:35.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.590+0000 7f6377d22b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561a12e7a200 0x561a12e73ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:35.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.590+0000 7f6377d22b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561a12e74500 0x561a12e74980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:35.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.590+0000 7f6377d22b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x561a12dc69d0 con 0x561a12e7a200 2024-05-03T14:48:35.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.590+0000 7f6377d22b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561a12d64710 con 0x561a12e798e0 2024-05-03T14:48:35.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.590+0000 7f6377d22b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x561a12dc2990 con 0x561a12e74500 2024-05-03T14:48:35.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.591+0000 7f635f400700 1 -- 172.21.15.60:0/1671853642 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f6358010430 con 0x7f636806b180 2024-05-03T14:48:35.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.591+0000 7f6367200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561a12e798e0 0x561a12e79cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:35.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.591+0000 7f6367200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561a12e798e0 0x561a12e79cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.60:45886/0 (socket says 172.21.15.60:45886) 2024-05-03T14:48:35.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.591+0000 7f6367200700 1 -- 172.21.15.60:0/4150550278 learned_addr learned my addr 172.21.15.60:0/4150550278 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:48:35.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.591+0000 7f6365e00700 1 --2- 172.21.15.60:0/4150550278 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561a12e74500 0x561a12e74980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:35.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.591+0000 7f6366800700 1 --2- 172.21.15.60:0/4150550278 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561a12e7a200 0x561a12e73ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:35.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.591+0000 7f6367200700 1 -- 172.21.15.60:0/4150550278 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561a12e74500 msgr2=0x561a12e74980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:48:35.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.591+0000 7f6367200700 1 --2- 172.21.15.60:0/4150550278 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561a12e74500 0x561a12e74980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:35.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.591+0000 7f6367200700 1 -- 172.21.15.60:0/4150550278 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561a12e7a200 msgr2=0x561a12e73ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:48:35.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.591+0000 7f6367200700 1 --2- 172.21.15.60:0/4150550278 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561a12e7a200 0x561a12e73ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:35.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.591+0000 7f6367200700 1 -- 172.21.15.60:0/4150550278 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a12d13880 con 0x561a12e798e0 2024-05-03T14:48:35.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.592+0000 7f6367200700 1 --2- 172.21.15.60:0/4150550278 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561a12e798e0 0x561a12e79cc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6360009510 tx=0x7f6360009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=defdfbe16d52a8e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:48:35.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.592+0000 7f6353e00700 1 -- 172.21.15.60:0/4150550278 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636001b020 con 0x561a12e798e0 2024-05-03T14:48:35.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.592+0000 7f6353e00700 1 -- 172.21.15.60:0/4150550278 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f636000ebf0 con 0x561a12e798e0 2024-05-03T14:48:35.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.592+0000 7f6353e00700 1 -- 172.21.15.60:0/4150550278 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6360017e20 con 0x561a12e798e0 2024-05-03T14:48:35.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.593+0000 7f635f400700 1 --2- 172.21.15.60:0/1671853642 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f634c0b2910 0x7f634c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:35.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.593+0000 7f635f400700 1 -- 172.21.15.60:0/1671853642 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f63580e29b0 con 0x7f636806b180 2024-05-03T14:48:35.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.593+0000 7f6377d22b80 1 -- 172.21.15.60:0/4150550278 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561a12e798e0 msgr2=0x561a12e79cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:35.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.593+0000 7f6377d22b80 1 --2- 172.21.15.60:0/4150550278 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561a12e798e0 0x561a12e79cc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6360009510 tx=0x7f6360009920 comp rx=0 tx=0).stop 2024-05-03T14:48:35.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.593+0000 7f6366800700 1 --2- 172.21.15.60:0/1671853642 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f634c0b2910 0x7f634c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:35.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.593+0000 7f6377d22b80 1 -- 172.21.15.60:0/4150550278 shutdown_connections 2024-05-03T14:48:35.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.593+0000 7f6377d22b80 1 --2- 172.21.15.60:0/4150550278 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561a12e74500 0x561a12e74980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:35.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.593+0000 7f6377d22b80 1 --2- 172.21.15.60:0/4150550278 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561a12e798e0 0x561a12e79cc0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:35.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.593+0000 7f6377d22b80 1 --2- 172.21.15.60:0/4150550278 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561a12e7a200 0x561a12e73ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:35.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.593+0000 7f6377d22b80 1 -- 172.21.15.60:0/4150550278 >> 172.21.15.60:0/4150550278 conn(0x561a12dec680 msgr2=0x561a12e6e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:48:35.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.594+0000 7f6377d22b80 1 -- 172.21.15.60:0/4150550278 shutdown_connections 2024-05-03T14:48:35.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.594+0000 7f6377d22b80 1 -- 172.21.15.60:0/4150550278 wait complete. 2024-05-03T14:48:35.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.594+0000 7f6366800700 1 --2- 172.21.15.60:0/1671853642 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f634c0b2910 0x7f634c0b4dd0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f63580a9a20 tx=0x7f63580e5f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:48:35.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.594+0000 7f6377d22b80 1 -- start start 2024-05-03T14:48:35.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.595+0000 7f6377d22b80 1 Processor -- start 2024-05-03T14:48:35.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.595+0000 7f6377d22b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561a12e74500 0x561a12e7fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:35.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.595+0000 7f6377d22b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561a12e798e0 0x561a12e80280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:35.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.595+0000 7f6377d22b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561a12e7a200 0x561a12e7d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:35.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.595+0000 7f6377d22b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x561a12e80cb0 con 0x561a12e798e0 2024-05-03T14:48:35.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.595+0000 7f6377d22b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561a12d2cad0 con 0x561a12e7a200 2024-05-03T14:48:35.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.595+0000 7f6377d22b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x561a12e80e30 con 0x561a12e74500 2024-05-03T14:48:35.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.595+0000 7f6377d22b80 10 client.0 _ll_register_callbacks cb 0x561a12d6dc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T14:48:35.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.596+0000 7f6365e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561a12e798e0 0x561a12e80280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:35.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.596+0000 7f6365e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561a12e798e0 0x561a12e80280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41906/0 (socket says 172.21.15.60:41906) 2024-05-03T14:48:35.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.596+0000 7f6365e00700 1 -- 172.21.15.60:0/3877576968 learned_addr learned my addr 172.21.15.60:0/3877576968 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:48:35.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.596+0000 7f6367200700 1 --2- 172.21.15.60:0/3877576968 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561a12e7a200 0x561a12e7d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:35.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.596+0000 7f6367200700 1 --2- 172.21.15.60:0/3877576968 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561a12e74500 0x561a12e7fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:35.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.596+0000 7f6365e00700 1 -- 172.21.15.60:0/3877576968 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561a12e74500 msgr2=0x561a12e7fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:48:35.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.596+0000 7f6365e00700 1 --2- 172.21.15.60:0/3877576968 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561a12e74500 0x561a12e7fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:35.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.596+0000 7f6365e00700 1 -- 172.21.15.60:0/3877576968 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561a12e7a200 msgr2=0x561a12e7d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:48:35.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.596+0000 7f6365e00700 1 --2- 172.21.15.60:0/3877576968 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561a12e7a200 0x561a12e7d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:35.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.596+0000 7f6365e00700 1 -- 172.21.15.60:0/3877576968 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a12d8c3d0 con 0x561a12e798e0 2024-05-03T14:48:35.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.596+0000 7f6367200700 1 --2- 172.21.15.60:0/3877576968 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561a12e74500 0x561a12e7fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T14:48:35.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.597+0000 7f6365e00700 1 --2- 172.21.15.60:0/3877576968 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561a12e798e0 0x561a12e80280 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6354000c00 tx=0x7f6354005ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9e380d806b481e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:48:35.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.597+0000 7f6350c00700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-05-03T14:48:35.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.597+0000 7f6350c00700 1 -- 172.21.15.60:0/3877576968 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6354027c30 con 0x561a12e798e0 2024-05-03T14:48:35.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.597+0000 7f6350c00700 1 -- 172.21.15.60:0/3877576968 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f6354027dd0 con 0x561a12e798e0 2024-05-03T14:48:35.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.597+0000 7f6377d22b80 1 -- 172.21.15.60:0/3877576968 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f636000ebf0 con 0x561a12e798e0 2024-05-03T14:48:35.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.597+0000 7f6350c00700 1 -- 172.21.15.60:0/3877576968 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6354038050 con 0x561a12e798e0 2024-05-03T14:48:35.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.597+0000 7f6377d22b80 10 client.14922 fetch_fsmap learned FSMap version 29 2024-05-03T14:48:35.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.597+0000 7f6377d22b80 1 -- 172.21.15.60:0/3877576968 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561a12e8b340 con 0x561a12e798e0 2024-05-03T14:48:35.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.598+0000 7f6350c00700 1 -- 172.21.15.60:0/3877576968 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f636000ebf0 con 0x561a12e798e0 2024-05-03T14:48:35.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.598+0000 7f6350c00700 1 -- 172.21.15.60:0/3877576968 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6354003680 con 0x561a12e798e0 2024-05-03T14:48:35.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.598+0000 7f6377d22b80 10 client.14922 fetch_fsmap finished waiting for FSMap version 29 2024-05-03T14:48:35.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.598+0000 7f6377d22b80 10 client.14922 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-03T14:48:35.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.598+0000 7f6377d22b80 20 client.14922 populate_metadata read hostname 'smithi060' 2024-05-03T14:48:35.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.598+0000 7f6377d22b80 1 --2- 172.21.15.60:0/3877576968 >> [v2:172.21.15.60:6840/924163182,v1:172.21.15.60:6841/924163182] conn(0x561a12e8d940 0x561a12e8edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:35.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.598+0000 7f6377d22b80 4 client.14922 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T14:48:35.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.598+0000 7f6377d22b80 1 -- 172.21.15.60:0/3877576968 --> [v2:172.21.15.60:6840/924163182,v1:172.21.15.60:6841/924163182] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a12e40fa0 con 0x561a12e8d940 2024-05-03T14:48:35.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.599+0000 7f6367200700 1 --2- 172.21.15.60:0/3877576968 >> [v2:172.21.15.60:6840/924163182,v1:172.21.15.60:6841/924163182] conn(0x561a12e8d940 0x561a12e8edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:35.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.599+0000 7f6367200700 1 --2- 172.21.15.60:0/3877576968 >> [v2:172.21.15.60:6840/924163182,v1:172.21.15.60:6841/924163182] conn(0x561a12e8d940 0x561a12e8edb0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=896d323b9a475035 server_cookie=beac30243a6b68d3 in_seq=0 out_seq=0 2024-05-03T14:48:35.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.599+0000 7f6350c00700 10 client.14922 ms_handle_connect on v2:172.21.15.60:6840/924163182 2024-05-03T14:48:35.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.601+0000 7f6350c00700 1 -- 172.21.15.60:0/3877576968 <== mds.0 v2:172.21.15.60:6840/924163182 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561a12e40fa0 con 0x561a12e8d940 2024-05-03T14:48:35.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.601+0000 7f6350c00700 10 client.14922 handle_command_reply: tid=0 2024-05-03T14:48:35.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.601+0000 7f6377d22b80 1 client.14922 shutdown 2024-05-03T14:48:35.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.601+0000 7f6377d22b80 1 -- 172.21.15.60:0/3877576968 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561a12e798e0 msgr2=0x561a12e80280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:35.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.601+0000 7f6377d22b80 1 --2- 172.21.15.60:0/3877576968 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561a12e798e0 0x561a12e80280 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6354000c00 tx=0x7f6354005ec0 comp rx=0 tx=0).stop 2024-05-03T14:48:35.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.601+0000 7f6377d22b80 1 -- 172.21.15.60:0/3877576968 shutdown_connections 2024-05-03T14:48:35.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.601+0000 7f6377d22b80 1 --2- 172.21.15.60:0/3877576968 >> [v2:172.21.15.60:6840/924163182,v1:172.21.15.60:6841/924163182] conn(0x561a12e8d940 0x561a12e8edb0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:35.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.601+0000 7f6377d22b80 1 --2- 172.21.15.60:0/3877576968 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561a12e7a200 0x561a12e7d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:35.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.601+0000 7f6377d22b80 1 --2- 172.21.15.60:0/3877576968 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561a12e74500 0x561a12e7fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:35.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.601+0000 7f6377d22b80 1 --2- 172.21.15.60:0/3877576968 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561a12e798e0 0x561a12e80280 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:35.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.601+0000 7f6350c00700 0 client.14922 ms_handle_reset on v2:172.21.15.60:6840/924163182 2024-05-03T14:48:35.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.601+0000 7f6377d22b80 1 -- 172.21.15.60:0/3877576968 >> 172.21.15.60:0/3877576968 conn(0x561a12dec680 msgr2=0x561a12e69360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:48:35.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.601+0000 7f6377d22b80 1 -- 172.21.15.60:0/3877576968 shutdown_connections 2024-05-03T14:48:35.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.602+0000 7f6377d22b80 1 -- 172.21.15.60:0/3877576968 wait complete. 2024-05-03T14:48:35.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.602+0000 7f6377d22b80 20 client.14922 trim_cache size 0 max 16384 2024-05-03T14:48:35.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.625+0000 7f6377d22b80 1 Processor -- start 2024-05-03T14:48:35.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.625+0000 7f6377d22b80 1 -- start start 2024-05-03T14:48:35.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.625+0000 7f6377d22b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561a12e74500 0x561a12e73ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:35.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.625+0000 7f6377d22b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561a12e798e0 0x561a12e70160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:35.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.625+0000 7f6377d22b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561a12e7a200 0x561a12e7fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:35.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.625+0000 7f6377d22b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x561a12e8d0f0 con 0x561a12e7a200 2024-05-03T14:48:35.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.625+0000 7f6377d22b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561a12e8d7c0 con 0x561a12e74500 2024-05-03T14:48:35.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.625+0000 7f6377d22b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x561a12e8de90 con 0x561a12e798e0 2024-05-03T14:48:35.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.626+0000 7f6365e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561a12e798e0 0x561a12e70160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:35.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.626+0000 7f6365e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561a12e798e0 0x561a12e70160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.60:34836/0 (socket says 172.21.15.60:34836) 2024-05-03T14:48:35.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.626+0000 7f6365e00700 1 -- 172.21.15.60:0/326406555 learned_addr learned my addr 172.21.15.60:0/326406555 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:48:35.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.626+0000 7f6367200700 1 --2- 172.21.15.60:0/326406555 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561a12e74500 0x561a12e73ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:35.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.626+0000 7f6367200700 1 --2- 172.21.15.60:0/326406555 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561a12e7a200 0x561a12e7fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:35.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.626+0000 7f6365e00700 1 -- 172.21.15.60:0/326406555 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561a12e74500 msgr2=0x561a12e73ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:48:35.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.626+0000 7f6365e00700 1 --2- 172.21.15.60:0/326406555 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561a12e74500 0x561a12e73ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:35.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.626+0000 7f6365e00700 1 -- 172.21.15.60:0/326406555 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561a12e7a200 msgr2=0x561a12e7fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:48:35.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.626+0000 7f6365e00700 1 --2- 172.21.15.60:0/326406555 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561a12e7a200 0x561a12e7fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:35.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.626+0000 7f6365e00700 1 -- 172.21.15.60:0/326406555 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a12e7b650 con 0x561a12e798e0 2024-05-03T14:48:35.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.626+0000 7f6367200700 1 --2- 172.21.15.60:0/326406555 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561a12e7a200 0x561a12e7fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T14:48:35.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.627+0000 7f6365e00700 1 --2- 172.21.15.60:0/326406555 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561a12e798e0 0x561a12e70160 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f635400b860 tx=0x7f6354003cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afd2e1163c2613e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:48:35.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.627+0000 7f6353400700 1 -- 172.21.15.60:0/326406555 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6354003e60 con 0x561a12e798e0 2024-05-03T14:48:35.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.627+0000 7f6353400700 1 -- 172.21.15.60:0/326406555 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f635402b030 con 0x561a12e798e0 2024-05-03T14:48:35.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.627+0000 7f6353400700 1 -- 172.21.15.60:0/326406555 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6354030420 con 0x561a12e798e0 2024-05-03T14:48:35.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.628+0000 7f6377d22b80 1 -- 172.21.15.60:0/326406555 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561a12e798e0 msgr2=0x561a12e70160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:35.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.628+0000 7f6377d22b80 1 --2- 172.21.15.60:0/326406555 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561a12e798e0 0x561a12e70160 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f635400b860 tx=0x7f6354003cc0 comp rx=0 tx=0).stop 2024-05-03T14:48:35.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.628+0000 7f6377d22b80 1 -- 172.21.15.60:0/326406555 shutdown_connections 2024-05-03T14:48:35.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.628+0000 7f6377d22b80 1 --2- 172.21.15.60:0/326406555 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561a12e7a200 0x561a12e7fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:35.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.628+0000 7f6377d22b80 1 --2- 172.21.15.60:0/326406555 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561a12e74500 0x561a12e73ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:35.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.628+0000 7f6377d22b80 1 --2- 172.21.15.60:0/326406555 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561a12e798e0 0x561a12e70160 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:35.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.628+0000 7f6377d22b80 1 -- 172.21.15.60:0/326406555 >> 172.21.15.60:0/326406555 conn(0x561a12dec680 msgr2=0x561a12e773f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:48:35.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.628+0000 7f6377d22b80 1 -- 172.21.15.60:0/326406555 shutdown_connections 2024-05-03T14:48:35.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.628+0000 7f6377d22b80 1 -- 172.21.15.60:0/326406555 wait complete. 2024-05-03T14:48:35.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.629+0000 7f6377d22b80 1 -- start start 2024-05-03T14:48:35.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.629+0000 7f6377d22b80 1 Processor -- start 2024-05-03T14:48:35.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.629+0000 7f6377d22b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561a12e74500 0x561a12e98230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:35.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.629+0000 7f6377d22b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561a12e798e0 0x561a12e98770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:35.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.629+0000 7f6377d22b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561a12e7a200 0x561a12e931e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:35.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.629+0000 7f6377d22b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x561a12e8ec30 con 0x561a12e74500 2024-05-03T14:48:35.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.629+0000 7f6377d22b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561a12e8e560 con 0x561a12e7a200 2024-05-03T14:48:35.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.629+0000 7f6377d22b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x561a12e8f300 con 0x561a12e798e0 2024-05-03T14:48:35.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.629+0000 7f6377d22b80 10 client.0 _ll_register_callbacks cb 0x561a12d6dc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T14:48:35.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.630+0000 7f6367200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561a12e74500 0x561a12e98230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:35.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.630+0000 7f6365e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561a12e798e0 0x561a12e98770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:35.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.630+0000 7f6365e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561a12e798e0 0x561a12e98770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.60:34838/0 (socket says 172.21.15.60:34838) 2024-05-03T14:48:35.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.630+0000 7f6365e00700 1 -- 172.21.15.60:0/718283128 learned_addr learned my addr 172.21.15.60:0/718283128 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:48:35.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.630+0000 7f6367200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561a12e74500 0x561a12e98230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41922/0 (socket says 172.21.15.60:41922) 2024-05-03T14:48:35.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.630+0000 7f6367200700 1 --2- 172.21.15.60:0/718283128 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561a12e7a200 0x561a12e931e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:35.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.630+0000 7f6365e00700 1 -- 172.21.15.60:0/718283128 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561a12e7a200 msgr2=0x561a12e931e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:48:35.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.630+0000 7f6365e00700 1 --2- 172.21.15.60:0/718283128 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561a12e7a200 0x561a12e931e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:35.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.630+0000 7f6365e00700 1 -- 172.21.15.60:0/718283128 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561a12e74500 msgr2=0x561a12e98230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:48:35.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.630+0000 7f6365e00700 1 --2- 172.21.15.60:0/718283128 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561a12e74500 0x561a12e98230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:35.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.630+0000 7f6365e00700 1 -- 172.21.15.60:0/718283128 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a12e74240 con 0x561a12e798e0 2024-05-03T14:48:35.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.630+0000 7f6365e00700 1 --2- 172.21.15.60:0/718283128 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561a12e798e0 0x561a12e98770 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6354003a50 tx=0x7f63540096e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b58b8bf0e2f58e13 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:48:35.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.630+0000 7f6350c00700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-05-03T14:48:35.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.631+0000 7f6350c00700 1 -- 172.21.15.60:0/718283128 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6354027c30 con 0x561a12e798e0 2024-05-03T14:48:35.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.631+0000 7f6350c00700 1 -- 172.21.15.60:0/718283128 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f6354027dd0 con 0x561a12e798e0 2024-05-03T14:48:35.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.631+0000 7f6377d22b80 1 -- 172.21.15.60:0/718283128 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f635402b030 con 0x561a12e798e0 2024-05-03T14:48:35.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.631+0000 7f6350c00700 1 -- 172.21.15.60:0/718283128 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f635404b050 con 0x561a12e798e0 2024-05-03T14:48:35.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.631+0000 7f6377d22b80 10 client.25036 fetch_fsmap learned FSMap version 29 2024-05-03T14:48:35.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.631+0000 7f6377d22b80 1 -- 172.21.15.60:0/718283128 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561a12ef9410 con 0x561a12e798e0 2024-05-03T14:48:35.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.632+0000 7f6350c00700 1 -- 172.21.15.60:0/718283128 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f635402b030 con 0x561a12e798e0 2024-05-03T14:48:35.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.632+0000 7f6350c00700 1 -- 172.21.15.60:0/718283128 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6354043020 con 0x561a12e798e0 2024-05-03T14:48:35.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.632+0000 7f6377d22b80 10 client.25036 fetch_fsmap finished waiting for FSMap version 29 2024-05-03T14:48:35.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.632+0000 7f6377d22b80 10 client.25036 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-03T14:48:35.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.632+0000 7f6377d22b80 20 client.25036 populate_metadata read hostname 'smithi060' 2024-05-03T14:48:35.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.632+0000 7f6377d22b80 1 --2- 172.21.15.60:0/718283128 >> [v2:172.21.15.60:6840/924163182,v1:172.21.15.60:6841/924163182] conn(0x561a12efdea0 0x561a12efcbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:35.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.632+0000 7f6377d22b80 4 client.25036 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-03T14:48:35.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.632+0000 7f6377d22b80 1 -- 172.21.15.60:0/718283128 --> [v2:172.21.15.60:6840/924163182,v1:172.21.15.60:6841/924163182] -- command(tid 0: {"prefix": "status"}) -- 0x561a12dd43b0 con 0x561a12efdea0 2024-05-03T14:48:35.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.632+0000 7f6367200700 1 --2- 172.21.15.60:0/718283128 >> [v2:172.21.15.60:6840/924163182,v1:172.21.15.60:6841/924163182] conn(0x561a12efdea0 0x561a12efcbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:35.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.633+0000 7f6367200700 1 --2- 172.21.15.60:0/718283128 >> [v2:172.21.15.60:6840/924163182,v1:172.21.15.60:6841/924163182] conn(0x561a12efdea0 0x561a12efcbb0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afb68a60b39622d2 server_cookie=bd564815e1d6e7c3 in_seq=0 out_seq=0 2024-05-03T14:48:35.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.633+0000 7f6350c00700 10 client.25036 ms_handle_connect on v2:172.21.15.60:6840/924163182 2024-05-03T14:48:35.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.633+0000 7f6350c00700 1 -- 172.21.15.60:0/718283128 <== mds.0 v2:172.21.15.60:6840/924163182 1 ==== command_reply(tid 0: 0 ) ==== 8+0+320 (crc 0 0 0) 0x561a12dd43b0 con 0x561a12efdea0 2024-05-03T14:48:35.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.633+0000 7f6350c00700 10 client.25036 handle_command_reply: tid=0 2024-05-03T14:48:35.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.633+0000 7f6377d22b80 1 client.25036 shutdown 2024-05-03T14:48:35.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.634+0000 7f6377d22b80 1 -- 172.21.15.60:0/718283128 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561a12e798e0 msgr2=0x561a12e98770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:35.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.634+0000 7f6377d22b80 1 --2- 172.21.15.60:0/718283128 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561a12e798e0 0x561a12e98770 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6354003a50 tx=0x7f63540096e0 comp rx=0 tx=0).stop 2024-05-03T14:48:35.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.634+0000 7f6377d22b80 1 -- 172.21.15.60:0/718283128 shutdown_connections 2024-05-03T14:48:35.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.634+0000 7f6377d22b80 1 --2- 172.21.15.60:0/718283128 >> [v2:172.21.15.60:6840/924163182,v1:172.21.15.60:6841/924163182] conn(0x561a12efdea0 0x561a12efcbb0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:35.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.634+0000 7f6377d22b80 1 --2- 172.21.15.60:0/718283128 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561a12e7a200 0x561a12e931e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:35.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.634+0000 7f6377d22b80 1 --2- 172.21.15.60:0/718283128 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561a12e74500 0x561a12e98230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:35.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.634+0000 7f6377d22b80 1 --2- 172.21.15.60:0/718283128 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561a12e798e0 0x561a12e98770 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:35.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.634+0000 7f6377d22b80 1 -- 172.21.15.60:0/718283128 >> 172.21.15.60:0/718283128 conn(0x561a12dec680 msgr2=0x561a12e70160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:48:35.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.634+0000 7f6350c00700 0 client.25036 ms_handle_reset on v2:172.21.15.60:6840/924163182 2024-05-03T14:48:35.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.634+0000 7f6377d22b80 1 -- 172.21.15.60:0/718283128 shutdown_connections 2024-05-03T14:48:35.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.634+0000 7f6377d22b80 1 -- 172.21.15.60:0/718283128 wait complete. 2024-05-03T14:48:35.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.634+0000 7f6377d22b80 20 client.25036 trim_cache size 0 max 16384 2024-05-03T14:48:35.639 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-05-03T14:48:35.639 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "285d64b2-095a-11ef-bc97-c7b262605968", 2024-05-03T14:48:35.639 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2024-05-03T14:48:35.639 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 24511, 2024-05-03T14:48:35.639 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-05-03T14:48:35.639 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-05-03T14:48:35.639 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-05-03T14:48:35.639 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 241.170298917, 2024-05-03T14:48:35.640 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 29, 2024-05-03T14:48:35.640 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 89, 2024-05-03T14:48:35.640 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 87, 2024-05-03T14:48:35.640 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 264.1190689 2024-05-03T14:48:35.640 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-05-03T14:48:35.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.635+0000 7f636d400700 1 -- 172.21.15.60:0/1671853642 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f634c0b2910 msgr2=0x7f634c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:35.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.635+0000 7f636d400700 1 --2- 172.21.15.60:0/1671853642 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f634c0b2910 0x7f634c0b4dd0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f63580a9a20 tx=0x7f63580e5f90 comp rx=0 tx=0).stop 2024-05-03T14:48:35.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.635+0000 7f636d400700 1 -- 172.21.15.60:0/1671853642 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f636806b180 msgr2=0x7f636819bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:35.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.635+0000 7f636d400700 1 --2- 172.21.15.60:0/1671853642 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f636806b180 0x7f636819bf20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f635800e9a0 tx=0x7f635800ed60 comp rx=0 tx=0).stop 2024-05-03T14:48:35.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.635+0000 7f636d400700 1 -- 172.21.15.60:0/1671853642 shutdown_connections 2024-05-03T14:48:35.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.635+0000 7f636d400700 1 --2- 172.21.15.60:0/1671853642 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f634c0b2910 0x7f634c0b4dd0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:35.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.635+0000 7f636d400700 1 --2- 172.21.15.60:0/1671853642 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6368109510 0x7f63681a07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:35.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.635+0000 7f636d400700 1 --2- 172.21.15.60:0/1671853642 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f636806b180 0x7f636819bf20 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:35.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.635+0000 7f636d400700 1 --2- 172.21.15.60:0/1671853642 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f636806bb40 0x7f636819c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:35.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.635+0000 7f636d400700 1 -- 172.21.15.60:0/1671853642 >> 172.21.15.60:0/1671853642 conn(0x7f6368079350 msgr2=0x7f636810e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:48:35.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.636+0000 7f636d400700 1 -- 172.21.15.60:0/1671853642 shutdown_connections 2024-05-03T14:48:35.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:35.636+0000 7f636d400700 1 -- 172.21.15.60:0/1671853642 wait complete. 2024-05-03T14:48:36.082 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:48:35 smithi060 ceph-mon[28873]: pgmap v454: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-03T14:48:36.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:48:35 smithi090 ceph-mon[33568]: pgmap v454: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-03T14:48:36.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:48:35 smithi149 ceph-mon[33332]: pgmap v454: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-03T14:48:36.566 INFO:tasks.cephadm:Setup complete, yielding 2024-05-03T14:48:36.567 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-03T14:48:36.578 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi060.front.sepia.ceph.com 2024-05-03T14:48:36.578 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- bash -c 'ceph orch status' 2024-05-03T14:48:37.043 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:48:37.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:48:37 smithi060 ceph-mon[28873]: pgmap v455: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-03T14:48:38.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:48:37 smithi090 ceph-mon[33568]: pgmap v455: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-03T14:48:38.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:48:37 smithi149 ceph-mon[33332]: pgmap v455: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-03T14:48:39.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:39.327+0000 7f3b2ea00700 1 -- 172.21.15.60:0/2673556162 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3b28109b40 msgr2=0x7f3b28109f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:39.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:39.327+0000 7f3b2ea00700 1 --2- 172.21.15.60:0/2673556162 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3b28109b40 0x7f3b28109f60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3b180098f0 tx=0x7f3b18009c00 comp rx=0 tx=0).stop 2024-05-03T14:48:39.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:39.327+0000 7f3b2ea00700 1 -- 172.21.15.60:0/2673556162 shutdown_connections 2024-05-03T14:48:39.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:39.327+0000 7f3b2ea00700 1 --2- 172.21.15.60:0/2673556162 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b2810b6e0 0x7f3b28111f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:39.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:39.327+0000 7f3b2ea00700 1 --2- 172.21.15.60:0/2673556162 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3b28109b40 0x7f3b28109f60 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:39.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:39.327+0000 7f3b2ea00700 1 --2- 172.21.15.60:0/2673556162 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b2810ad40 0x7f3b2810b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:39.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:39.327+0000 7f3b2ea00700 1 -- 172.21.15.60:0/2673556162 >> 172.21.15.60:0/2673556162 conn(0x7f3b28078b10 msgr2=0x7f3b2807af50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:48:39.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:39.327+0000 7f3b2ea00700 1 -- 172.21.15.60:0/2673556162 shutdown_connections 2024-05-03T14:48:39.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:39.327+0000 7f3b2ea00700 1 -- 172.21.15.60:0/2673556162 wait complete. 2024-05-03T14:48:39.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:39.328+0000 7f3b2ea00700 1 Processor -- start 2024-05-03T14:48:39.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:39.328+0000 7f3b2ea00700 1 -- start start 2024-05-03T14:48:39.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:39.329+0000 7f3b2ea00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3b28109b40 0x7f3b2819fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:39.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:39.329+0000 7f3b2ea00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b2810ad40 0x7f3b281a0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:39.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:39.329+0000 7f3b2ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b2810b6e0 0x7f3b281a7400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:39.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:39.329+0000 7f3b2ea00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f3b28115890 con 0x7f3b2810ad40 2024-05-03T14:48:39.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:39.329+0000 7f3b2ea00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3b28115710 con 0x7f3b2810b6e0 2024-05-03T14:48:39.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:39.329+0000 7f3b2ea00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3b28115a10 con 0x7f3b28109b40 2024-05-03T14:48:39.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:39.329+0000 7f3b27e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3b28109b40 0x7f3b2819fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:39.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:39.329+0000 7f3b27400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b2810ad40 0x7f3b281a0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:39.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:39.329+0000 7f3b27e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3b28109b40 0x7f3b2819fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.60:34866/0 (socket says 172.21.15.60:34866) 2024-05-03T14:48:39.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:39.329+0000 7f3b27e00700 1 -- 172.21.15.60:0/4043428053 learned_addr learned my addr 172.21.15.60:0/4043428053 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:48:39.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:39.329+0000 7f3b2ca00700 1 --2- 172.21.15.60:0/4043428053 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b2810b6e0 0x7f3b281a7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:39.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:39.329+0000 7f3b27e00700 1 -- 172.21.15.60:0/4043428053 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b2810b6e0 msgr2=0x7f3b281a7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:39.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:39.329+0000 7f3b27e00700 1 --2- 172.21.15.60:0/4043428053 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b2810b6e0 0x7f3b281a7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:39.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:39.329+0000 7f3b27e00700 1 -- 172.21.15.60:0/4043428053 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b2810ad40 msgr2=0x7f3b281a0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:39.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:39.329+0000 7f3b27e00700 1 --2- 172.21.15.60:0/4043428053 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b2810ad40 0x7f3b281a0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:39.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:39.329+0000 7f3b27e00700 1 -- 172.21.15.60:0/4043428053 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b281a7aa0 con 0x7f3b28109b40 2024-05-03T14:48:39.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:39.330+0000 7f3b27e00700 1 --2- 172.21.15.60:0/4043428053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3b28109b40 0x7f3b2819fe40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3b18009230 tx=0x7f3b18005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:48:39.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:39.330+0000 7f3b24c00700 1 -- 172.21.15.60:0/4043428053 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b180056a0 con 0x7f3b28109b40 2024-05-03T14:48:39.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:39.330+0000 7f3b24c00700 1 -- 172.21.15.60:0/4043428053 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f3b1800ba60 con 0x7f3b28109b40 2024-05-03T14:48:39.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:39.330+0000 7f3b2ea00700 1 -- 172.21.15.60:0/4043428053 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b281a7d30 con 0x7f3b28109b40 2024-05-03T14:48:39.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:39.330+0000 7f3b24c00700 1 -- 172.21.15.60:0/4043428053 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b180216c0 con 0x7f3b28109b40 2024-05-03T14:48:39.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:39.330+0000 7f3b2ea00700 1 -- 172.21.15.60:0/4043428053 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b281a8190 con 0x7f3b28109b40 2024-05-03T14:48:39.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:39.331+0000 7f3b2ea00700 1 -- 172.21.15.60:0/4043428053 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b2806a190 con 0x7f3b28109b40 2024-05-03T14:48:39.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:39.334+0000 7f3b24c00700 1 -- 172.21.15.60:0/4043428053 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f3b180037a0 con 0x7f3b28109b40 2024-05-03T14:48:39.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:39.335+0000 7f3b24c00700 1 --2- 172.21.15.60:0/4043428053 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f3b0c0b2990 0x7f3b0c0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:39.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:39.335+0000 7f3b24c00700 1 -- 172.21.15.60:0/4043428053 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f3b180e3540 con 0x7f3b28109b40 2024-05-03T14:48:39.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:39.335+0000 7f3b27400700 1 --2- 172.21.15.60:0/4043428053 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f3b0c0b2990 0x7f3b0c0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:39.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:39.336+0000 7f3b27400700 1 --2- 172.21.15.60:0/4043428053 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f3b0c0b2990 0x7f3b0c0b4e50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3b14009750 tx=0x7f3b14009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:48:39.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:39.340+0000 7f3b24c00700 1 -- 172.21.15.60:0/4043428053 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f3b18013030 con 0x7f3b28109b40 2024-05-03T14:48:39.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:39.608+0000 7f3b2ea00700 1 -- 172.21.15.60:0/4043428053 --> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f3b280638e0 con 0x7f3b0c0b2990 2024-05-03T14:48:39.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:39.610+0000 7f3b24c00700 1 -- 172.21.15.60:0/4043428053 <== mgr.14152 v2:172.21.15.60:6800/3066142956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f3b280638e0 con 0x7f3b0c0b2990 2024-05-03T14:48:39.611 INFO:teuthology.orchestra.run.smithi060.stdout:Backend: cephadm 2024-05-03T14:48:39.611 INFO:teuthology.orchestra.run.smithi060.stdout:Available: Yes 2024-05-03T14:48:39.612 INFO:teuthology.orchestra.run.smithi060.stdout:Paused: No 2024-05-03T14:48:39.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:39.613+0000 7f3b2ea00700 1 -- 172.21.15.60:0/4043428053 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f3b0c0b2990 msgr2=0x7f3b0c0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:39.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:39.613+0000 7f3b2ea00700 1 --2- 172.21.15.60:0/4043428053 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f3b0c0b2990 0x7f3b0c0b4e50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3b14009750 tx=0x7f3b14009310 comp rx=0 tx=0).stop 2024-05-03T14:48:39.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:39.613+0000 7f3b2ea00700 1 -- 172.21.15.60:0/4043428053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3b28109b40 msgr2=0x7f3b2819fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:39.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:39.613+0000 7f3b2ea00700 1 --2- 172.21.15.60:0/4043428053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3b28109b40 0x7f3b2819fe40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3b18009230 tx=0x7f3b18005270 comp rx=0 tx=0).stop 2024-05-03T14:48:39.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:39.613+0000 7f3b2ea00700 1 -- 172.21.15.60:0/4043428053 shutdown_connections 2024-05-03T14:48:39.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:39.613+0000 7f3b2ea00700 1 --2- 172.21.15.60:0/4043428053 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f3b0c0b2990 0x7f3b0c0b4e50 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:39.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:39.613+0000 7f3b2ea00700 1 --2- 172.21.15.60:0/4043428053 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b2810b6e0 0x7f3b281a7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:39.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:39.613+0000 7f3b2ea00700 1 --2- 172.21.15.60:0/4043428053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3b28109b40 0x7f3b2819fe40 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:39.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:39.613+0000 7f3b2ea00700 1 --2- 172.21.15.60:0/4043428053 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b2810ad40 0x7f3b281a0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:39.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:39.613+0000 7f3b2ea00700 1 -- 172.21.15.60:0/4043428053 >> 172.21.15.60:0/4043428053 conn(0x7f3b28078b10 msgr2=0x7f3b2810e040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:48:39.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:39.613+0000 7f3b2ea00700 1 -- 172.21.15.60:0/4043428053 shutdown_connections 2024-05-03T14:48:39.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:39.614+0000 7f3b2ea00700 1 -- 172.21.15.60:0/4043428053 wait complete. 2024-05-03T14:48:40.032 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:48:39 smithi060 ceph-mon[28873]: pgmap v456: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-05-03T14:48:40.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:48:39 smithi090 ceph-mon[33568]: pgmap v456: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-05-03T14:48:40.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:48:39 smithi149 ceph-mon[33332]: pgmap v456: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-05-03T14:48:40.436 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-03T14:48:40.906 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:48:41.056 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:48:40 smithi060 ceph-mon[28873]: from='client.25042 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:48:41.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:48:40 smithi090 ceph-mon[33568]: from='client.25042 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:48:41.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:48:40 smithi149 ceph-mon[33332]: from='client.25042 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:48:42.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:48:41 smithi060 ceph-mon[28873]: pgmap v457: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-03T14:48:42.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:48:41 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:48:42.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:48:41 smithi090 ceph-mon[33568]: pgmap v457: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-03T14:48:42.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:48:41 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:48:42.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:48:41 smithi149 ceph-mon[33332]: pgmap v457: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-03T14:48:42.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:48:41 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:48:42.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:42.909+0000 7f6ed3e00700 1 -- 172.21.15.60:0/1813125698 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6ecc075e80 msgr2=0x7f6ecc076300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:42.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:42.909+0000 7f6ed3e00700 1 --2- 172.21.15.60:0/1813125698 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6ecc075e80 0x7f6ecc076300 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8009990 tx=0x7f6eb8009ca0 comp rx=0 tx=0).stop 2024-05-03T14:48:42.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:42.909+0000 7f6ed3e00700 1 -- 172.21.15.60:0/1813125698 shutdown_connections 2024-05-03T14:48:42.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:42.909+0000 7f6ed3e00700 1 --2- 172.21.15.60:0/1813125698 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ecc10b400 0x7f6ecc10d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:42.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:42.909+0000 7f6ed3e00700 1 --2- 172.21.15.60:0/1813125698 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ecc077450 0x7f6ecc0758b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:42.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:42.909+0000 7f6ed3e00700 1 --2- 172.21.15.60:0/1813125698 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6ecc075e80 0x7f6ecc076300 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:42.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:42.909+0000 7f6ed3e00700 1 -- 172.21.15.60:0/1813125698 >> 172.21.15.60:0/1813125698 conn(0x7f6ecc0feaa0 msgr2=0x7f6ecc100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:48:42.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:42.910+0000 7f6ed3e00700 1 -- 172.21.15.60:0/1813125698 shutdown_connections 2024-05-03T14:48:42.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:42.910+0000 7f6ed3e00700 1 -- 172.21.15.60:0/1813125698 wait complete. 2024-05-03T14:48:42.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:42.911+0000 7f6ed3e00700 1 Processor -- start 2024-05-03T14:48:42.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:42.911+0000 7f6ed3e00700 1 -- start start 2024-05-03T14:48:42.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:42.911+0000 7f6ed3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ecc075e80 0x7f6ecc19b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:42.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:42.911+0000 7f6ed3e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ecc077450 0x7f6ecc19bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:42.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:42.911+0000 7f6ed3e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6ecc10b400 0x7f6ecc1a2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:42.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:42.911+0000 7f6ed3e00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc1113f0 con 0x7f6ecc077450 2024-05-03T14:48:42.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:42.911+0000 7f6ed3e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc111270 con 0x7f6ecc075e80 2024-05-03T14:48:42.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:42.911+0000 7f6ed3e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc111570 con 0x7f6ecc10b400 2024-05-03T14:48:42.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:42.912+0000 7f6ed1400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ecc075e80 0x7f6ecc19b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:42.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:42.912+0000 7f6ed0a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ecc077450 0x7f6ecc19bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:42.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:42.912+0000 7f6ed1e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6ecc10b400 0x7f6ecc1a2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:42.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:42.912+0000 7f6ed1e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6ecc10b400 0x7f6ecc1a2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.60:39512/0 (socket says 172.21.15.60:39512) 2024-05-03T14:48:42.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:42.912+0000 7f6ed1400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ecc075e80 0x7f6ecc19b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.60:56792/0 (socket says 172.21.15.60:56792) 2024-05-03T14:48:42.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:42.912+0000 7f6ed1e00700 1 -- 172.21.15.60:0/1420198133 learned_addr learned my addr 172.21.15.60:0/1420198133 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:48:42.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:42.912+0000 7f6ed1400700 1 -- 172.21.15.60:0/1420198133 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6ecc10b400 msgr2=0x7f6ecc1a2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:42.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:42.912+0000 7f6ed1400700 1 --2- 172.21.15.60:0/1420198133 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6ecc10b400 0x7f6ecc1a2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:42.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:42.912+0000 7f6ed1400700 1 -- 172.21.15.60:0/1420198133 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ecc077450 msgr2=0x7f6ecc19bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:42.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:42.912+0000 7f6ed1400700 1 --2- 172.21.15.60:0/1420198133 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ecc077450 0x7f6ecc19bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:42.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:42.912+0000 7f6ed1400700 1 -- 172.21.15.60:0/1420198133 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ecc1a3650 con 0x7f6ecc075e80 2024-05-03T14:48:42.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:42.913+0000 7f6ed1400700 1 --2- 172.21.15.60:0/1420198133 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ecc075e80 0x7f6ecc19b9f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6ec000e9f0 tx=0x7f6ec000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:48:42.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:42.913+0000 7f6eca000700 1 -- 172.21.15.60:0/1420198133 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec000c9c0 con 0x7f6ecc075e80 2024-05-03T14:48:42.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:42.913+0000 7f6eca000700 1 -- 172.21.15.60:0/1420198133 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f6ec000cb60 con 0x7f6ecc075e80 2024-05-03T14:48:42.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:42.913+0000 7f6eca000700 1 -- 172.21.15.60:0/1420198133 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec0018710 con 0x7f6ecc075e80 2024-05-03T14:48:42.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:42.913+0000 7f6ed3e00700 1 -- 172.21.15.60:0/1420198133 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ecc1a3940 con 0x7f6ecc075e80 2024-05-03T14:48:42.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:42.913+0000 7f6ed3e00700 1 -- 172.21.15.60:0/1420198133 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ecc1a3e80 con 0x7f6ecc075e80 2024-05-03T14:48:42.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:42.914+0000 7f6ed3e00700 1 -- 172.21.15.60:0/1420198133 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ecc06a190 con 0x7f6ecc075e80 2024-05-03T14:48:42.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:42.916+0000 7f6eca000700 1 -- 172.21.15.60:0/1420198133 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f6ec000cd10 con 0x7f6ecc075e80 2024-05-03T14:48:42.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:42.918+0000 7f6eca000700 1 --2- 172.21.15.60:0/1420198133 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f6eb40b2910 0x7f6eb40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:42.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:42.919+0000 7f6eca000700 1 -- 172.21.15.60:0/1420198133 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f6ec00e2d00 con 0x7f6ecc075e80 2024-05-03T14:48:42.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:42.919+0000 7f6ed0a00700 1 --2- 172.21.15.60:0/1420198133 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f6eb40b2910 0x7f6eb40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:42.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:42.920+0000 7f6ed0a00700 1 --2- 172.21.15.60:0/1420198133 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f6eb40b2910 0x7f6eb40b4dd0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6eb800b2e0 tx=0x7f6eb800b8c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:48:42.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:42.920+0000 7f6eca000700 1 -- 172.21.15.60:0/1420198133 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f6ec00aa4e0 con 0x7f6ecc075e80 2024-05-03T14:48:43.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:48:42 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:48:43.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:48:42 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:48:43.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:48:42 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:48:43.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:48:42 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:48:43.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:48:42 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:48:43.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:48:42 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:48:43.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:43.168+0000 7f6ed3e00700 1 -- 172.21.15.60:0/1420198133 --> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f6ecc0639a0 con 0x7f6eb40b2910 2024-05-03T14:48:43.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:48:42 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:48:43.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:48:42 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:48:43.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:48:42 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:48:43.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:43.210+0000 7f6eca000700 1 -- 172.21.15.60:0/1420198133 <== mgr.14152 v2:172.21.15.60:6800/3066142956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f6ecc0639a0 con 0x7f6eb40b2910 2024-05-03T14:48:43.212 INFO:teuthology.orchestra.run.smithi060.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-03T14:48:43.213 INFO:teuthology.orchestra.run.smithi060.stdout:mds.a smithi060 running (4m) 2m ago 4m 25.0M - 19.0.0-3450-g9a047fe7 ee4d84620249 4750ab8adb52 2024-05-03T14:48:43.213 INFO:teuthology.orchestra.run.smithi060.stdout:mds.b smithi090 running (4m) 2m ago 4m 18.0M - 19.0.0-3450-g9a047fe7 ee4d84620249 0f91cd739243 2024-05-03T14:48:43.213 INFO:teuthology.orchestra.run.smithi060.stdout:mds.c smithi149 running (4m) 2m ago 4m 19.9M - 19.0.0-3450-g9a047fe7 ee4d84620249 45b18e5ebe10 2024-05-03T14:48:43.213 INFO:teuthology.orchestra.run.smithi060.stdout:mds.d smithi060 running (4m) 2m ago 4m 13.2M - 19.0.0-3450-g9a047fe7 ee4d84620249 5ffd9e3041e2 2024-05-03T14:48:43.213 INFO:teuthology.orchestra.run.smithi060.stdout:mds.e smithi090 running (4m) 2m ago 4m 16.8M - 19.0.0-3450-g9a047fe7 ee4d84620249 5daafecc9028 2024-05-03T14:48:43.213 INFO:teuthology.orchestra.run.smithi060.stdout:mds.f smithi149 running (4m) 2m ago 4m 17.6M - 19.0.0-3450-g9a047fe7 ee4d84620249 937f42513b12 2024-05-03T14:48:43.213 INFO:teuthology.orchestra.run.smithi060.stdout:mds.g smithi060 running (4m) 2m ago 4m 15.2M - 19.0.0-3450-g9a047fe7 ee4d84620249 596ec49ce14c 2024-05-03T14:48:43.213 INFO:teuthology.orchestra.run.smithi060.stdout:mds.h smithi090 running (4m) 2m ago 4m 17.8M - 19.0.0-3450-g9a047fe7 ee4d84620249 025a6e743988 2024-05-03T14:48:43.213 INFO:teuthology.orchestra.run.smithi060.stdout:mds.i smithi149 running (4m) 2m ago 4m 20.0M - 19.0.0-3450-g9a047fe7 ee4d84620249 2fa92e0610c5 2024-05-03T14:48:43.213 INFO:teuthology.orchestra.run.smithi060.stdout:mds.j smithi060 running (5m) 2m ago 5m 13.2M - 19.0.0-3450-g9a047fe7 ee4d84620249 6e88f6f17e7d 2024-05-03T14:48:43.213 INFO:teuthology.orchestra.run.smithi060.stdout:mds.k smithi090 running (5m) 2m ago 5m 19.1M - 19.0.0-3450-g9a047fe7 ee4d84620249 4b2334b62b44 2024-05-03T14:48:43.213 INFO:teuthology.orchestra.run.smithi060.stdout:mds.l smithi149 running (5m) 2m ago 5m 16.6M - 19.0.0-3450-g9a047fe7 ee4d84620249 9c403211bfc4 2024-05-03T14:48:43.214 INFO:teuthology.orchestra.run.smithi060.stdout:mgr.x smithi060 *:9283,8765 running (13m) 2m ago 13m 531M - 19.0.0-3450-g9a047fe7 ee4d84620249 5813c13738d0 2024-05-03T14:48:43.214 INFO:teuthology.orchestra.run.smithi060.stdout:mgr.y smithi090 *:8443,8765 running (11m) 2m ago 11m 465M - 19.0.0-3450-g9a047fe7 ee4d84620249 d260c61b912b 2024-05-03T14:48:43.214 INFO:teuthology.orchestra.run.smithi060.stdout:mgr.z smithi149 *:8443,8765 running (11m) 2m ago 11m 468M - 19.0.0-3450-g9a047fe7 ee4d84620249 5a2811f816a5 2024-05-03T14:48:43.214 INFO:teuthology.orchestra.run.smithi060.stdout:mon.a smithi060 running (13m) 2m ago 13m 53.4M 2048M 19.0.0-3450-g9a047fe7 ee4d84620249 1149168d47be 2024-05-03T14:48:43.214 INFO:teuthology.orchestra.run.smithi060.stdout:mon.b smithi090 running (11m) 2m ago 11m 43.5M 2048M 19.0.0-3450-g9a047fe7 ee4d84620249 308ea0b9e173 2024-05-03T14:48:43.214 INFO:teuthology.orchestra.run.smithi060.stdout:mon.c smithi149 running (11m) 2m ago 11m 42.8M 2048M 19.0.0-3450-g9a047fe7 ee4d84620249 2a9091c8d79b 2024-05-03T14:48:43.214 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0 smithi060 running (10m) 2m ago 10m 44.8M 4273M 19.0.0-3450-g9a047fe7 ee4d84620249 230339d13b11 2024-05-03T14:48:43.214 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1 smithi090 running (10m) 2m ago 10m 47.0M 4272M 19.0.0-3450-g9a047fe7 ee4d84620249 0da45666f424 2024-05-03T14:48:43.214 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2 smithi149 running (9m) 2m ago 9m 60.6M 1454M 19.0.0-3450-g9a047fe7 ee4d84620249 e1af27f902a9 2024-05-03T14:48:43.214 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3 smithi060 running (9m) 2m ago 9m 43.0M 4273M 19.0.0-3450-g9a047fe7 ee4d84620249 c18a7a436901 2024-05-03T14:48:43.214 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4 smithi090 running (8m) 2m ago 8m 54.6M 4272M 19.0.0-3450-g9a047fe7 ee4d84620249 212c59b75ed6 2024-05-03T14:48:43.214 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5 smithi149 running (8m) 2m ago 8m 55.4M 1454M 19.0.0-3450-g9a047fe7 ee4d84620249 cfb4e95629f9 2024-05-03T14:48:43.214 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6 smithi060 running (7m) 2m ago 7m 55.4M 4273M 19.0.0-3450-g9a047fe7 ee4d84620249 61c3db2ab9f3 2024-05-03T14:48:43.214 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7 smithi090 running (7m) 2m ago 7m 47.4M 4272M 19.0.0-3450-g9a047fe7 ee4d84620249 9d22c76a3a2e 2024-05-03T14:48:43.215 INFO:teuthology.orchestra.run.smithi060.stdout:osd.8 smithi149 running (7m) 2m ago 7m 55.7M 1454M 19.0.0-3450-g9a047fe7 ee4d84620249 cb0438d840fb 2024-05-03T14:48:43.215 INFO:teuthology.orchestra.run.smithi060.stdout:osd.9 smithi060 running (6m) 2m ago 6m 41.9M 4273M 19.0.0-3450-g9a047fe7 ee4d84620249 c4c65d055edd 2024-05-03T14:48:43.215 INFO:teuthology.orchestra.run.smithi060.stdout:osd.10 smithi090 running (6m) 2m ago 6m 46.6M 4272M 19.0.0-3450-g9a047fe7 ee4d84620249 95eea6266520 2024-05-03T14:48:43.215 INFO:teuthology.orchestra.run.smithi060.stdout:osd.11 smithi149 running (5m) 2m ago 5m 41.0M 1454M 19.0.0-3450-g9a047fe7 ee4d84620249 fe8dc32ddc6e 2024-05-03T14:48:43.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:43.213+0000 7f6ed3e00700 1 -- 172.21.15.60:0/1420198133 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f6eb40b2910 msgr2=0x7f6eb40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:43.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:43.213+0000 7f6ed3e00700 1 --2- 172.21.15.60:0/1420198133 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f6eb40b2910 0x7f6eb40b4dd0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6eb800b2e0 tx=0x7f6eb800b8c0 comp rx=0 tx=0).stop 2024-05-03T14:48:43.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:43.213+0000 7f6ed3e00700 1 -- 172.21.15.60:0/1420198133 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ecc075e80 msgr2=0x7f6ecc19b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:43.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:43.213+0000 7f6ed3e00700 1 --2- 172.21.15.60:0/1420198133 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ecc075e80 0x7f6ecc19b9f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6ec000e9f0 tx=0x7f6ec000ed00 comp rx=0 tx=0).stop 2024-05-03T14:48:43.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:43.214+0000 7f6ed3e00700 1 -- 172.21.15.60:0/1420198133 shutdown_connections 2024-05-03T14:48:43.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:43.214+0000 7f6ed3e00700 1 --2- 172.21.15.60:0/1420198133 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f6eb40b2910 0x7f6eb40b4dd0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:43.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:43.214+0000 7f6ed3e00700 1 --2- 172.21.15.60:0/1420198133 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6ecc10b400 0x7f6ecc1a2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:43.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:43.214+0000 7f6ed3e00700 1 --2- 172.21.15.60:0/1420198133 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ecc075e80 0x7f6ecc19b9f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:43.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:43.214+0000 7f6ed3e00700 1 --2- 172.21.15.60:0/1420198133 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ecc077450 0x7f6ecc19bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:43.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:43.214+0000 7f6ed3e00700 1 -- 172.21.15.60:0/1420198133 >> 172.21.15.60:0/1420198133 conn(0x7f6ecc0feaa0 msgr2=0x7f6ecc107350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:48:43.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:43.214+0000 7f6ed3e00700 1 -- 172.21.15.60:0/1420198133 shutdown_connections 2024-05-03T14:48:43.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:43.214+0000 7f6ed3e00700 1 -- 172.21.15.60:0/1420198133 wait complete. 2024-05-03T14:48:44.035 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:48:43 smithi060 ceph-mon[28873]: pgmap v458: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-03T14:48:44.108 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-03T14:48:44.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:48:43 smithi090 ceph-mon[33568]: pgmap v458: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-03T14:48:44.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:48:43 smithi149 ceph-mon[33332]: pgmap v458: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-03T14:48:44.573 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:48:45.023 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:48:44 smithi060 ceph-mon[28873]: from='client.25040 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:48:45.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:48:44 smithi090 ceph-mon[33568]: from='client.25040 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:48:45.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:48:44 smithi149 ceph-mon[33332]: from='client.25040 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:48:45.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:48:45 smithi060 ceph-mon[28873]: pgmap v459: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-03T14:48:46.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:48:45 smithi090 ceph-mon[33568]: pgmap v459: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-03T14:48:46.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:48:45 smithi149 ceph-mon[33332]: pgmap v459: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-03T14:48:46.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:46.835+0000 7f93bf200700 1 -- 172.21.15.60:0/3293882397 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93b8103f90 msgr2=0x7f93b8106380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:46.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:46.835+0000 7f93bf200700 1 --2- 172.21.15.60:0/3293882397 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93b8103f90 0x7f93b8106380 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f93a80098f0 tx=0x7f93a8009c00 comp rx=0 tx=0).stop 2024-05-03T14:48:46.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:46.835+0000 7f93bf200700 1 -- 172.21.15.60:0/3293882397 shutdown_connections 2024-05-03T14:48:46.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:46.835+0000 7f93bf200700 1 --2- 172.21.15.60:0/3293882397 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f93b81091f0 0x7f93b810b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:46.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:46.835+0000 7f93bf200700 1 --2- 172.21.15.60:0/3293882397 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93b8103f90 0x7f93b8106380 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:46.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:46.835+0000 7f93bf200700 1 --2- 172.21.15.60:0/3293882397 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93b81068c0 0x7f93b8108cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:46.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:46.835+0000 7f93bf200700 1 -- 172.21.15.60:0/3293882397 >> 172.21.15.60:0/3293882397 conn(0x7f93b80fd9a0 msgr2=0x7f93b80ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:48:46.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:46.835+0000 7f93bf200700 1 -- 172.21.15.60:0/3293882397 shutdown_connections 2024-05-03T14:48:46.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:46.835+0000 7f93bf200700 1 -- 172.21.15.60:0/3293882397 wait complete. 2024-05-03T14:48:46.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:46.836+0000 7f93bf200700 1 Processor -- start 2024-05-03T14:48:46.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:46.836+0000 7f93bf200700 1 -- start start 2024-05-03T14:48:46.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:46.837+0000 7f93bf200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f93b8103f90 0x7f93b819bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:46.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:46.837+0000 7f93bf200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93b81068c0 0x7f93b819c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:46.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:46.837+0000 7f93bf200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93b81091f0 0x7f93b81a3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:46.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:46.837+0000 7f93bf200700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f93b810f260 con 0x7f93b81091f0 2024-05-03T14:48:46.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:46.837+0000 7f93bf200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f93b810f0e0 con 0x7f93b8103f90 2024-05-03T14:48:46.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:46.837+0000 7f93bf200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f93b810f3e0 con 0x7f93b81068c0 2024-05-03T14:48:46.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:46.837+0000 7f93b7400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93b81068c0 0x7f93b819c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:46.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:46.837+0000 7f93b7400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93b81068c0 0x7f93b819c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.60:39524/0 (socket says 172.21.15.60:39524) 2024-05-03T14:48:46.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:46.837+0000 7f93bd200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93b81091f0 0x7f93b81a3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:46.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:46.837+0000 7f93b7400700 1 -- 172.21.15.60:0/1268985424 learned_addr learned my addr 172.21.15.60:0/1268985424 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:48:46.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:46.837+0000 7f93b7e00700 1 --2- 172.21.15.60:0/1268985424 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f93b8103f90 0x7f93b819bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:46.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:46.838+0000 7f93b7400700 1 -- 172.21.15.60:0/1268985424 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f93b8103f90 msgr2=0x7f93b819bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:46.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:46.838+0000 7f93b7400700 1 --2- 172.21.15.60:0/1268985424 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f93b8103f90 0x7f93b819bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:46.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:46.838+0000 7f93b7400700 1 -- 172.21.15.60:0/1268985424 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93b81091f0 msgr2=0x7f93b81a3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:46.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:46.838+0000 7f93b7400700 1 --2- 172.21.15.60:0/1268985424 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93b81091f0 0x7f93b81a3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:46.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:46.838+0000 7f93b7400700 1 -- 172.21.15.60:0/1268985424 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93b81a3720 con 0x7f93b81068c0 2024-05-03T14:48:46.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:46.838+0000 7f93b7400700 1 --2- 172.21.15.60:0/1268985424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93b81068c0 0x7f93b819c000 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f93a400ec40 tx=0x7f93a400ef50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:48:46.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:46.838+0000 7f93b4c00700 1 -- 172.21.15.60:0/1268985424 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a400ccf0 con 0x7f93b81068c0 2024-05-03T14:48:46.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:46.838+0000 7f93b4c00700 1 -- 172.21.15.60:0/1268985424 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f93a400ce90 con 0x7f93b81068c0 2024-05-03T14:48:46.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:46.838+0000 7f93b4c00700 1 -- 172.21.15.60:0/1268985424 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a4010690 con 0x7f93b81068c0 2024-05-03T14:48:46.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:46.839+0000 7f93bf200700 1 -- 172.21.15.60:0/1268985424 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93b81a3a10 con 0x7f93b81068c0 2024-05-03T14:48:46.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:46.839+0000 7f93bf200700 1 -- 172.21.15.60:0/1268985424 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93b81a3f00 con 0x7f93b81068c0 2024-05-03T14:48:46.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:46.840+0000 7f93bf200700 1 -- 172.21.15.60:0/1268985424 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93b806a190 con 0x7f93b81068c0 2024-05-03T14:48:46.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:46.844+0000 7f93b4c00700 1 -- 172.21.15.60:0/1268985424 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f93a4003680 con 0x7f93b81068c0 2024-05-03T14:48:46.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:46.845+0000 7f93b4c00700 1 --2- 172.21.15.60:0/1268985424 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f939c0b2990 0x7f939c0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:46.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:46.846+0000 7f93b4c00700 1 -- 172.21.15.60:0/1268985424 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f93a40e3080 con 0x7f93b81068c0 2024-05-03T14:48:46.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:46.846+0000 7f93b7e00700 1 --2- 172.21.15.60:0/1268985424 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f939c0b2990 0x7f939c0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:46.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:46.847+0000 7f93b4c00700 1 -- 172.21.15.60:0/1268985424 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f93a401d060 con 0x7f93b81068c0 2024-05-03T14:48:46.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:46.847+0000 7f93b7e00700 1 --2- 172.21.15.60:0/1268985424 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f939c0b2990 0x7f939c0b4e50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f93a8009810 tx=0x7f93a8019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:48:47.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:47.103+0000 7f93bf200700 1 -- 172.21.15.60:0/1268985424 --> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f93b80638e0 con 0x7f939c0b2990 2024-05-03T14:48:47.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:47.121+0000 7f93b4c00700 1 -- 172.21.15.60:0/1268985424 <== mgr.14152 v2:172.21.15.60:6800/3066142956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f93b80638e0 con 0x7f939c0b2990 2024-05-03T14:48:47.123 INFO:teuthology.orchestra.run.smithi060.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-03T14:48:47.123 INFO:teuthology.orchestra.run.smithi060.stdout:mds.all 12/12 2m ago 5m smithi060=a;smithi060=d;smithi060=g;smithi060=j;smithi090=b;smithi090=e;smithi090=h;smithi090=k;smithi149=c;smithi149=f;smithi149=i;smithi149=l;count:12 2024-05-03T14:48:47.123 INFO:teuthology.orchestra.run.smithi060.stdout:mgr 3/3 2m ago 11m smithi060=x;smithi090=y;smithi149=z;count:3 2024-05-03T14:48:47.123 INFO:teuthology.orchestra.run.smithi060.stdout:mon 3/3 2m ago 11m smithi060:172.21.15.60=a;smithi090:172.21.15.90=b;smithi149:172.21.15.149=c;count:3 2024-05-03T14:48:47.123 INFO:teuthology.orchestra.run.smithi060.stdout:osd 12 2m ago - 2024-05-03T14:48:47.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:47.125+0000 7f93bf200700 1 -- 172.21.15.60:0/1268985424 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f939c0b2990 msgr2=0x7f939c0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:47.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:47.125+0000 7f93bf200700 1 --2- 172.21.15.60:0/1268985424 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f939c0b2990 0x7f939c0b4e50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f93a8009810 tx=0x7f93a8019040 comp rx=0 tx=0).stop 2024-05-03T14:48:47.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:47.125+0000 7f93bf200700 1 -- 172.21.15.60:0/1268985424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93b81068c0 msgr2=0x7f93b819c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:47.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:47.125+0000 7f93bf200700 1 --2- 172.21.15.60:0/1268985424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93b81068c0 0x7f93b819c000 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f93a400ec40 tx=0x7f93a400ef50 comp rx=0 tx=0).stop 2024-05-03T14:48:47.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:47.125+0000 7f93bf200700 1 -- 172.21.15.60:0/1268985424 shutdown_connections 2024-05-03T14:48:47.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:47.125+0000 7f93bf200700 1 --2- 172.21.15.60:0/1268985424 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f939c0b2990 0x7f939c0b4e50 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:47.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:47.125+0000 7f93bf200700 1 --2- 172.21.15.60:0/1268985424 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93b81091f0 0x7f93b81a3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:47.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:47.125+0000 7f93bf200700 1 --2- 172.21.15.60:0/1268985424 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f93b8103f90 0x7f93b819bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:47.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:47.125+0000 7f93bf200700 1 --2- 172.21.15.60:0/1268985424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93b81068c0 0x7f93b819c000 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:47.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:47.125+0000 7f93bf200700 1 -- 172.21.15.60:0/1268985424 >> 172.21.15.60:0/1268985424 conn(0x7f93b80fd9a0 msgr2=0x7f93b8103750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:48:47.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:47.125+0000 7f93bf200700 1 -- 172.21.15.60:0/1268985424 shutdown_connections 2024-05-03T14:48:47.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:47.125+0000 7f93bf200700 1 -- 172.21.15.60:0/1268985424 wait complete. 2024-05-03T14:48:48.030 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:48:47 smithi060 ceph-mon[28873]: pgmap v460: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:48:48.034 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-03T14:48:48.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:48:47 smithi090 ceph-mon[33568]: pgmap v460: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:48:48.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:48:47 smithi149 ceph-mon[33332]: pgmap v460: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:48:48.510 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:48:48.946 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:48:48 smithi060 ceph-mon[28873]: from='client.25054 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:48:49.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:48:48 smithi090 ceph-mon[33568]: from='client.25054 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:48:49.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:48:48 smithi149 ceph-mon[33332]: from='client.25054 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:48:50.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:48:49 smithi060 ceph-mon[28873]: pgmap v461: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:48:50.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:48:49 smithi090 ceph-mon[33568]: pgmap v461: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:48:50.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:48:49 smithi149 ceph-mon[33332]: pgmap v461: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:48:50.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:50.699+0000 7f21dae00700 1 -- 172.21.15.60:0/3167261139 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f21d410fa10 msgr2=0x7f21d4111e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:50.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:50.699+0000 7f21dae00700 1 --2- 172.21.15.60:0/3167261139 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f21d410fa10 0x7f21d4111e00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f21cc009f90 tx=0x7f21cc00b330 comp rx=0 tx=0).stop 2024-05-03T14:48:50.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:50.699+0000 7f21dae00700 1 -- 172.21.15.60:0/3167261139 shutdown_connections 2024-05-03T14:48:50.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:50.699+0000 7f21dae00700 1 --2- 172.21.15.60:0/3167261139 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f21d410fa10 0x7f21d4111e00 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:50.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:50.699+0000 7f21dae00700 1 --2- 172.21.15.60:0/3167261139 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21d40774d0 0x7f21d4075930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:50.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:50.699+0000 7f21dae00700 1 --2- 172.21.15.60:0/3167261139 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21d4075f00 0x7f21d4076380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:50.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:50.699+0000 7f21dae00700 1 -- 172.21.15.60:0/3167261139 >> 172.21.15.60:0/3167261139 conn(0x7f21d40fe9d0 msgr2=0x7f21d4100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:48:50.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:50.699+0000 7f21dae00700 1 -- 172.21.15.60:0/3167261139 shutdown_connections 2024-05-03T14:48:50.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:50.699+0000 7f21dae00700 1 -- 172.21.15.60:0/3167261139 wait complete. 2024-05-03T14:48:50.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:50.700+0000 7f21dae00700 1 Processor -- start 2024-05-03T14:48:50.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:50.701+0000 7f21dae00700 1 -- start start 2024-05-03T14:48:50.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:50.701+0000 7f21dae00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21d4075f00 0x7f21d419fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:50.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:50.701+0000 7f21dae00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21d40774d0 0x7f21d41a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:50.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:50.701+0000 7f21dae00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f21d410fa10 0x7f21d41a7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:50.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:50.701+0000 7f21dae00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f21d41157a0 con 0x7f21d40774d0 2024-05-03T14:48:50.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:50.701+0000 7f21dae00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f21d4115620 con 0x7f21d410fa10 2024-05-03T14:48:50.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:50.701+0000 7f21dae00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f21d4115920 con 0x7f21d4075f00 2024-05-03T14:48:50.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:50.701+0000 7f21d9000700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21d40774d0 0x7f21d41a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:50.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:50.701+0000 7f21d9a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21d4075f00 0x7f21d419fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:50.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:50.701+0000 7f21da400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f21d410fa10 0x7f21d41a7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:50.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:50.701+0000 7f21d9000700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21d40774d0 0x7f21d41a01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45224/0 (socket says 172.21.15.60:45224) 2024-05-03T14:48:50.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:50.701+0000 7f21d9000700 1 -- 172.21.15.60:0/3994916857 learned_addr learned my addr 172.21.15.60:0/3994916857 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:48:50.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:50.702+0000 7f21d9a00700 1 -- 172.21.15.60:0/3994916857 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f21d410fa10 msgr2=0x7f21d41a7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:50.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:50.702+0000 7f21d9a00700 1 --2- 172.21.15.60:0/3994916857 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f21d410fa10 0x7f21d41a7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:50.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:50.702+0000 7f21d9a00700 1 -- 172.21.15.60:0/3994916857 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21d40774d0 msgr2=0x7f21d41a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:50.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:50.702+0000 7f21d9a00700 1 --2- 172.21.15.60:0/3994916857 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21d40774d0 0x7f21d41a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:50.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:50.702+0000 7f21d9a00700 1 -- 172.21.15.60:0/3994916857 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21d41a78e0 con 0x7f21d4075f00 2024-05-03T14:48:50.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:50.702+0000 7f21d9a00700 1 --2- 172.21.15.60:0/3994916857 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21d4075f00 0x7f21d419fc60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f21c400e9c0 tx=0x7f21c400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:48:50.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:50.703+0000 7f21d2000700 1 -- 172.21.15.60:0/3994916857 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c400c9c0 con 0x7f21d4075f00 2024-05-03T14:48:50.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:50.703+0000 7f21d2000700 1 -- 172.21.15.60:0/3994916857 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f21c400cb60 con 0x7f21d4075f00 2024-05-03T14:48:50.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:50.703+0000 7f21dae00700 1 -- 172.21.15.60:0/3994916857 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21d41a7bd0 con 0x7f21d4075f00 2024-05-03T14:48:50.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:50.703+0000 7f21d2000700 1 -- 172.21.15.60:0/3994916857 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c4018710 con 0x7f21d4075f00 2024-05-03T14:48:50.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:50.703+0000 7f21dae00700 1 -- 172.21.15.60:0/3994916857 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21d41a8110 con 0x7f21d4075f00 2024-05-03T14:48:50.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:50.704+0000 7f21dae00700 1 -- 172.21.15.60:0/3994916857 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21d406a190 con 0x7f21d4075f00 2024-05-03T14:48:50.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:50.706+0000 7f21d2000700 1 -- 172.21.15.60:0/3994916857 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f21c40188b0 con 0x7f21d4075f00 2024-05-03T14:48:50.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:50.708+0000 7f21d2000700 1 --2- 172.21.15.60:0/3994916857 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f21bc0b2910 0x7f21bc0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:50.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:50.708+0000 7f21d2000700 1 -- 172.21.15.60:0/3994916857 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f21c40e2880 con 0x7f21d4075f00 2024-05-03T14:48:50.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:50.708+0000 7f21d9000700 1 --2- 172.21.15.60:0/3994916857 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f21bc0b2910 0x7f21bc0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:50.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:50.709+0000 7f21d9000700 1 --2- 172.21.15.60:0/3994916857 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f21bc0b2910 0x7f21bc0b4dd0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f21c8009750 tx=0x7f21c8009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:48:50.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:50.712+0000 7f21d2000700 1 -- 172.21.15.60:0/3994916857 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f21c40aa060 con 0x7f21d4075f00 2024-05-03T14:48:50.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:50.955+0000 7f21dae00700 1 -- 172.21.15.60:0/3994916857 --> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f21d40639a0 con 0x7f21bc0b2910 2024-05-03T14:48:50.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:50.958+0000 7f21d2000700 1 -- 172.21.15.60:0/3994916857 <== mgr.14152 v2:172.21.15.60:6800/3066142956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f21d40639a0 con 0x7f21bc0b2910 2024-05-03T14:48:50.960 INFO:teuthology.orchestra.run.smithi060.stdout:HOST ADDR LABELS STATUS 2024-05-03T14:48:50.960 INFO:teuthology.orchestra.run.smithi060.stdout:smithi060 172.21.15.60 2024-05-03T14:48:50.960 INFO:teuthology.orchestra.run.smithi060.stdout:smithi090 172.21.15.90 2024-05-03T14:48:50.960 INFO:teuthology.orchestra.run.smithi060.stdout:smithi149 172.21.15.149 2024-05-03T14:48:50.960 INFO:teuthology.orchestra.run.smithi060.stdout:3 hosts in cluster 2024-05-03T14:48:50.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:50.961+0000 7f21dae00700 1 -- 172.21.15.60:0/3994916857 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f21bc0b2910 msgr2=0x7f21bc0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:50.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:50.961+0000 7f21dae00700 1 --2- 172.21.15.60:0/3994916857 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f21bc0b2910 0x7f21bc0b4dd0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f21c8009750 tx=0x7f21c8009310 comp rx=0 tx=0).stop 2024-05-03T14:48:50.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:50.961+0000 7f21dae00700 1 -- 172.21.15.60:0/3994916857 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21d4075f00 msgr2=0x7f21d419fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:50.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:50.961+0000 7f21dae00700 1 --2- 172.21.15.60:0/3994916857 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21d4075f00 0x7f21d419fc60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f21c400e9c0 tx=0x7f21c400ecd0 comp rx=0 tx=0).stop 2024-05-03T14:48:50.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:50.961+0000 7f21dae00700 1 -- 172.21.15.60:0/3994916857 shutdown_connections 2024-05-03T14:48:50.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:50.961+0000 7f21dae00700 1 --2- 172.21.15.60:0/3994916857 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f21bc0b2910 0x7f21bc0b4dd0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:50.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:50.961+0000 7f21dae00700 1 --2- 172.21.15.60:0/3994916857 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f21d410fa10 0x7f21d41a7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:50.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:50.961+0000 7f21dae00700 1 --2- 172.21.15.60:0/3994916857 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21d4075f00 0x7f21d419fc60 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:50.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:50.961+0000 7f21dae00700 1 --2- 172.21.15.60:0/3994916857 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21d40774d0 0x7f21d41a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:50.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:50.961+0000 7f21dae00700 1 -- 172.21.15.60:0/3994916857 >> 172.21.15.60:0/3994916857 conn(0x7f21d40fe9d0 msgr2=0x7f21d4102db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:48:50.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:50.961+0000 7f21dae00700 1 -- 172.21.15.60:0/3994916857 shutdown_connections 2024-05-03T14:48:50.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:50.961+0000 7f21dae00700 1 -- 172.21.15.60:0/3994916857 wait complete. 2024-05-03T14:48:51.865 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-03T14:48:52.045 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:48:51 smithi060 ceph-mon[28873]: pgmap v462: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:48:52.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:48:51 smithi090 ceph-mon[33568]: pgmap v462: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:48:52.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:48:51 smithi149 ceph-mon[33332]: pgmap v462: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:48:52.328 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:48:53.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:48:52 smithi060 ceph-mon[28873]: from='client.25060 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:48:53.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:48:52 smithi090 ceph-mon[33568]: from='client.25060 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:48:53.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:48:52 smithi149 ceph-mon[33332]: from='client.25060 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:48:54.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:48:53 smithi060 ceph-mon[28873]: pgmap v463: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T14:48:54.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:48:53 smithi090 ceph-mon[33568]: pgmap v463: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T14:48:54.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:48:53 smithi149 ceph-mon[33332]: pgmap v463: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T14:48:54.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:54.539+0000 7f98bbe00700 1 -- 172.21.15.60:0/511893364 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98bc109120 msgr2=0x7f98bc10b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:54.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:54.539+0000 7f98bbe00700 1 --2- 172.21.15.60:0/511893364 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98bc109120 0x7f98bc10b590 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f98b40098f0 tx=0x7f98b4009c00 comp rx=0 tx=0).stop 2024-05-03T14:48:54.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:54.539+0000 7f98bbe00700 1 -- 172.21.15.60:0/511893364 shutdown_connections 2024-05-03T14:48:54.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:54.539+0000 7f98bbe00700 1 --2- 172.21.15.60:0/511893364 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98bc109120 0x7f98bc10b590 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:54.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:54.539+0000 7f98bbe00700 1 --2- 172.21.15.60:0/511893364 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f98bc103ec0 0x7f98bc1062b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:54.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:54.539+0000 7f98bbe00700 1 --2- 172.21.15.60:0/511893364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f98bc1067f0 0x7f98bc108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:54.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:54.539+0000 7f98bbe00700 1 -- 172.21.15.60:0/511893364 >> 172.21.15.60:0/511893364 conn(0x7f98bc0fd8d0 msgr2=0x7f98bc0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:48:54.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:54.539+0000 7f98bbe00700 1 -- 172.21.15.60:0/511893364 shutdown_connections 2024-05-03T14:48:54.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:54.539+0000 7f98bbe00700 1 -- 172.21.15.60:0/511893364 wait complete. 2024-05-03T14:48:54.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:54.540+0000 7f98bbe00700 1 Processor -- start 2024-05-03T14:48:54.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:54.540+0000 7f98bbe00700 1 -- start start 2024-05-03T14:48:54.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:54.541+0000 7f98bbe00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98bc103ec0 0x7f98bc19b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:54.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:54.541+0000 7f98bbe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f98bc1067f0 0x7f98bc19be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:54.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:54.541+0000 7f98bbe00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f98bc109120 0x7f98bc1a2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:54.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:54.541+0000 7f98bbe00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc10f190 con 0x7f98bc103ec0 2024-05-03T14:48:54.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:54.541+0000 7f98bbe00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc10f010 con 0x7f98bc1067f0 2024-05-03T14:48:54.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:54.541+0000 7f98bbe00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc10f310 con 0x7f98bc109120 2024-05-03T14:48:54.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:54.541+0000 7f98baa00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98bc103ec0 0x7f98bc19b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:54.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:54.541+0000 7f98b2000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f98bc1067f0 0x7f98bc19be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:54.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:54.541+0000 7f98bb400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f98bc109120 0x7f98bc1a2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:54.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:54.541+0000 7f98baa00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98bc103ec0 0x7f98bc19b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41810/0 (socket says 172.21.15.60:41810) 2024-05-03T14:48:54.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:54.541+0000 7f98b2000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f98bc1067f0 0x7f98bc19be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.60:37576/0 (socket says 172.21.15.60:37576) 2024-05-03T14:48:54.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:54.541+0000 7f98baa00700 1 -- 172.21.15.60:0/3604908218 learned_addr learned my addr 172.21.15.60:0/3604908218 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:48:54.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:54.542+0000 7f98b2000700 1 -- 172.21.15.60:0/3604908218 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f98bc109120 msgr2=0x7f98bc1a2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:54.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:54.542+0000 7f98b2000700 1 --2- 172.21.15.60:0/3604908218 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f98bc109120 0x7f98bc1a2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:54.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:54.542+0000 7f98b2000700 1 -- 172.21.15.60:0/3604908218 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98bc103ec0 msgr2=0x7f98bc19b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:54.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:54.542+0000 7f98b2000700 1 --2- 172.21.15.60:0/3604908218 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98bc103ec0 0x7f98bc19b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:54.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:54.542+0000 7f98b2000700 1 -- 172.21.15.60:0/3604908218 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98bc1a3550 con 0x7f98bc1067f0 2024-05-03T14:48:54.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:54.542+0000 7f98b2000700 1 --2- 172.21.15.60:0/3604908218 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f98bc1067f0 0x7f98bc19be30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f98a400e9c0 tx=0x7f98a400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:48:54.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:54.542+0000 7f98c0a00700 1 -- 172.21.15.60:0/3604908218 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a4004220 con 0x7f98bc1067f0 2024-05-03T14:48:54.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:54.543+0000 7f98bbe00700 1 -- 172.21.15.60:0/3604908218 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98bc1a3840 con 0x7f98bc1067f0 2024-05-03T14:48:54.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:54.543+0000 7f98c0a00700 1 -- 172.21.15.60:0/3604908218 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f98a4004d30 con 0x7f98bc1067f0 2024-05-03T14:48:54.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:54.543+0000 7f98bbe00700 1 -- 172.21.15.60:0/3604908218 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98bc1a3d30 con 0x7f98bc1067f0 2024-05-03T14:48:54.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:54.543+0000 7f98c0a00700 1 -- 172.21.15.60:0/3604908218 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a401c070 con 0x7f98bc1067f0 2024-05-03T14:48:54.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:54.544+0000 7f98bbe00700 1 -- 172.21.15.60:0/3604908218 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98bc06a190 con 0x7f98bc1067f0 2024-05-03T14:48:54.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:54.546+0000 7f98c0a00700 1 -- 172.21.15.60:0/3604908218 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f98a4004750 con 0x7f98bc1067f0 2024-05-03T14:48:54.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:54.548+0000 7f98c0a00700 1 --2- 172.21.15.60:0/3604908218 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f98a00b2910 0x7f98a00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:54.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:54.548+0000 7f98c0a00700 1 -- 172.21.15.60:0/3604908218 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f98a40e2850 con 0x7f98bc1067f0 2024-05-03T14:48:54.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:54.548+0000 7f98baa00700 1 --2- 172.21.15.60:0/3604908218 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f98a00b2910 0x7f98a00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:54.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:54.549+0000 7f98baa00700 1 --2- 172.21.15.60:0/3604908218 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f98a00b2910 0x7f98a00b4dd0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f98ac009c10 tx=0x7f98ac00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:48:54.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:54.550+0000 7f98c0a00700 1 -- 172.21.15.60:0/3604908218 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f98a40aa0c0 con 0x7f98bc1067f0 2024-05-03T14:48:54.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:54.794+0000 7f98bbe00700 1 -- 172.21.15.60:0/3604908218 --> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f98bc0638e0 con 0x7f98a00b2910 2024-05-03T14:48:54.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:54.797+0000 7f98c0a00700 1 -- 172.21.15.60:0/3604908218 <== mgr.14152 v2:172.21.15.60:6800/3066142956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f98bc0638e0 con 0x7f98a00b2910 2024-05-03T14:48:54.799 INFO:teuthology.orchestra.run.smithi060.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-03T14:48:54.799 INFO:teuthology.orchestra.run.smithi060.stdout:smithi060 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7476008A480DGN 447G No 6m ago Has a FileSystem, LVM detected 2024-05-03T14:48:54.799 INFO:teuthology.orchestra.run.smithi060.stdout:smithi090 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400UK400BGN 372G No 6m ago Has a FileSystem, LVM detected 2024-05-03T14:48:54.799 INFO:teuthology.orchestra.run.smithi060.stdout:smithi149 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300CK400BGN 372G No 5m ago Has a FileSystem, LVM detected 2024-05-03T14:48:54.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:54.800+0000 7f98bbe00700 1 -- 172.21.15.60:0/3604908218 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f98a00b2910 msgr2=0x7f98a00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:54.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:54.800+0000 7f98bbe00700 1 --2- 172.21.15.60:0/3604908218 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f98a00b2910 0x7f98a00b4dd0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f98ac009c10 tx=0x7f98ac00b040 comp rx=0 tx=0).stop 2024-05-03T14:48:54.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:54.800+0000 7f98bbe00700 1 -- 172.21.15.60:0/3604908218 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f98bc1067f0 msgr2=0x7f98bc19be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:54.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:54.800+0000 7f98bbe00700 1 --2- 172.21.15.60:0/3604908218 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f98bc1067f0 0x7f98bc19be30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f98a400e9c0 tx=0x7f98a400ecd0 comp rx=0 tx=0).stop 2024-05-03T14:48:54.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:54.800+0000 7f98bbe00700 1 -- 172.21.15.60:0/3604908218 shutdown_connections 2024-05-03T14:48:54.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:54.800+0000 7f98bbe00700 1 --2- 172.21.15.60:0/3604908218 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f98a00b2910 0x7f98a00b4dd0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:54.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:54.800+0000 7f98bbe00700 1 --2- 172.21.15.60:0/3604908218 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f98bc109120 0x7f98bc1a2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:54.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:54.800+0000 7f98bbe00700 1 --2- 172.21.15.60:0/3604908218 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98bc103ec0 0x7f98bc19b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:54.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:54.800+0000 7f98bbe00700 1 --2- 172.21.15.60:0/3604908218 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f98bc1067f0 0x7f98bc19be30 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:54.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:54.800+0000 7f98bbe00700 1 -- 172.21.15.60:0/3604908218 >> 172.21.15.60:0/3604908218 conn(0x7f98bc0fd8d0 msgr2=0x7f98bc103500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:48:54.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:54.800+0000 7f98bbe00700 1 -- 172.21.15.60:0/3604908218 shutdown_connections 2024-05-03T14:48:54.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:54.800+0000 7f98bbe00700 1 -- 172.21.15.60:0/3604908218 wait complete. 2024-05-03T14:48:55.757 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-03T14:48:55.766 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi060.front.sepia.ceph.com 2024-05-03T14:48:55.766 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-03T14:48:56.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:48:55 smithi060 ceph-mon[28873]: pgmap v464: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:48:56.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:48:55 smithi060 ceph-mon[28873]: from='client.25058 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:48:56.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:48:55 smithi090 ceph-mon[33568]: pgmap v464: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:48:56.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:48:55 smithi090 ceph-mon[33568]: from='client.25058 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:48:56.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:48:55 smithi149 ceph-mon[33332]: pgmap v464: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:48:56.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:48:55 smithi149 ceph-mon[33332]: from='client.25058 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:48:56.238 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:48:58.002 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:48:57 smithi060 ceph-mon[28873]: pgmap v465: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:48:58.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:48:57 smithi090 ceph-mon[33568]: pgmap v465: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:48:58.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:48:57 smithi149 ceph-mon[33332]: pgmap v465: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:48:58.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:58.493+0000 7f27a6800700 1 -- 172.21.15.60:0/2695596201 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27a0105190 msgr2=0x7f27a010ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:58.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:58.493+0000 7f27a6800700 1 --2- 172.21.15.60:0/2695596201 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27a0105190 0x7f27a010ba40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f279800b0a0 tx=0x7f279800b3b0 comp rx=0 tx=0).stop 2024-05-03T14:48:58.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:58.494+0000 7f27a6800700 1 -- 172.21.15.60:0/2695596201 shutdown_connections 2024-05-03T14:48:58.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:58.494+0000 7f27a6800700 1 --2- 172.21.15.60:0/2695596201 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27a0105190 0x7f27a010ba40 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:58.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:58.494+0000 7f27a6800700 1 --2- 172.21.15.60:0/2695596201 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27a0101810 0x7f27a0101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:58.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:58.494+0000 7f27a6800700 1 --2- 172.21.15.60:0/2695596201 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f27a01047f0 0x7f27a0104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:58.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:58.494+0000 7f27a6800700 1 -- 172.21.15.60:0/2695596201 >> 172.21.15.60:0/2695596201 conn(0x7f27a00fd190 msgr2=0x7f27a00ff5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:48:58.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:58.494+0000 7f27a6800700 1 -- 172.21.15.60:0/2695596201 shutdown_connections 2024-05-03T14:48:58.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:58.494+0000 7f27a6800700 1 -- 172.21.15.60:0/2695596201 wait complete. 2024-05-03T14:48:58.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:58.495+0000 7f27a6800700 1 Processor -- start 2024-05-03T14:48:58.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:58.495+0000 7f27a6800700 1 -- start start 2024-05-03T14:48:58.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:58.496+0000 7f27a6800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27a0101810 0x7f27a019b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:58.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:58.496+0000 7f27a6800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27a01047f0 0x7f27a019bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:58.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:58.496+0000 7f27a6800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f27a0105190 0x7f27a01a2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:58.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:58.496+0000 7f27a6800700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f27a010f2a0 con 0x7f27a0101810 2024-05-03T14:48:58.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:58.496+0000 7f27a6800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f27a010f120 con 0x7f27a01047f0 2024-05-03T14:48:58.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:58.496+0000 7f27a6800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f27a010f420 con 0x7f27a0105190 2024-05-03T14:48:58.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:58.496+0000 7f279f400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27a0101810 0x7f27a019b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:58.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:58.496+0000 7f279f400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27a0101810 0x7f27a019b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41840/0 (socket says 172.21.15.60:41840) 2024-05-03T14:48:58.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:58.496+0000 7f279f400700 1 -- 172.21.15.60:0/416611538 learned_addr learned my addr 172.21.15.60:0/416611538 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:48:58.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:58.496+0000 7f279ea00700 1 --2- 172.21.15.60:0/416611538 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27a01047f0 0x7f27a019bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:58.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:58.497+0000 7f279fe00700 1 --2- 172.21.15.60:0/416611538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f27a0105190 0x7f27a01a2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:58.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:58.497+0000 7f279ea00700 1 -- 172.21.15.60:0/416611538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f27a0105190 msgr2=0x7f27a01a2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:58.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:58.497+0000 7f279ea00700 1 --2- 172.21.15.60:0/416611538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f27a0105190 0x7f27a01a2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:58.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:58.497+0000 7f279ea00700 1 -- 172.21.15.60:0/416611538 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27a0101810 msgr2=0x7f27a019b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:58.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:58.497+0000 7f279ea00700 1 --2- 172.21.15.60:0/416611538 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27a0101810 0x7f27a019b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:58.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:58.497+0000 7f279ea00700 1 -- 172.21.15.60:0/416611538 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27a01a3600 con 0x7f27a01047f0 2024-05-03T14:48:58.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:58.497+0000 7f279f400700 1 --2- 172.21.15.60:0/416611538 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27a0101810 0x7f27a019b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:48:58.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:58.497+0000 7f279ea00700 1 --2- 172.21.15.60:0/416611538 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27a01047f0 0x7f27a019bee0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f278c00e9f0 tx=0x7f278c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:48:58.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:58.497+0000 7f2797e00700 1 -- 172.21.15.60:0/416611538 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278c00c9c0 con 0x7f27a01047f0 2024-05-03T14:48:58.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:58.498+0000 7f27a6800700 1 -- 172.21.15.60:0/416611538 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27a01a38f0 con 0x7f27a01047f0 2024-05-03T14:48:58.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:58.498+0000 7f2797e00700 1 -- 172.21.15.60:0/416611538 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f278c00cb60 con 0x7f27a01047f0 2024-05-03T14:48:58.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:58.498+0000 7f2797e00700 1 -- 172.21.15.60:0/416611538 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278c018710 con 0x7f27a01047f0 2024-05-03T14:48:58.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:58.498+0000 7f27a6800700 1 -- 172.21.15.60:0/416611538 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27a01a3e80 con 0x7f27a01047f0 2024-05-03T14:48:58.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:58.499+0000 7f27a6800700 1 -- 172.21.15.60:0/416611538 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27a006a190 con 0x7f27a01047f0 2024-05-03T14:48:58.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:58.502+0000 7f2797e00700 1 -- 172.21.15.60:0/416611538 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f278c00cd10 con 0x7f27a01047f0 2024-05-03T14:48:58.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:58.504+0000 7f2797e00700 1 --2- 172.21.15.60:0/416611538 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f27840b2990 0x7f27840b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:48:58.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:58.504+0000 7f2797e00700 1 -- 172.21.15.60:0/416611538 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f278c0e2c80 con 0x7f27a01047f0 2024-05-03T14:48:58.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:58.504+0000 7f279f400700 1 --2- 172.21.15.60:0/416611538 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f27840b2990 0x7f27840b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:48:58.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:58.505+0000 7f2797e00700 1 -- 172.21.15.60:0/416611538 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f278c0aa460 con 0x7f27a01047f0 2024-05-03T14:48:58.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:58.505+0000 7f279f400700 1 --2- 172.21.15.60:0/416611538 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f27840b2990 0x7f27840b4e50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2790005950 tx=0x7f2790009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:48:58.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:58.790+0000 7f27a6800700 1 -- 172.21.15.60:0/416611538 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f27a01a4270 con 0x7f27a01047f0 2024-05-03T14:48:58.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:58.790+0000 7f2797e00700 1 -- 172.21.15.60:0/416611538 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) ==== 76+0+3090 (secure 0 0 0) 0x7f278c0a9d60 con 0x7f27a01047f0 2024-05-03T14:48:58.793 INFO:teuthology.orchestra.run.smithi060.stdout:e29 2024-05-03T14:48:58.793 INFO:teuthology.orchestra.run.smithi060.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-03T14:48:58.793 INFO:teuthology.orchestra.run.smithi060.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-03T14:48:58.793 INFO:teuthology.orchestra.run.smithi060.stdout:legacy client fscid: 1 2024-05-03T14:48:58.793 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:48:58.793 INFO:teuthology.orchestra.run.smithi060.stdout:Filesystem 'cephfs' (1) 2024-05-03T14:48:58.793 INFO:teuthology.orchestra.run.smithi060.stdout:fs_name cephfs 2024-05-03T14:48:58.793 INFO:teuthology.orchestra.run.smithi060.stdout:epoch 29 2024-05-03T14:48:58.793 INFO:teuthology.orchestra.run.smithi060.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-03T14:48:58.794 INFO:teuthology.orchestra.run.smithi060.stdout:created 2024-05-03T14:44:34.226052+0000 2024-05-03T14:48:58.794 INFO:teuthology.orchestra.run.smithi060.stdout:modified 2024-05-03T14:46:01.367913+0000 2024-05-03T14:48:58.794 INFO:teuthology.orchestra.run.smithi060.stdout:tableserver 0 2024-05-03T14:48:58.794 INFO:teuthology.orchestra.run.smithi060.stdout:root 0 2024-05-03T14:48:58.794 INFO:teuthology.orchestra.run.smithi060.stdout:session_timeout 300 2024-05-03T14:48:58.794 INFO:teuthology.orchestra.run.smithi060.stdout:session_autoclose 300 2024-05-03T14:48:58.794 INFO:teuthology.orchestra.run.smithi060.stdout:max_file_size 1099511627776 2024-05-03T14:48:58.794 INFO:teuthology.orchestra.run.smithi060.stdout:max_xattr_size 65536 2024-05-03T14:48:58.794 INFO:teuthology.orchestra.run.smithi060.stdout:required_client_features {} 2024-05-03T14:48:58.794 INFO:teuthology.orchestra.run.smithi060.stdout:last_failure 0 2024-05-03T14:48:58.794 INFO:teuthology.orchestra.run.smithi060.stdout:last_failure_osd_epoch 0 2024-05-03T14:48:58.795 INFO:teuthology.orchestra.run.smithi060.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-03T14:48:58.795 INFO:teuthology.orchestra.run.smithi060.stdout:max_mds 5 2024-05-03T14:48:58.795 INFO:teuthology.orchestra.run.smithi060.stdout:in 0,1,2,3,4 2024-05-03T14:48:58.795 INFO:teuthology.orchestra.run.smithi060.stdout:up {0=24511,1=24509,2=24487,3=24475,4=24470} 2024-05-03T14:48:58.795 INFO:teuthology.orchestra.run.smithi060.stdout:failed 2024-05-03T14:48:58.795 INFO:teuthology.orchestra.run.smithi060.stdout:damaged 2024-05-03T14:48:58.795 INFO:teuthology.orchestra.run.smithi060.stdout:stopped 2024-05-03T14:48:58.795 INFO:teuthology.orchestra.run.smithi060.stdout:data_pools [3,4] 2024-05-03T14:48:58.795 INFO:teuthology.orchestra.run.smithi060.stdout:metadata_pool 2 2024-05-03T14:48:58.795 INFO:teuthology.orchestra.run.smithi060.stdout:inline_data disabled 2024-05-03T14:48:58.795 INFO:teuthology.orchestra.run.smithi060.stdout:balancer 2024-05-03T14:48:58.796 INFO:teuthology.orchestra.run.smithi060.stdout:bal_rank_mask -1 2024-05-03T14:48:58.796 INFO:teuthology.orchestra.run.smithi060.stdout:standby_count_wanted 0 2024-05-03T14:48:58.796 INFO:teuthology.orchestra.run.smithi060.stdout:qdb_cluster leader: 24511 members: 24470,24475,24487,24509,24511 2024-05-03T14:48:58.796 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.a{0:24511} state up:active seq 7 addr [v2:172.21.15.60:6840/924163182,v1:172.21.15.60:6841/924163182] compat {c=[1],r=[1],i=[1fff]}] 2024-05-03T14:48:58.796 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.f{0:24469} state up:standby-replay seq 1 addr [v2:172.21.15.149:6834/882239924,v1:172.21.15.149:6835/882239924] compat {c=[1],r=[1],i=[1fff]}] 2024-05-03T14:48:58.796 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.b{1:24509} state up:active seq 26 addr [v2:172.21.15.90:6838/3512381172,v1:172.21.15.90:6839/3512381172] compat {c=[1],r=[1],i=[1fff]}] 2024-05-03T14:48:58.796 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.e{1:24455} state up:standby-replay seq 1 addr [v2:172.21.15.90:6834/1057049585,v1:172.21.15.90:6835/1057049585] compat {c=[1],r=[1],i=[1fff]}] 2024-05-03T14:48:58.796 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.i{2:24487} state up:active seq 30 addr [v2:172.21.15.149:6838/2527622212,v1:172.21.15.149:6839/2527622212] compat {c=[1],r=[1],i=[1fff]}] 2024-05-03T14:48:58.796 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.l{2:24427} state up:standby-replay seq 1 addr [v2:172.21.15.149:6832/3454021291,v1:172.21.15.149:6833/3454021291] compat {c=[1],r=[1],i=[1fff]}] 2024-05-03T14:48:58.796 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.c{3:24475} state up:active seq 32 addr [v2:172.21.15.149:6836/2611278690,v1:172.21.15.149:6837/2611278690] compat {c=[1],r=[1],i=[1fff]}] 2024-05-03T14:48:58.796 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.k{3:24425} state up:standby-replay seq 1 addr [v2:172.21.15.90:6832/3033424631,v1:172.21.15.90:6833/3033424631] compat {c=[1],r=[1],i=[1fff]}] 2024-05-03T14:48:58.796 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.h{4:24470} state up:active seq 31 addr [v2:172.21.15.90:6836/2243702844,v1:172.21.15.90:6837/2243702844] compat {c=[1],r=[1],i=[1fff]}] 2024-05-03T14:48:58.796 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.g{4:14397} state up:standby-replay seq 1 addr [v2:172.21.15.60:6838/3846205424,v1:172.21.15.60:6839/3846205424] compat {c=[1],r=[1],i=[1fff]}] 2024-05-03T14:48:58.796 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:48:58.796 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:48:58.797 INFO:teuthology.orchestra.run.smithi060.stdout:Standby daemons: 2024-05-03T14:48:58.797 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:48:58.797 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.j{-1:14358} state up:standby seq 1 addr [v2:172.21.15.60:6834/1849151522,v1:172.21.15.60:6835/1849151522] compat {c=[1],r=[1],i=[1fff]}] 2024-05-03T14:48:58.797 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.d{-1:14364} state up:standby seq 1 addr [v2:172.21.15.60:6836/523710346,v1:172.21.15.60:6837/523710346] compat {c=[1],r=[1],i=[1fff]}] 2024-05-03T14:48:58.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:58.794+0000 7f27a6800700 1 -- 172.21.15.60:0/416611538 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f27840b2990 msgr2=0x7f27840b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:58.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:58.794+0000 7f27a6800700 1 --2- 172.21.15.60:0/416611538 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f27840b2990 0x7f27840b4e50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2790005950 tx=0x7f2790009260 comp rx=0 tx=0).stop 2024-05-03T14:48:58.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:58.794+0000 7f27a6800700 1 -- 172.21.15.60:0/416611538 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27a01047f0 msgr2=0x7f27a019bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:48:58.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:58.794+0000 7f27a6800700 1 --2- 172.21.15.60:0/416611538 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27a01047f0 0x7f27a019bee0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f278c00e9f0 tx=0x7f278c00ed00 comp rx=0 tx=0).stop 2024-05-03T14:48:58.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:58.794+0000 7f27a6800700 1 -- 172.21.15.60:0/416611538 shutdown_connections 2024-05-03T14:48:58.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:58.794+0000 7f27a6800700 1 --2- 172.21.15.60:0/416611538 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f27840b2990 0x7f27840b4e50 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:58.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:58.794+0000 7f27a6800700 1 --2- 172.21.15.60:0/416611538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f27a0105190 0x7f27a01a2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:58.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:58.794+0000 7f27a6800700 1 --2- 172.21.15.60:0/416611538 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27a0101810 0x7f27a019b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:58.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:58.794+0000 7f27a6800700 1 --2- 172.21.15.60:0/416611538 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27a01047f0 0x7f27a019bee0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:48:58.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:58.794+0000 7f27a6800700 1 -- 172.21.15.60:0/416611538 >> 172.21.15.60:0/416611538 conn(0x7f27a00fd190 msgr2=0x7f27a01025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:48:58.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:58.794+0000 7f27a6800700 1 -- 172.21.15.60:0/416611538 shutdown_connections 2024-05-03T14:48:58.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:48:58.794+0000 7f27a6800700 1 -- 172.21.15.60:0/416611538 wait complete. 2024-05-03T14:48:58.799 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 29 2024-05-03T14:48:59.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:48:58 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/416611538' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-03T14:48:59.126 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:48:58 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/416611538' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-03T14:48:59.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:48:58 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/416611538' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-03T14:48:59.550 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- bash -c 'ceph osd dump' 2024-05-03T14:49:00.018 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:49:00.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:48:59 smithi060 ceph-mon[28873]: pgmap v466: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:49:00.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:48:59 smithi090 ceph-mon[33568]: pgmap v466: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:49:00.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:48:59 smithi149 ceph-mon[33332]: pgmap v466: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:49:02.068 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:49:01 smithi060 ceph-mon[28873]: pgmap v467: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:49:02.127 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:49:01 smithi090 ceph-mon[33568]: pgmap v467: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:49:02.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:49:01 smithi149 ceph-mon[33332]: pgmap v467: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:49:02.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:02.360+0000 7f6aed400700 1 -- 172.21.15.60:0/3674554766 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ae8106940 msgr2=0x7f6ae8106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:02.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:02.360+0000 7f6aed400700 1 --2- 172.21.15.60:0/3674554766 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ae8106940 0x7f6ae8106da0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6ad000b0d0 tx=0x7f6ad000b3e0 comp rx=0 tx=0).stop 2024-05-03T14:49:02.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:02.360+0000 7f6aed400700 1 -- 172.21.15.60:0/3674554766 shutdown_connections 2024-05-03T14:49:02.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:02.360+0000 7f6aed400700 1 --2- 172.21.15.60:0/3674554766 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ae81072e0 0x7f6ae810db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:02.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:02.360+0000 7f6aed400700 1 --2- 172.21.15.60:0/3674554766 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6ae8105740 0x7f6ae8105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:02.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:02.360+0000 7f6aed400700 1 --2- 172.21.15.60:0/3674554766 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ae8106940 0x7f6ae8106da0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:02.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:02.360+0000 7f6aed400700 1 -- 172.21.15.60:0/3674554766 >> 172.21.15.60:0/3674554766 conn(0x7f6ae8100d00 msgr2=0x7f6ae8103120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:49:02.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:02.360+0000 7f6aed400700 1 -- 172.21.15.60:0/3674554766 shutdown_connections 2024-05-03T14:49:02.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:02.360+0000 7f6aed400700 1 -- 172.21.15.60:0/3674554766 wait complete. 2024-05-03T14:49:02.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:02.362+0000 7f6aed400700 1 Processor -- start 2024-05-03T14:49:02.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:02.362+0000 7f6aed400700 1 -- start start 2024-05-03T14:49:02.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:02.362+0000 7f6aed400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6ae8105740 0x7f6ae819b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:02.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:02.362+0000 7f6aed400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ae8106940 0x7f6ae819bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:02.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:02.362+0000 7f6aed400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ae81072e0 0x7f6ae81a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:02.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:02.362+0000 7f6aed400700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae8111440 con 0x7f6ae81072e0 2024-05-03T14:49:02.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:02.363+0000 7f6aed400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae81112c0 con 0x7f6ae8106940 2024-05-03T14:49:02.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:02.363+0000 7f6aed400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae81115c0 con 0x7f6ae8105740 2024-05-03T14:49:02.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:02.363+0000 7f6ae7200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ae81072e0 0x7f6ae81a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:02.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:02.363+0000 7f6adde00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ae8106940 0x7f6ae819bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:02.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:02.363+0000 7f6adde00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ae8106940 0x7f6ae819bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.60:51706/0 (socket says 172.21.15.60:51706) 2024-05-03T14:49:02.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:02.363+0000 7f6ae6800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6ae8105740 0x7f6ae819b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:02.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:02.363+0000 7f6adde00700 1 -- 172.21.15.60:0/2587581414 learned_addr learned my addr 172.21.15.60:0/2587581414 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:49:02.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:02.363+0000 7f6ae7200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ae81072e0 0x7f6ae81a2f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34200/0 (socket says 172.21.15.60:34200) 2024-05-03T14:49:02.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:02.363+0000 7f6adde00700 1 -- 172.21.15.60:0/2587581414 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6ae8105740 msgr2=0x7f6ae819b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:02.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:02.363+0000 7f6adde00700 1 --2- 172.21.15.60:0/2587581414 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6ae8105740 0x7f6ae819b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:02.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:02.363+0000 7f6adde00700 1 -- 172.21.15.60:0/2587581414 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ae81072e0 msgr2=0x7f6ae81a2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:02.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:02.363+0000 7f6adde00700 1 --2- 172.21.15.60:0/2587581414 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ae81072e0 0x7f6ae81a2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:02.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:02.363+0000 7f6adde00700 1 -- 172.21.15.60:0/2587581414 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ae81a35f0 con 0x7f6ae8106940 2024-05-03T14:49:02.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:02.364+0000 7f6adde00700 1 --2- 172.21.15.60:0/2587581414 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ae8106940 0x7f6ae819bed0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0007d90 tx=0x7f6ad0007dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:49:02.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:02.364+0000 7f6adfe00700 1 -- 172.21.15.60:0/2587581414 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad0021020 con 0x7f6ae8106940 2024-05-03T14:49:02.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:02.364+0000 7f6aed400700 1 -- 172.21.15.60:0/2587581414 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ae81a3880 con 0x7f6ae8106940 2024-05-03T14:49:02.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:02.364+0000 7f6adfe00700 1 -- 172.21.15.60:0/2587581414 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f6ad00092e0 con 0x7f6ae8106940 2024-05-03T14:49:02.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:02.364+0000 7f6adfe00700 1 -- 172.21.15.60:0/2587581414 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad0005340 con 0x7f6ae8106940 2024-05-03T14:49:02.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:02.364+0000 7f6aed400700 1 -- 172.21.15.60:0/2587581414 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ae81a3d30 con 0x7f6ae8106940 2024-05-03T14:49:02.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:02.365+0000 7f6aed400700 1 -- 172.21.15.60:0/2587581414 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ae806a190 con 0x7f6ae8106940 2024-05-03T14:49:02.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:02.368+0000 7f6adfe00700 1 -- 172.21.15.60:0/2587581414 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f6ad0024020 con 0x7f6ae8106940 2024-05-03T14:49:02.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:02.369+0000 7f6adfe00700 1 --2- 172.21.15.60:0/2587581414 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f6acc0b2910 0x7f6acc0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:02.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:02.370+0000 7f6adfe00700 1 -- 172.21.15.60:0/2587581414 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f6ad000f020 con 0x7f6ae8106940 2024-05-03T14:49:02.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:02.370+0000 7f6ae6800700 1 --2- 172.21.15.60:0/2587581414 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f6acc0b2910 0x7f6acc0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:02.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:02.370+0000 7f6ae6800700 1 --2- 172.21.15.60:0/2587581414 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f6acc0b2910 0x7f6acc0b4dd0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6ae81067a0 tx=0x7f6ad8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:49:02.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:02.371+0000 7f6adfe00700 1 -- 172.21.15.60:0/2587581414 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f6ad00ab4e0 con 0x7f6ae8106940 2024-05-03T14:49:02.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:02.614+0000 7f6aed400700 1 -- 172.21.15.60:0/2587581414 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f6ae802e510 con 0x7f6ae8106940 2024-05-03T14:49:02.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:02.615+0000 7f6adfe00700 1 -- 172.21.15.60:0/2587581414 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v89) ==== 56+0+5616 (secure 0 0 0) 0x7f6ad00aade0 con 0x7f6ae8106940 2024-05-03T14:49:02.618 INFO:teuthology.orchestra.run.smithi060.stdout:epoch 89 2024-05-03T14:49:02.618 INFO:teuthology.orchestra.run.smithi060.stdout:fsid 285d64b2-095a-11ef-bc97-c7b262605968 2024-05-03T14:49:02.619 INFO:teuthology.orchestra.run.smithi060.stdout:created 2024-05-03T14:34:53.560460+0000 2024-05-03T14:49:02.619 INFO:teuthology.orchestra.run.smithi060.stdout:modified 2024-05-03T14:46:05.582395+0000 2024-05-03T14:49:02.619 INFO:teuthology.orchestra.run.smithi060.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-03T14:49:02.619 INFO:teuthology.orchestra.run.smithi060.stdout:crush_version 27 2024-05-03T14:49:02.619 INFO:teuthology.orchestra.run.smithi060.stdout:full_ratio 0.95 2024-05-03T14:49:02.619 INFO:teuthology.orchestra.run.smithi060.stdout:backfillfull_ratio 0.9 2024-05-03T14:49:02.619 INFO:teuthology.orchestra.run.smithi060.stdout:nearfull_ratio 0.85 2024-05-03T14:49:02.619 INFO:teuthology.orchestra.run.smithi060.stdout:require_min_compat_client luminous 2024-05-03T14:49:02.619 INFO:teuthology.orchestra.run.smithi060.stdout:min_compat_client luminous 2024-05-03T14:49:02.619 INFO:teuthology.orchestra.run.smithi060.stdout:require_osd_release squid 2024-05-03T14:49:02.619 INFO:teuthology.orchestra.run.smithi060.stdout:stretch_mode_enabled false 2024-05-03T14:49:02.619 INFO:teuthology.orchestra.run.smithi060.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-05-03T14:49:02.620 INFO:teuthology.orchestra.run.smithi060.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 78 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-05-03T14:49:02.620 INFO:teuthology.orchestra.run.smithi060.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 78 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-05-03T14:49:02.620 INFO:teuthology.orchestra.run.smithi060.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-05-03T14:49:02.620 INFO:teuthology.orchestra.run.smithi060.stdout:max_osd 12 2024-05-03T14:49:02.620 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0 up in weight 1 up_from 8 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.60:6802/2952930162,v1:172.21.15.60:6803/2952930162] [v2:172.21.15.60:6804/2952930162,v1:172.21.15.60:6805/2952930162] exists,up f0797730-c4c1-47ea-9dce-9270d5ebb4ce 2024-05-03T14:49:02.620 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1 up in weight 1 up_from 13 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.90:6800/720063242,v1:172.21.15.90:6801/720063242] [v2:172.21.15.90:6802/720063242,v1:172.21.15.90:6803/720063242] exists,up 96d0ad08-cda4-4824-bad4-e64973508c0b 2024-05-03T14:49:02.620 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2 up in weight 1 up_from 18 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.149:6800/1359662077,v1:172.21.15.149:6801/1359662077] [v2:172.21.15.149:6802/1359662077,v1:172.21.15.149:6803/1359662077] exists,up ce95fb7c-1611-4dd1-aee7-97c420d6a466 2024-05-03T14:49:02.620 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3 up in weight 1 up_from 25 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.60:6810/3226133334,v1:172.21.15.60:6811/3226133334] [v2:172.21.15.60:6812/3226133334,v1:172.21.15.60:6813/3226133334] exists,up ef31136a-9b36-4b87-8389-f53d311bc5cc 2024-05-03T14:49:02.620 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4 up in weight 1 up_from 31 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.90:6808/2201391883,v1:172.21.15.90:6809/2201391883] [v2:172.21.15.90:6810/2201391883,v1:172.21.15.90:6811/2201391883] exists,up 73db7408-a2ff-47e9-b13d-d31688a93610 2024-05-03T14:49:02.620 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5 up in weight 1 up_from 36 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.149:6808/2139732836,v1:172.21.15.149:6809/2139732836] [v2:172.21.15.149:6810/2139732836,v1:172.21.15.149:6811/2139732836] exists,up aefb78a2-522a-4b35-92b8-e5c04fa47b04 2024-05-03T14:49:02.620 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6 up in weight 1 up_from 42 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.60:6818/3738521002,v1:172.21.15.60:6819/3738521002] [v2:172.21.15.60:6820/3738521002,v1:172.21.15.60:6821/3738521002] exists,up 7c887819-6182-4636-8cfd-d845c30bd13c 2024-05-03T14:49:02.621 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7 up in weight 1 up_from 47 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.90:6816/1686298661,v1:172.21.15.90:6817/1686298661] [v2:172.21.15.90:6818/1686298661,v1:172.21.15.90:6819/1686298661] exists,up ebce6acf-8873-4b35-bd5f-80de2163a307 2024-05-03T14:49:02.621 INFO:teuthology.orchestra.run.smithi060.stdout:osd.8 up in weight 1 up_from 53 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.149:6816/2608348248,v1:172.21.15.149:6817/2608348248] [v2:172.21.15.149:6818/2608348248,v1:172.21.15.149:6819/2608348248] exists,up 4ae04b98-e9b2-4956-83d9-17e5ac3c963f 2024-05-03T14:49:02.621 INFO:teuthology.orchestra.run.smithi060.stdout:osd.9 up in weight 1 up_from 59 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.60:6826/2156130999,v1:172.21.15.60:6827/2156130999] [v2:172.21.15.60:6828/2156130999,v1:172.21.15.60:6829/2156130999] exists,up d0c0a70d-aa76-47e4-91e8-c343c8c41197 2024-05-03T14:49:02.621 INFO:teuthology.orchestra.run.smithi060.stdout:osd.10 up in weight 1 up_from 64 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.90:6824/2492593590,v1:172.21.15.90:6825/2492593590] [v2:172.21.15.90:6826/2492593590,v1:172.21.15.90:6827/2492593590] exists,up cdc3d7b8-33c7-4cc4-be4e-6887d15a3d8c 2024-05-03T14:49:02.621 INFO:teuthology.orchestra.run.smithi060.stdout:osd.11 up in weight 1 up_from 70 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.149:6824/1063245126,v1:172.21.15.149:6825/1063245126] [v2:172.21.15.149:6826/1063245126,v1:172.21.15.149:6827/1063245126] exists,up 8d17ec1a-bdd5-41f1-9b59-c2b6a527bfbb 2024-05-03T14:49:02.621 INFO:teuthology.orchestra.run.smithi060.stdout:pg_upmap_items 2.5 [8,11] 2024-05-03T14:49:02.621 INFO:teuthology.orchestra.run.smithi060.stdout:pg_upmap_items 2.16 [2,4] 2024-05-03T14:49:02.621 INFO:teuthology.orchestra.run.smithi060.stdout:pg_upmap_items 2.1b [2,1] 2024-05-03T14:49:02.621 INFO:teuthology.orchestra.run.smithi060.stdout:pg_upmap_items 2.1f [7,11] 2024-05-03T14:49:02.621 INFO:teuthology.orchestra.run.smithi060.stdout:pg_upmap_items 3.20 [5,9] 2024-05-03T14:49:02.621 INFO:teuthology.orchestra.run.smithi060.stdout:pg_upmap_items 3.34 [4,9] 2024-05-03T14:49:02.621 INFO:teuthology.orchestra.run.smithi060.stdout:pg_upmap_items 3.36 [4,9] 2024-05-03T14:49:02.621 INFO:teuthology.orchestra.run.smithi060.stdout:pg_upmap_items 4.b [9,8] 2024-05-03T14:49:02.621 INFO:teuthology.orchestra.run.smithi060.stdout:pg_upmap_items 4.c [9,10] 2024-05-03T14:49:02.622 INFO:teuthology.orchestra.run.smithi060.stdout:pg_upmap_items 4.e [5,8] 2024-05-03T14:49:02.622 INFO:teuthology.orchestra.run.smithi060.stdout:pg_upmap_items 4.19 [9,8] 2024-05-03T14:49:02.622 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:6801/424391417 expires 2024-05-04T14:36:04.154333+0000 2024-05-03T14:49:02.622 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:6800/424391417 expires 2024-05-04T14:36:04.154333+0000 2024-05-03T14:49:02.622 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:0/1426532561 expires 2024-05-04T14:36:04.154333+0000 2024-05-03T14:49:02.622 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:0/3230079926 expires 2024-05-04T14:35:28.066961+0000 2024-05-03T14:49:02.622 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:0/2334895610 expires 2024-05-04T14:35:28.066961+0000 2024-05-03T14:49:02.622 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:6800/888447185 expires 2024-05-04T14:35:28.066961+0000 2024-05-03T14:49:02.622 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:0/3996899937 expires 2024-05-04T14:35:28.066961+0000 2024-05-03T14:49:02.622 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:0/2779644018 expires 2024-05-04T14:36:04.154333+0000 2024-05-03T14:49:02.622 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:0/71440838 expires 2024-05-04T14:36:04.154333+0000 2024-05-03T14:49:02.622 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:6801/888447185 expires 2024-05-04T14:35:28.066961+0000 2024-05-03T14:49:02.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:02.620+0000 7f6aed400700 1 -- 172.21.15.60:0/2587581414 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f6acc0b2910 msgr2=0x7f6acc0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:02.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:02.620+0000 7f6aed400700 1 --2- 172.21.15.60:0/2587581414 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f6acc0b2910 0x7f6acc0b4dd0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6ae81067a0 tx=0x7f6ad8009260 comp rx=0 tx=0).stop 2024-05-03T14:49:02.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:02.620+0000 7f6aed400700 1 -- 172.21.15.60:0/2587581414 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ae8106940 msgr2=0x7f6ae819bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:02.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:02.620+0000 7f6aed400700 1 --2- 172.21.15.60:0/2587581414 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ae8106940 0x7f6ae819bed0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0007d90 tx=0x7f6ad0007dc0 comp rx=0 tx=0).stop 2024-05-03T14:49:02.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:02.620+0000 7f6aed400700 1 -- 172.21.15.60:0/2587581414 shutdown_connections 2024-05-03T14:49:02.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:02.620+0000 7f6aed400700 1 --2- 172.21.15.60:0/2587581414 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f6acc0b2910 0x7f6acc0b4dd0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:02.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:02.620+0000 7f6aed400700 1 --2- 172.21.15.60:0/2587581414 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ae81072e0 0x7f6ae81a2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:02.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:02.620+0000 7f6aed400700 1 --2- 172.21.15.60:0/2587581414 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6ae8105740 0x7f6ae819b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:02.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:02.620+0000 7f6aed400700 1 --2- 172.21.15.60:0/2587581414 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ae8106940 0x7f6ae819bed0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:02.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:02.620+0000 7f6aed400700 1 -- 172.21.15.60:0/2587581414 >> 172.21.15.60:0/2587581414 conn(0x7f6ae8100d00 msgr2=0x7f6ae8109be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:49:02.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:02.620+0000 7f6aed400700 1 -- 172.21.15.60:0/2587581414 shutdown_connections 2024-05-03T14:49:02.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:02.620+0000 7f6aed400700 1 -- 172.21.15.60:0/2587581414 wait complete. 2024-05-03T14:49:03.012 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:49:02 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/2587581414' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-03T14:49:03.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:49:02 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/2587581414' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-03T14:49:03.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:49:02 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/2587581414' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-03T14:49:03.524 INFO:teuthology.run_tasks:Running task fs.ready... 2024-05-03T14:49:03.535 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T14:49:04.006 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:49:04.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:49:03 smithi060 ceph-mon[28873]: pgmap v468: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T14:49:04.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:49:03 smithi090 ceph-mon[33568]: pgmap v468: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T14:49:04.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:49:03 smithi149 ceph-mon[33332]: pgmap v468: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T14:49:05.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:49:04 smithi090 ceph-mon[33568]: pgmap v469: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:49:05.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:49:04 smithi149 ceph-mon[33332]: pgmap v469: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:49:05.196 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:49:04 smithi060 ceph-mon[28873]: pgmap v469: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:49:06.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:06.276+0000 7f1007200700 1 -- 172.21.15.60:0/2991366863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1000103fd0 msgr2=0x7f1000108020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:06.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:06.276+0000 7f1007200700 1 --2- 172.21.15.60:0/2991366863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1000103fd0 0x7f1000108020 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0009920 tx=0x7f0ff0009c30 comp rx=0 tx=0).stop 2024-05-03T14:49:06.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:06.276+0000 7f1007200700 1 -- 172.21.15.60:0/2991366863 shutdown_connections 2024-05-03T14:49:06.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:06.276+0000 7f1007200700 1 --2- 172.21.15.60:0/2991366863 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1000108700 0x7f10001102d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:06.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:06.276+0000 7f1007200700 1 --2- 172.21.15.60:0/2991366863 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1000103620 0x7f1000103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:06.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:06.276+0000 7f1007200700 1 --2- 172.21.15.60:0/2991366863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1000103fd0 0x7f1000108020 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:06.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:06.276+0000 7f1007200700 1 -- 172.21.15.60:0/2991366863 >> 172.21.15.60:0/2991366863 conn(0x7f10000feeb0 msgr2=0x7f10001012d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:49:06.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:06.276+0000 7f1007200700 1 -- 172.21.15.60:0/2991366863 shutdown_connections 2024-05-03T14:49:06.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:06.276+0000 7f1007200700 1 -- 172.21.15.60:0/2991366863 wait complete. 2024-05-03T14:49:06.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:06.277+0000 7f1007200700 1 Processor -- start 2024-05-03T14:49:06.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:06.278+0000 7f1007200700 1 -- start start 2024-05-03T14:49:06.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:06.278+0000 7f1007200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1000103620 0x7f100019bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:06.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:06.278+0000 7f1007200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1000103fd0 0x7f100019c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:06.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:06.278+0000 7f1007200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1000108700 0x7f10001a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:06.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:06.278+0000 7f1007200700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f1000113800 con 0x7f1000103fd0 2024-05-03T14:49:06.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:06.278+0000 7f1007200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1000113680 con 0x7f1000103620 2024-05-03T14:49:06.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:06.278+0000 7f1007200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1000113980 con 0x7f1000108700 2024-05-03T14:49:06.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:06.278+0000 7f1005400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1000103fd0 0x7f100019c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:06.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:06.278+0000 7f1005e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1000103620 0x7f100019bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:06.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:06.279+0000 7f1005e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1000103620 0x7f100019bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.60:51722/0 (socket says 172.21.15.60:51722) 2024-05-03T14:49:06.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:06.279+0000 7f1005e00700 1 -- 172.21.15.60:0/2214965238 learned_addr learned my addr 172.21.15.60:0/2214965238 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:49:06.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:06.279+0000 7f1005400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1000103fd0 0x7f100019c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34226/0 (socket says 172.21.15.60:34226) 2024-05-03T14:49:06.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:06.279+0000 7f1006800700 1 --2- 172.21.15.60:0/2214965238 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1000108700 0x7f10001a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:06.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:06.279+0000 7f1005400700 1 -- 172.21.15.60:0/2214965238 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1000108700 msgr2=0x7f10001a07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:06.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:06.279+0000 7f1005400700 1 --2- 172.21.15.60:0/2214965238 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1000108700 0x7f10001a07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:06.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:06.279+0000 7f1005400700 1 -- 172.21.15.60:0/2214965238 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1000103620 msgr2=0x7f100019bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:06.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:06.279+0000 7f1005400700 1 --2- 172.21.15.60:0/2214965238 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1000103620 0x7f100019bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:06.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:06.279+0000 7f1005400700 1 -- 172.21.15.60:0/2214965238 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10001a0ee0 con 0x7f1000103fd0 2024-05-03T14:49:06.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:06.279+0000 7f1006800700 1 --2- 172.21.15.60:0/2214965238 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1000108700 0x7f10001a07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T14:49:06.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:06.279+0000 7f1005e00700 1 --2- 172.21.15.60:0/2214965238 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1000103620 0x7f100019bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:49:06.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:06.279+0000 7f1005400700 1 --2- 172.21.15.60:0/2214965238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1000103fd0 0x7f100019c410 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0009230 tx=0x7f0ff0010f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:49:06.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:06.280+0000 7f0ffaa00700 1 -- 172.21.15.60:0/2214965238 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff0027070 con 0x7f1000103fd0 2024-05-03T14:49:06.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:06.280+0000 7f1007200700 1 -- 172.21.15.60:0/2214965238 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10001a1170 con 0x7f1000103fd0 2024-05-03T14:49:06.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:06.280+0000 7f0ffaa00700 1 -- 172.21.15.60:0/2214965238 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f0ff000bb60 con 0x7f1000103fd0 2024-05-03T14:49:06.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:06.280+0000 7f0ffaa00700 1 -- 172.21.15.60:0/2214965238 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff00049c0 con 0x7f1000103fd0 2024-05-03T14:49:06.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:06.280+0000 7f1007200700 1 -- 172.21.15.60:0/2214965238 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10001a8a50 con 0x7f1000103fd0 2024-05-03T14:49:06.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:06.281+0000 7f1007200700 1 -- 172.21.15.60:0/2214965238 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f100006a190 con 0x7f1000103fd0 2024-05-03T14:49:06.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:06.283+0000 7f0ffaa00700 1 -- 172.21.15.60:0/2214965238 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f0ff002a070 con 0x7f1000103fd0 2024-05-03T14:49:06.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:06.285+0000 7f0ffaa00700 1 --2- 172.21.15.60:0/2214965238 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f0fe80b2910 0x7f0fe80b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:06.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:06.285+0000 7f0ffaa00700 1 -- 172.21.15.60:0/2214965238 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f0ff00e2ff0 con 0x7f1000103fd0 2024-05-03T14:49:06.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:06.285+0000 7f1005e00700 1 --2- 172.21.15.60:0/2214965238 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f0fe80b2910 0x7f0fe80b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:06.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:06.286+0000 7f1005e00700 1 --2- 172.21.15.60:0/2214965238 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f0fe80b2910 0x7f0fe80b4dd0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4011550 tx=0x7f0ff4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:49:06.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:06.290+0000 7f0ffaa00700 1 -- 172.21.15.60:0/2214965238 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f0ff0013030 con 0x7f1000103fd0 2024-05-03T14:49:06.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:06.577+0000 7f1007200700 1 -- 172.21.15.60:0/2214965238 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f10001a8c20 con 0x7f1000103fd0 2024-05-03T14:49:06.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:06.579+0000 7f0ffaa00700 1 -- 172.21.15.60:0/2214965238 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11929 (secure 0 0 0) 0x7f0ff00aa8c0 con 0x7f1000103fd0 2024-05-03T14:49:06.581 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:49:06.581 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1849151522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1849151522},{"type":"v1","addr":"172.21.15.60:6835","nonce":1849151522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/523710346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":523710346},{"type":"v1","addr":"172.21.15.60:6837","nonce":523710346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T14:44:34.226052+0000","modified":"2024-05-03T14:46:01.367913+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24511,"mds_1":24509,"mds_2":24487,"mds_3":24475,"mds_4":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14397":{"gid":14397,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.60:6839/3846205424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3846205424},{"type":"v1","addr":"172.21.15.60:6839","nonce":3846205424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6833/3033424631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":3033424631},{"type":"v1","addr":"172.21.15.90:6833","nonce":3033424631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6833/3454021291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3454021291},{"type":"v1","addr":"172.21.15.149:6833","nonce":3454021291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6835/1057049585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1057049585},{"type":"v1","addr":"172.21.15.90:6835","nonce":1057049585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/882239924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":882239924},{"type":"v1","addr":"172.21.15.149:6835","nonce":882239924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":31,"addr":"172.21.15.90:6837/2243702844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":2243702844},{"type":"v1","addr":"172.21.15.90:6837","nonce":2243702844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":32,"addr":"172.21.15.149:6837/2611278690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2611278690},{"type":"v1","addr":"172.21.15.149:6837","nonce":2611278690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.149:6839/2527622212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2527622212},{"type":"v1","addr":"172.21.15.149:6839","nonce":2527622212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.90:6839/3512381172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3512381172},{"type":"v1","addr":"172.21.15.90:6839","nonce":3512381172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.60:6841/924163182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":924163182},{"type":"v1","addr":"172.21.15.60:6841","nonce":924163182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24511,"qdb_cluster":[24470,24475,24487,24509,24511]},"id":1}]} 2024-05-03T14:49:06.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:06.583+0000 7f1007200700 1 -- 172.21.15.60:0/2214965238 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f0fe80b2910 msgr2=0x7f0fe80b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:06.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:06.583+0000 7f1007200700 1 --2- 172.21.15.60:0/2214965238 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f0fe80b2910 0x7f0fe80b4dd0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4011550 tx=0x7f0ff4009260 comp rx=0 tx=0).stop 2024-05-03T14:49:06.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:06.583+0000 7f1007200700 1 -- 172.21.15.60:0/2214965238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1000103fd0 msgr2=0x7f100019c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:06.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:06.583+0000 7f1007200700 1 --2- 172.21.15.60:0/2214965238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1000103fd0 0x7f100019c410 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0009230 tx=0x7f0ff0010f80 comp rx=0 tx=0).stop 2024-05-03T14:49:06.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:06.583+0000 7f1007200700 1 -- 172.21.15.60:0/2214965238 shutdown_connections 2024-05-03T14:49:06.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:06.583+0000 7f1007200700 1 --2- 172.21.15.60:0/2214965238 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f0fe80b2910 0x7f0fe80b4dd0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:06.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:06.583+0000 7f1007200700 1 --2- 172.21.15.60:0/2214965238 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1000108700 0x7f10001a07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:06.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:06.583+0000 7f1007200700 1 --2- 172.21.15.60:0/2214965238 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1000103620 0x7f100019bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:06.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:06.583+0000 7f1007200700 1 --2- 172.21.15.60:0/2214965238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1000103fd0 0x7f100019c410 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:06.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:06.583+0000 7f1007200700 1 -- 172.21.15.60:0/2214965238 >> 172.21.15.60:0/2214965238 conn(0x7f10000feeb0 msgr2=0x7f1000100250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:49:06.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:06.583+0000 7f1007200700 1 -- 172.21.15.60:0/2214965238 shutdown_connections 2024-05-03T14:49:06.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:06.583+0000 7f1007200700 1 -- 172.21.15.60:0/2214965238 wait complete. 2024-05-03T14:49:06.586 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 29 2024-05-03T14:49:07.432 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T14:49:07.836 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:49:07 smithi060 ceph-mon[28873]: pgmap v470: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:49:07.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:49:07 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/2214965238' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T14:49:07.868 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:49:07.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:49:07 smithi090 ceph-mon[33568]: pgmap v470: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:49:07.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:49:07 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/2214965238' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T14:49:07.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:49:07 smithi149 ceph-mon[33332]: pgmap v470: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:49:07.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:49:07 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/2214965238' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T14:49:09.755 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:49:09 smithi060 ceph-mon[28873]: pgmap v471: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:49:09.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:49:09 smithi090 ceph-mon[33568]: pgmap v471: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:49:09.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:49:09 smithi149 ceph-mon[33332]: pgmap v471: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:49:10.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:10.029+0000 7f2e88e00700 1 -- 172.21.15.60:0/885302478 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e84075ff0 msgr2=0x7f2e8410f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:10.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:10.029+0000 7f2e88e00700 1 --2- 172.21.15.60:0/885302478 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e84075ff0 0x7f2e8410f9c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c00b0a0 tx=0x7f2e6c00b3b0 comp rx=0 tx=0).stop 2024-05-03T14:49:10.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:10.029+0000 7f2e88e00700 1 -- 172.21.15.60:0/885302478 shutdown_connections 2024-05-03T14:49:10.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:10.029+0000 7f2e88e00700 1 --2- 172.21.15.60:0/885302478 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2e8410ff00 0x7f2e841122f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:10.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:10.029+0000 7f2e88e00700 1 --2- 172.21.15.60:0/885302478 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2e840756d0 0x7f2e84075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:10.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:10.029+0000 7f2e88e00700 1 --2- 172.21.15.60:0/885302478 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e84075ff0 0x7f2e8410f9c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:10.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:10.029+0000 7f2e88e00700 1 -- 172.21.15.60:0/885302478 >> 172.21.15.60:0/885302478 conn(0x7f2e8407a600 msgr2=0x7f2e8407aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:49:10.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:10.030+0000 7f2e88e00700 1 -- 172.21.15.60:0/885302478 shutdown_connections 2024-05-03T14:49:10.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:10.030+0000 7f2e88e00700 1 -- 172.21.15.60:0/885302478 wait complete. 2024-05-03T14:49:10.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:10.031+0000 7f2e88e00700 1 Processor -- start 2024-05-03T14:49:10.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:10.031+0000 7f2e88e00700 1 -- start start 2024-05-03T14:49:10.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:10.031+0000 7f2e88e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2e840756d0 0x7f2e8419be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:10.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:10.031+0000 7f2e88e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e84075ff0 0x7f2e8419c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:10.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:10.031+0000 7f2e88e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2e8410ff00 0x7f2e841a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:10.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:10.031+0000 7f2e88e00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f2e84115b20 con 0x7f2e84075ff0 2024-05-03T14:49:10.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:10.031+0000 7f2e88e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2e841159a0 con 0x7f2e8410ff00 2024-05-03T14:49:10.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:10.031+0000 7f2e88e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2e84115ca0 con 0x7f2e840756d0 2024-05-03T14:49:10.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:10.032+0000 7f2e79400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e84075ff0 0x7f2e8419c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:10.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:10.032+0000 7f2e79400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e84075ff0 0x7f2e8419c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34236/0 (socket says 172.21.15.60:34236) 2024-05-03T14:49:10.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:10.032+0000 7f2e79400700 1 -- 172.21.15.60:0/1707617769 learned_addr learned my addr 172.21.15.60:0/1707617769 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:49:10.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:10.032+0000 7f2e81e00700 1 --2- 172.21.15.60:0/1707617769 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2e840756d0 0x7f2e8419be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:10.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:10.032+0000 7f2e82800700 1 --2- 172.21.15.60:0/1707617769 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2e8410ff00 0x7f2e841a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:10.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:10.032+0000 7f2e79400700 1 -- 172.21.15.60:0/1707617769 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2e840756d0 msgr2=0x7f2e8419be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:10.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:10.032+0000 7f2e79400700 1 --2- 172.21.15.60:0/1707617769 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2e840756d0 0x7f2e8419be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:10.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:10.032+0000 7f2e79400700 1 -- 172.21.15.60:0/1707617769 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2e8410ff00 msgr2=0x7f2e841a0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:10.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:10.032+0000 7f2e79400700 1 --2- 172.21.15.60:0/1707617769 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2e8410ff00 0x7f2e841a0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:10.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:10.032+0000 7f2e79400700 1 -- 172.21.15.60:0/1707617769 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e841a0ea0 con 0x7f2e84075ff0 2024-05-03T14:49:10.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:10.032+0000 7f2e81e00700 1 --2- 172.21.15.60:0/1707617769 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2e840756d0 0x7f2e8419be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:49:10.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:10.033+0000 7f2e79400700 1 --2- 172.21.15.60:0/1707617769 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e84075ff0 0x7f2e8419c3d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c0062a0 tx=0x7f2e6c004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:49:10.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:10.033+0000 7f2e7b400700 1 -- 172.21.15.60:0/1707617769 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c021070 con 0x7f2e84075ff0 2024-05-03T14:49:10.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:10.033+0000 7f2e7b400700 1 -- 172.21.15.60:0/1707617769 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f2e6c007b80 con 0x7f2e84075ff0 2024-05-03T14:49:10.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:10.033+0000 7f2e88e00700 1 -- 172.21.15.60:0/1707617769 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e841a1130 con 0x7f2e84075ff0 2024-05-03T14:49:10.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:10.033+0000 7f2e7b400700 1 -- 172.21.15.60:0/1707617769 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c005280 con 0x7f2e84075ff0 2024-05-03T14:49:10.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:10.033+0000 7f2e88e00700 1 -- 172.21.15.60:0/1707617769 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e841a8a10 con 0x7f2e84075ff0 2024-05-03T14:49:10.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:10.035+0000 7f2e88e00700 1 -- 172.21.15.60:0/1707617769 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e8406a190 con 0x7f2e84075ff0 2024-05-03T14:49:10.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:10.037+0000 7f2e7b400700 1 -- 172.21.15.60:0/1707617769 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f2e6c024070 con 0x7f2e84075ff0 2024-05-03T14:49:10.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:10.038+0000 7f2e7b400700 1 --2- 172.21.15.60:0/1707617769 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f2e680b2910 0x7f2e680b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:10.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:10.038+0000 7f2e7b400700 1 -- 172.21.15.60:0/1707617769 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f2e6c00f020 con 0x7f2e84075ff0 2024-05-03T14:49:10.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:10.039+0000 7f2e81e00700 1 --2- 172.21.15.60:0/1707617769 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f2e680b2910 0x7f2e680b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:10.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:10.039+0000 7f2e81e00700 1 --2- 172.21.15.60:0/1707617769 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f2e680b2910 0x7f2e680b4dd0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2e74011490 tx=0x7f2e74009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:49:10.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:10.043+0000 7f2e7b400700 1 -- 172.21.15.60:0/1707617769 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f2e6c02b030 con 0x7f2e84075ff0 2024-05-03T14:49:10.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:10.331+0000 7f2e88e00700 1 -- 172.21.15.60:0/1707617769 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2e841a8be0 con 0x7f2e84075ff0 2024-05-03T14:49:10.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:10.333+0000 7f2e7b400700 1 -- 172.21.15.60:0/1707617769 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11929 (secure 0 0 0) 0x7f2e6c0aae50 con 0x7f2e84075ff0 2024-05-03T14:49:10.335 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:49:10.335 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1849151522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1849151522},{"type":"v1","addr":"172.21.15.60:6835","nonce":1849151522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/523710346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":523710346},{"type":"v1","addr":"172.21.15.60:6837","nonce":523710346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T14:44:34.226052+0000","modified":"2024-05-03T14:46:01.367913+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24511,"mds_1":24509,"mds_2":24487,"mds_3":24475,"mds_4":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14397":{"gid":14397,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.60:6839/3846205424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3846205424},{"type":"v1","addr":"172.21.15.60:6839","nonce":3846205424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6833/3033424631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":3033424631},{"type":"v1","addr":"172.21.15.90:6833","nonce":3033424631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6833/3454021291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3454021291},{"type":"v1","addr":"172.21.15.149:6833","nonce":3454021291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6835/1057049585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1057049585},{"type":"v1","addr":"172.21.15.90:6835","nonce":1057049585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/882239924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":882239924},{"type":"v1","addr":"172.21.15.149:6835","nonce":882239924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":31,"addr":"172.21.15.90:6837/2243702844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":2243702844},{"type":"v1","addr":"172.21.15.90:6837","nonce":2243702844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":32,"addr":"172.21.15.149:6837/2611278690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2611278690},{"type":"v1","addr":"172.21.15.149:6837","nonce":2611278690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.149:6839/2527622212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2527622212},{"type":"v1","addr":"172.21.15.149:6839","nonce":2527622212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.90:6839/3512381172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3512381172},{"type":"v1","addr":"172.21.15.90:6839","nonce":3512381172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.60:6841/924163182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":924163182},{"type":"v1","addr":"172.21.15.60:6841","nonce":924163182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24511,"qdb_cluster":[24470,24475,24487,24509,24511]},"id":1}]} 2024-05-03T14:49:10.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:10.336+0000 7f2e88e00700 1 -- 172.21.15.60:0/1707617769 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f2e680b2910 msgr2=0x7f2e680b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:10.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:10.336+0000 7f2e88e00700 1 --2- 172.21.15.60:0/1707617769 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f2e680b2910 0x7f2e680b4dd0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2e74011490 tx=0x7f2e74009260 comp rx=0 tx=0).stop 2024-05-03T14:49:10.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:10.336+0000 7f2e88e00700 1 -- 172.21.15.60:0/1707617769 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e84075ff0 msgr2=0x7f2e8419c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:10.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:10.336+0000 7f2e88e00700 1 --2- 172.21.15.60:0/1707617769 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e84075ff0 0x7f2e8419c3d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c0062a0 tx=0x7f2e6c004d00 comp rx=0 tx=0).stop 2024-05-03T14:49:10.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:10.336+0000 7f2e88e00700 1 -- 172.21.15.60:0/1707617769 shutdown_connections 2024-05-03T14:49:10.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:10.336+0000 7f2e88e00700 1 --2- 172.21.15.60:0/1707617769 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f2e680b2910 0x7f2e680b4dd0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:10.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:10.336+0000 7f2e88e00700 1 --2- 172.21.15.60:0/1707617769 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2e8410ff00 0x7f2e841a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:10.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:10.336+0000 7f2e88e00700 1 --2- 172.21.15.60:0/1707617769 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2e840756d0 0x7f2e8419be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:10.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:10.336+0000 7f2e88e00700 1 --2- 172.21.15.60:0/1707617769 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e84075ff0 0x7f2e8419c3d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:10.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:10.336+0000 7f2e88e00700 1 -- 172.21.15.60:0/1707617769 >> 172.21.15.60:0/1707617769 conn(0x7f2e8407a600 msgr2=0x7f2e84100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:49:10.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:10.337+0000 7f2e88e00700 1 -- 172.21.15.60:0/1707617769 shutdown_connections 2024-05-03T14:49:10.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:10.337+0000 7f2e88e00700 1 -- 172.21.15.60:0/1707617769 wait complete. 2024-05-03T14:49:10.340 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 29 2024-05-03T14:49:10.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:49:10 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/1707617769' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T14:49:10.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:49:10 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/1707617769' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T14:49:10.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:49:10 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/1707617769' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T14:49:11.127 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-03T14:49:11.604 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:49:11.779 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:49:11 smithi060 ceph-mon[28873]: pgmap v472: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:49:11.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:49:11 smithi090 ceph-mon[33568]: pgmap v472: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:49:11.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:49:11 smithi149 ceph-mon[33332]: pgmap v472: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:49:13.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:13.766+0000 7fba82600700 1 -- 172.21.15.60:0/1234953685 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba7c0756d0 msgr2=0x7fba7c075ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:13.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:13.766+0000 7fba82600700 1 --2- 172.21.15.60:0/1234953685 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba7c0756d0 0x7fba7c075ab0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fba6c009990 tx=0x7fba6c009ca0 comp rx=0 tx=0).stop 2024-05-03T14:49:13.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:13.766+0000 7fba82600700 1 -- 172.21.15.60:0/1234953685 shutdown_connections 2024-05-03T14:49:13.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:13.766+0000 7fba82600700 1 --2- 172.21.15.60:0/1234953685 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba7c114670 0x7fba7c116a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:13.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:13.766+0000 7fba82600700 1 --2- 172.21.15.60:0/1234953685 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba7c0756d0 0x7fba7c075ab0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:13.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:13.766+0000 7fba82600700 1 --2- 172.21.15.60:0/1234953685 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba7c075ff0 0x7fba7c114130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:13.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:13.766+0000 7fba82600700 1 -- 172.21.15.60:0/1234953685 >> 172.21.15.60:0/1234953685 conn(0x7fba7c07a600 msgr2=0x7fba7c07aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:49:13.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:13.766+0000 7fba82600700 1 -- 172.21.15.60:0/1234953685 shutdown_connections 2024-05-03T14:49:13.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:13.766+0000 7fba82600700 1 -- 172.21.15.60:0/1234953685 wait complete. 2024-05-03T14:49:13.769 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:49:13 smithi060 ceph-mon[28873]: pgmap v473: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T14:49:13.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:13.767+0000 7fba82600700 1 Processor -- start 2024-05-03T14:49:13.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:13.768+0000 7fba82600700 1 -- start start 2024-05-03T14:49:13.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:13.768+0000 7fba82600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba7c0756d0 0x7fba7c1a0340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:13.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:13.768+0000 7fba82600700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba7c075ff0 0x7fba7c1a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:13.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:13.768+0000 7fba82600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba7c114670 0x7fba7c1a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:13.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:13.768+0000 7fba82600700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fba7c119f50 con 0x7fba7c075ff0 2024-05-03T14:49:13.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:13.768+0000 7fba82600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fba7c119dd0 con 0x7fba7c0756d0 2024-05-03T14:49:13.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:13.768+0000 7fba82600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fba7c11a0d0 con 0x7fba7c114670 2024-05-03T14:49:13.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:13.768+0000 7fba7b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba7c0756d0 0x7fba7c1a0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:13.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:13.768+0000 7fba72a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba7c075ff0 0x7fba7c1a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:13.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:13.769+0000 7fba7b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba7c0756d0 0x7fba7c1a0340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.60:38762/0 (socket says 172.21.15.60:38762) 2024-05-03T14:49:13.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:13.769+0000 7fba7b400700 1 -- 172.21.15.60:0/3830150067 learned_addr learned my addr 172.21.15.60:0/3830150067 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:49:13.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:13.769+0000 7fba7be00700 1 --2- 172.21.15.60:0/3830150067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba7c114670 0x7fba7c1a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:13.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:13.769+0000 7fba7b400700 1 -- 172.21.15.60:0/3830150067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba7c114670 msgr2=0x7fba7c1a4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:13.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:13.769+0000 7fba7b400700 1 --2- 172.21.15.60:0/3830150067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba7c114670 0x7fba7c1a4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:13.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:13.769+0000 7fba7b400700 1 -- 172.21.15.60:0/3830150067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba7c075ff0 msgr2=0x7fba7c1a0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:13.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:13.769+0000 7fba7b400700 1 --2- 172.21.15.60:0/3830150067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba7c075ff0 0x7fba7c1a0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:13.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:13.769+0000 7fba7b400700 1 -- 172.21.15.60:0/3830150067 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba7c1a5350 con 0x7fba7c0756d0 2024-05-03T14:49:13.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:13.769+0000 7fba7b400700 1 --2- 172.21.15.60:0/3830150067 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba7c0756d0 0x7fba7c1a0340 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fba6c000c00 tx=0x7fba6c005590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:49:13.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:13.770+0000 7fba78c00700 1 -- 172.21.15.60:0/3830150067 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6c027070 con 0x7fba7c0756d0 2024-05-03T14:49:13.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:13.770+0000 7fba82600700 1 -- 172.21.15.60:0/3830150067 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba7c1a55e0 con 0x7fba7c0756d0 2024-05-03T14:49:13.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:13.770+0000 7fba78c00700 1 -- 172.21.15.60:0/3830150067 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7fba6c00fdc0 con 0x7fba7c0756d0 2024-05-03T14:49:13.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:13.770+0000 7fba78c00700 1 -- 172.21.15.60:0/3830150067 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6c0173f0 con 0x7fba7c0756d0 2024-05-03T14:49:13.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:13.770+0000 7fba82600700 1 -- 172.21.15.60:0/3830150067 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba7c1acec0 con 0x7fba7c0756d0 2024-05-03T14:49:13.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:13.771+0000 7fba82600700 1 -- 172.21.15.60:0/3830150067 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba7c06a190 con 0x7fba7c0756d0 2024-05-03T14:49:13.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:13.774+0000 7fba78c00700 1 -- 172.21.15.60:0/3830150067 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fba6c02a070 con 0x7fba7c0756d0 2024-05-03T14:49:13.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:13.775+0000 7fba78c00700 1 --2- 172.21.15.60:0/3830150067 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fba600b2990 0x7fba600b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:13.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:13.775+0000 7fba78c00700 1 -- 172.21.15.60:0/3830150067 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7fba6c0e33c0 con 0x7fba7c0756d0 2024-05-03T14:49:13.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:13.775+0000 7fba72a00700 1 --2- 172.21.15.60:0/3830150067 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fba600b2990 0x7fba600b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:13.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:13.776+0000 7fba72a00700 1 --2- 172.21.15.60:0/3830150067 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fba600b2990 0x7fba600b4e50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fba7c110f90 tx=0x7fba6400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:49:13.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:13.779+0000 7fba78c00700 1 -- 172.21.15.60:0/3830150067 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fba6c013070 con 0x7fba7c0756d0 2024-05-03T14:49:13.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:49:13 smithi090 ceph-mon[33568]: pgmap v473: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T14:49:13.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:49:13 smithi149 ceph-mon[33332]: pgmap v473: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T14:49:14.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:14.018+0000 7fba82600700 1 -- 172.21.15.60:0/3830150067 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fba7c1ad090 con 0x7fba7c0756d0 2024-05-03T14:49:14.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:14.020+0000 7fba78c00700 1 -- 172.21.15.60:0/3830150067 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24674 (secure 0 0 0) 0x7fba6c0aac30 con 0x7fba7c0756d0 2024-05-03T14:49:14.022 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T14:49:14.023 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":89,"fsid":"285d64b2-095a-11ef-bc97-c7b262605968","created":"2024-05-03T14:34:53.560460+0000","modified":"2024-05-03T14:46:05.582395+0000","last_up_change":"2024-05-03T14:43:14.877043+0000","last_in_change":"2024-05-03T14:42:55.085959+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-03T14:38:54.333714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-03T14:44:25.608599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-03T14:44:30.154009+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-03T14:44:42.549502+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f0797730-c4c1-47ea-9dce-9270d5ebb4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2952930162},{"type":"v1","addr":"172.21.15.60:6803","nonce":2952930162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2952930162},{"type":"v1","addr":"172.21.15.60:6805","nonce":2952930162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":2952930162},{"type":"v1","addr":"172.21.15.60:6809","nonce":2952930162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2952930162},{"type":"v1","addr":"172.21.15.60:6807","nonce":2952930162}]},"public_addr":"172.21.15.60:6803/2952930162","cluster_addr":"172.21.15.60:6805/2952930162","heartbeat_back_addr":"172.21.15.60:6809/2952930162","heartbeat_front_addr":"172.21.15.60:6807/2952930162","state":["exists","up"]},{"osd":1,"uuid":"96d0ad08-cda4-4824-bad4-e64973508c0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":720063242},{"type":"v1","addr":"172.21.15.90:6801","nonce":720063242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":720063242},{"type":"v1","addr":"172.21.15.90:6803","nonce":720063242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":720063242},{"type":"v1","addr":"172.21.15.90:6807","nonce":720063242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":720063242},{"type":"v1","addr":"172.21.15.90:6805","nonce":720063242}]},"public_addr":"172.21.15.90:6801/720063242","cluster_addr":"172.21.15.90:6803/720063242","heartbeat_back_addr":"172.21.15.90:6807/720063242","heartbeat_front_addr":"172.21.15.90:6805/720063242","state":["exists","up"]},{"osd":2,"uuid":"ce95fb7c-1611-4dd1-aee7-97c420d6a466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1359662077},{"type":"v1","addr":"172.21.15.149:6801","nonce":1359662077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1359662077},{"type":"v1","addr":"172.21.15.149:6803","nonce":1359662077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1359662077},{"type":"v1","addr":"172.21.15.149:6807","nonce":1359662077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1359662077},{"type":"v1","addr":"172.21.15.149:6805","nonce":1359662077}]},"public_addr":"172.21.15.149:6801/1359662077","cluster_addr":"172.21.15.149:6803/1359662077","heartbeat_back_addr":"172.21.15.149:6807/1359662077","heartbeat_front_addr":"172.21.15.149:6805/1359662077","state":["exists","up"]},{"osd":3,"uuid":"ef31136a-9b36-4b87-8389-f53d311bc5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3226133334},{"type":"v1","addr":"172.21.15.60:6811","nonce":3226133334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3226133334},{"type":"v1","addr":"172.21.15.60:6813","nonce":3226133334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3226133334},{"type":"v1","addr":"172.21.15.60:6817","nonce":3226133334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3226133334},{"type":"v1","addr":"172.21.15.60:6815","nonce":3226133334}]},"public_addr":"172.21.15.60:6811/3226133334","cluster_addr":"172.21.15.60:6813/3226133334","heartbeat_back_addr":"172.21.15.60:6817/3226133334","heartbeat_front_addr":"172.21.15.60:6815/3226133334","state":["exists","up"]},{"osd":4,"uuid":"73db7408-a2ff-47e9-b13d-d31688a93610","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2201391883},{"type":"v1","addr":"172.21.15.90:6809","nonce":2201391883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":2201391883},{"type":"v1","addr":"172.21.15.90:6811","nonce":2201391883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":2201391883},{"type":"v1","addr":"172.21.15.90:6815","nonce":2201391883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":2201391883},{"type":"v1","addr":"172.21.15.90:6813","nonce":2201391883}]},"public_addr":"172.21.15.90:6809/2201391883","cluster_addr":"172.21.15.90:6811/2201391883","heartbeat_back_addr":"172.21.15.90:6815/2201391883","heartbeat_front_addr":"172.21.15.90:6813/2201391883","state":["exists","up"]},{"osd":5,"uuid":"aefb78a2-522a-4b35-92b8-e5c04fa47b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2139732836},{"type":"v1","addr":"172.21.15.149:6809","nonce":2139732836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":2139732836},{"type":"v1","addr":"172.21.15.149:6811","nonce":2139732836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":2139732836},{"type":"v1","addr":"172.21.15.149:6815","nonce":2139732836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":2139732836},{"type":"v1","addr":"172.21.15.149:6813","nonce":2139732836}]},"public_addr":"172.21.15.149:6809/2139732836","cluster_addr":"172.21.15.149:6811/2139732836","heartbeat_back_addr":"172.21.15.149:6815/2139732836","heartbeat_front_addr":"172.21.15.149:6813/2139732836","state":["exists","up"]},{"osd":6,"uuid":"7c887819-6182-4636-8cfd-d845c30bd13c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3738521002},{"type":"v1","addr":"172.21.15.60:6819","nonce":3738521002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3738521002},{"type":"v1","addr":"172.21.15.60:6821","nonce":3738521002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3738521002},{"type":"v1","addr":"172.21.15.60:6825","nonce":3738521002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3738521002},{"type":"v1","addr":"172.21.15.60:6823","nonce":3738521002}]},"public_addr":"172.21.15.60:6819/3738521002","cluster_addr":"172.21.15.60:6821/3738521002","heartbeat_back_addr":"172.21.15.60:6825/3738521002","heartbeat_front_addr":"172.21.15.60:6823/3738521002","state":["exists","up"]},{"osd":7,"uuid":"ebce6acf-8873-4b35-bd5f-80de2163a307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":1686298661},{"type":"v1","addr":"172.21.15.90:6817","nonce":1686298661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":1686298661},{"type":"v1","addr":"172.21.15.90:6819","nonce":1686298661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":1686298661},{"type":"v1","addr":"172.21.15.90:6823","nonce":1686298661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1686298661},{"type":"v1","addr":"172.21.15.90:6821","nonce":1686298661}]},"public_addr":"172.21.15.90:6817/1686298661","cluster_addr":"172.21.15.90:6819/1686298661","heartbeat_back_addr":"172.21.15.90:6823/1686298661","heartbeat_front_addr":"172.21.15.90:6821/1686298661","state":["exists","up"]},{"osd":8,"uuid":"4ae04b98-e9b2-4956-83d9-17e5ac3c963f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":2608348248},{"type":"v1","addr":"172.21.15.149:6817","nonce":2608348248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2608348248},{"type":"v1","addr":"172.21.15.149:6819","nonce":2608348248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2608348248},{"type":"v1","addr":"172.21.15.149:6823","nonce":2608348248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2608348248},{"type":"v1","addr":"172.21.15.149:6821","nonce":2608348248}]},"public_addr":"172.21.15.149:6817/2608348248","cluster_addr":"172.21.15.149:6819/2608348248","heartbeat_back_addr":"172.21.15.149:6823/2608348248","heartbeat_front_addr":"172.21.15.149:6821/2608348248","state":["exists","up"]},{"osd":9,"uuid":"d0c0a70d-aa76-47e4-91e8-c343c8c41197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":2156130999},{"type":"v1","addr":"172.21.15.60:6827","nonce":2156130999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":2156130999},{"type":"v1","addr":"172.21.15.60:6829","nonce":2156130999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":2156130999},{"type":"v1","addr":"172.21.15.60:6833","nonce":2156130999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":2156130999},{"type":"v1","addr":"172.21.15.60:6831","nonce":2156130999}]},"public_addr":"172.21.15.60:6827/2156130999","cluster_addr":"172.21.15.60:6829/2156130999","heartbeat_back_addr":"172.21.15.60:6833/2156130999","heartbeat_front_addr":"172.21.15.60:6831/2156130999","state":["exists","up"]},{"osd":10,"uuid":"cdc3d7b8-33c7-4cc4-be4e-6887d15a3d8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":2492593590},{"type":"v1","addr":"172.21.15.90:6825","nonce":2492593590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":2492593590},{"type":"v1","addr":"172.21.15.90:6827","nonce":2492593590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":2492593590},{"type":"v1","addr":"172.21.15.90:6831","nonce":2492593590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":2492593590},{"type":"v1","addr":"172.21.15.90:6829","nonce":2492593590}]},"public_addr":"172.21.15.90:6825/2492593590","cluster_addr":"172.21.15.90:6827/2492593590","heartbeat_back_addr":"172.21.15.90:6831/2492593590","heartbeat_front_addr":"172.21.15.90:6829/2492593590","state":["exists","up"]},{"osd":11,"uuid":"8d17ec1a-bdd5-41f1-9b59-c2b6a527bfbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":1063245126},{"type":"v1","addr":"172.21.15.149:6825","nonce":1063245126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":1063245126},{"type":"v1","addr":"172.21.15.149:6827","nonce":1063245126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":1063245126},{"type":"v1","addr":"172.21.15.149:6831","nonce":1063245126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":1063245126},{"type":"v1","addr":"172.21.15.149:6829","nonce":1063245126}]},"public_addr":"172.21.15.149:6825/1063245126","cluster_addr":"172.21.15.149:6827/1063245126","heartbeat_back_addr":"172.21.15.149:6831/1063245126","heartbeat_front_addr":"172.21.15.149:6829/1063245126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:38:02.056563+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:38:27.140677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:38:51.287070+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:39:25.484410+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:39:51.483133+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:40:17.200423+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:40:51.979804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:41:18.742542+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:41:44.922541+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:42:19.778451+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:42:47.438440+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:43:13.429419+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6801/424391417":"2024-05-04T14:36:04.154333+0000","172.21.15.60:6800/424391417":"2024-05-04T14:36:04.154333+0000","172.21.15.60:0/1426532561":"2024-05-04T14:36:04.154333+0000","172.21.15.60:0/3230079926":"2024-05-04T14:35:28.066961+0000","172.21.15.60:0/2334895610":"2024-05-04T14:35:28.066961+0000","172.21.15.60:6800/888447185":"2024-05-04T14:35:28.066961+0000","172.21.15.60:0/3996899937":"2024-05-04T14:35:28.066961+0000","172.21.15.60:0/2779644018":"2024-05-04T14:36:04.154333+0000","172.21.15.60:0/71440838":"2024-05-04T14:36:04.154333+0000","172.21.15.60:6801/888447185":"2024-05-04T14:35:28.066961+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-03T14:49:14.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:14.023+0000 7fba82600700 1 -- 172.21.15.60:0/3830150067 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fba600b2990 msgr2=0x7fba600b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:14.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:14.023+0000 7fba82600700 1 --2- 172.21.15.60:0/3830150067 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fba600b2990 0x7fba600b4e50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fba7c110f90 tx=0x7fba6400b040 comp rx=0 tx=0).stop 2024-05-03T14:49:14.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:14.023+0000 7fba82600700 1 -- 172.21.15.60:0/3830150067 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba7c0756d0 msgr2=0x7fba7c1a0340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:14.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:14.023+0000 7fba82600700 1 --2- 172.21.15.60:0/3830150067 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba7c0756d0 0x7fba7c1a0340 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fba6c000c00 tx=0x7fba6c005590 comp rx=0 tx=0).stop 2024-05-03T14:49:14.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:14.023+0000 7fba82600700 1 -- 172.21.15.60:0/3830150067 shutdown_connections 2024-05-03T14:49:14.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:14.023+0000 7fba82600700 1 --2- 172.21.15.60:0/3830150067 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fba600b2990 0x7fba600b4e50 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:14.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:14.023+0000 7fba82600700 1 --2- 172.21.15.60:0/3830150067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba7c114670 0x7fba7c1a4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:14.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:14.023+0000 7fba82600700 1 --2- 172.21.15.60:0/3830150067 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba7c0756d0 0x7fba7c1a0340 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:14.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:14.023+0000 7fba82600700 1 --2- 172.21.15.60:0/3830150067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba7c075ff0 0x7fba7c1a0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:14.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:14.023+0000 7fba82600700 1 -- 172.21.15.60:0/3830150067 >> 172.21.15.60:0/3830150067 conn(0x7fba7c07a600 msgr2=0x7fba7c113430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:49:14.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:14.023+0000 7fba82600700 1 -- 172.21.15.60:0/3830150067 shutdown_connections 2024-05-03T14:49:14.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:14.023+0000 7fba82600700 1 -- 172.21.15.60:0/3830150067 wait complete. 2024-05-03T14:49:14.744 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:49:14 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/3830150067' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T14:49:14.816 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-03T14:44:34.226052+0000', 'modified': '2024-05-03T14:46:01.367913+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24511, 'mds_1': 24509, 'mds_2': 24487, 'mds_3': 24475, 'mds_4': 24470}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14397': {'gid': 14397, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.60:6839/3846205424', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6838', 'nonce': 3846205424}, {'type': 'v1', 'addr': '172.21.15.60:6839', 'nonce': 3846205424}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24425': {'gid': 24425, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.90:6833/3033424631', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.90:6832', 'nonce': 3033424631}, {'type': 'v1', 'addr': '172.21.15.90:6833', 'nonce': 3033424631}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24427': {'gid': 24427, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.149:6833/3454021291', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6832', 'nonce': 3454021291}, {'type': 'v1', 'addr': '172.21.15.149:6833', 'nonce': 3454021291}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24455': {'gid': 24455, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.90:6835/1057049585', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.90:6834', 'nonce': 1057049585}, {'type': 'v1', 'addr': '172.21.15.90:6835', 'nonce': 1057049585}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.149:6835/882239924', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6834', 'nonce': 882239924}, {'type': 'v1', 'addr': '172.21.15.149:6835', 'nonce': 882239924}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24470': {'gid': 24470, 'name': 'h', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 31, 'addr': '172.21.15.90:6837/2243702844', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.90:6836', 'nonce': 2243702844}, {'type': 'v1', 'addr': '172.21.15.90:6837', 'nonce': 2243702844}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24475': {'gid': 24475, 'name': 'c', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 32, 'addr': '172.21.15.149:6837/2611278690', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6836', 'nonce': 2611278690}, {'type': 'v1', 'addr': '172.21.15.149:6837', 'nonce': 2611278690}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 30, 'addr': '172.21.15.149:6839/2527622212', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6838', 'nonce': 2527622212}, {'type': 'v1', 'addr': '172.21.15.149:6839', 'nonce': 2527622212}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24509': {'gid': 24509, 'name': 'b', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 26, 'addr': '172.21.15.90:6839/3512381172', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.90:6838', 'nonce': 3512381172}, {'type': 'v1', 'addr': '172.21.15.90:6839', 'nonce': 3512381172}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24511': {'gid': 24511, 'name': 'a', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.60:6841/924163182', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6840', 'nonce': 924163182}, {'type': 'v1', 'addr': '172.21.15.60:6841', 'nonce': 924163182}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24511, 'qdb_cluster': [24470, 24475, 24487, 24509, 24511]} 2024-05-03T14:49:14.816 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-03T14:49:14.816 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph tell mds.h status 2024-05-03T14:49:14.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:49:14 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/3830150067' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T14:49:14.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:49:14 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/3830150067' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T14:49:15.294 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:49:15.782 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:49:15 smithi060 ceph-mon[28873]: pgmap v474: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:49:15.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:49:15 smithi090 ceph-mon[33568]: pgmap v474: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:49:15.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:49:15 smithi149 ceph-mon[33332]: pgmap v474: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:49:17.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.556+0000 7fc232600700 1 -- 172.21.15.60:0/2833189536 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc22c1055d0 msgr2=0x7fc22c105a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:17.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.556+0000 7fc232600700 1 --2- 172.21.15.60:0/2833189536 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc22c1055d0 0x7fc22c105a50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc218009990 tx=0x7fc218009ca0 comp rx=0 tx=0).stop 2024-05-03T14:49:17.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.556+0000 7fc232600700 1 -- 172.21.15.60:0/2833189536 shutdown_connections 2024-05-03T14:49:17.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.556+0000 7fc232600700 1 --2- 172.21.15.60:0/2833189536 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc22c105f90 0x7fc22c1124a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:17.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.556+0000 7fc232600700 1 --2- 172.21.15.60:0/2833189536 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc22c10b660 0x7fc22c10ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:17.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.556+0000 7fc232600700 1 --2- 172.21.15.60:0/2833189536 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc22c1055d0 0x7fc22c105a50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:17.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.556+0000 7fc232600700 1 -- 172.21.15.60:0/2833189536 >> 172.21.15.60:0/2833189536 conn(0x7fc22c1010c0 msgr2=0x7fc22c1034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:49:17.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.556+0000 7fc232600700 1 -- 172.21.15.60:0/2833189536 shutdown_connections 2024-05-03T14:49:17.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.556+0000 7fc232600700 1 -- 172.21.15.60:0/2833189536 wait complete. 2024-05-03T14:49:17.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.558+0000 7fc232600700 1 Processor -- start 2024-05-03T14:49:17.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.558+0000 7fc232600700 1 -- start start 2024-05-03T14:49:17.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.558+0000 7fc232600700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc22c1055d0 0x7fc22c19bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:17.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.558+0000 7fc232600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc22c105f90 0x7fc22c19c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:17.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.558+0000 7fc232600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc22c10b660 0x7fc22c1a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:17.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.558+0000 7fc232600700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc22c115a40 con 0x7fc22c1055d0 2024-05-03T14:49:17.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.559+0000 7fc232600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc22c1158c0 con 0x7fc22c10b660 2024-05-03T14:49:17.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.559+0000 7fc232600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc22c115bc0 con 0x7fc22c105f90 2024-05-03T14:49:17.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.559+0000 7fc231200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc22c1055d0 0x7fc22c19bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:17.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.559+0000 7fc22be00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc22c105f90 0x7fc22c19c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:17.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.559+0000 7fc22be00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc22c105f90 0x7fc22c19c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.60:57210/0 (socket says 172.21.15.60:57210) 2024-05-03T14:49:17.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.559+0000 7fc22be00700 1 -- 172.21.15.60:0/1931960985 learned_addr learned my addr 172.21.15.60:0/1931960985 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:49:17.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.559+0000 7fc231200700 1 --2- 172.21.15.60:0/1931960985 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc22c1055d0 0x7fc22c19bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44614/0 (socket says 172.21.15.60:44614) 2024-05-03T14:49:17.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.559+0000 7fc231c00700 1 --2- 172.21.15.60:0/1931960985 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc22c10b660 0x7fc22c1a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:17.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.559+0000 7fc231200700 1 -- 172.21.15.60:0/1931960985 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc22c105f90 msgr2=0x7fc22c19c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:17.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.559+0000 7fc231200700 1 --2- 172.21.15.60:0/1931960985 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc22c105f90 0x7fc22c19c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:17.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.559+0000 7fc231200700 1 -- 172.21.15.60:0/1931960985 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc22c10b660 msgr2=0x7fc22c1a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:17.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.559+0000 7fc231200700 1 --2- 172.21.15.60:0/1931960985 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc22c10b660 0x7fc22c1a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:17.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.559+0000 7fc231200700 1 -- 172.21.15.60:0/1931960985 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc22c1a0d70 con 0x7fc22c1055d0 2024-05-03T14:49:17.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.560+0000 7fc231200700 1 --2- 172.21.15.60:0/1931960985 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc22c1055d0 0x7fc22c19bdb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc22000e9c0 tx=0x7fc22000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:49:17.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.560+0000 7fc229600700 1 -- 172.21.15.60:0/1931960985 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc220004220 con 0x7fc22c1055d0 2024-05-03T14:49:17.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.560+0000 7fc229600700 1 -- 172.21.15.60:0/1931960985 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7fc2200043c0 con 0x7fc22c1055d0 2024-05-03T14:49:17.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.560+0000 7fc232600700 1 -- 172.21.15.60:0/1931960985 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc22c1a1060 con 0x7fc22c1055d0 2024-05-03T14:49:17.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.560+0000 7fc229600700 1 -- 172.21.15.60:0/1931960985 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc220003dc0 con 0x7fc22c1055d0 2024-05-03T14:49:17.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.561+0000 7fc232600700 1 -- 172.21.15.60:0/1931960985 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc22c1a88e0 con 0x7fc22c1055d0 2024-05-03T14:49:17.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.564+0000 7fc229600700 1 -- 172.21.15.60:0/1931960985 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fc22000c7c0 con 0x7fc22c1055d0 2024-05-03T14:49:17.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.565+0000 7fc23e4e2b80 1 Processor -- start 2024-05-03T14:49:17.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.565+0000 7fc23e4e2b80 1 -- start start 2024-05-03T14:49:17.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.566+0000 7fc23e4e2b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612f3040b80 0x5612f3040f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:17.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.566+0000 7fc23e4e2b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612f303aee0 0x5612f303b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:17.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.566+0000 7fc229600700 1 --2- 172.21.15.60:0/1931960985 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fc2140b2990 0x7fc2140b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:17.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.566+0000 7fc23e4e2b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5612f303b8a0 0x5612f3047940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:17.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.566+0000 7fc23e4e2b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5612f2f8d9d0 con 0x5612f303b8a0 2024-05-03T14:49:17.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.566+0000 7fc23e4e2b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5612f2f2b710 con 0x5612f303aee0 2024-05-03T14:49:17.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.566+0000 7fc23e4e2b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5612f2f89990 con 0x5612f3040b80 2024-05-03T14:49:17.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.566+0000 7fc229600700 1 -- 172.21.15.60:0/1931960985 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7fc2200e2960 con 0x7fc22c1055d0 2024-05-03T14:49:17.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.566+0000 7fc231c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612f3040b80 0x5612f3040f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:17.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.566+0000 7fc22be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612f303aee0 0x5612f303b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:17.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.566+0000 7fc231c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612f3040b80 0x5612f3040f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.60:57212/0 (socket says 172.21.15.60:57212) 2024-05-03T14:49:17.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.566+0000 7fc231c00700 1 -- 172.21.15.60:0/3636989033 learned_addr learned my addr 172.21.15.60:0/3636989033 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:49:17.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.566+0000 7fc231200700 1 --2- 172.21.15.60:0/1931960985 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fc2140b2990 0x7fc2140b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:17.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.567+0000 7fc231c00700 1 --2- 172.21.15.60:0/3636989033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5612f303b8a0 0x5612f3047940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:17.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.567+0000 7fc22be00700 1 -- 172.21.15.60:0/3636989033 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612f3040b80 msgr2=0x5612f3040f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:49:17.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.567+0000 7fc22be00700 1 --2- 172.21.15.60:0/3636989033 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612f3040b80 0x5612f3040f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:17.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.567+0000 7fc22be00700 1 -- 172.21.15.60:0/3636989033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5612f303b8a0 msgr2=0x5612f3047940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:49:17.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.567+0000 7fc22be00700 1 --2- 172.21.15.60:0/3636989033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5612f303b8a0 0x5612f3047940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:17.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.567+0000 7fc22be00700 1 -- 172.21.15.60:0/3636989033 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612f2eda880 con 0x5612f303aee0 2024-05-03T14:49:17.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.567+0000 7fc231c00700 1 --2- 172.21.15.60:0/3636989033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5612f303b8a0 0x5612f3047940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T14:49:17.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.567+0000 7fc22be00700 1 --2- 172.21.15.60:0/3636989033 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612f303aee0 0x5612f303b360 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc22c10abc0 tx=0x7fc218005c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ddb3a52903817be server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:49:17.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.567+0000 7fc231200700 1 --2- 172.21.15.60:0/1931960985 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fc2140b2990 0x7fc2140b4e50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc22000f010 tx=0x7fc220009ef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:49:17.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.568+0000 7fc21e000700 1 -- 172.21.15.60:0/3636989033 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc218027070 con 0x5612f303aee0 2024-05-03T14:49:17.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.568+0000 7fc21e000700 1 -- 172.21.15.60:0/3636989033 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7fc21800b6d0 con 0x5612f303aee0 2024-05-03T14:49:17.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.568+0000 7fc21e000700 1 -- 172.21.15.60:0/3636989033 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc218003e60 con 0x5612f303aee0 2024-05-03T14:49:17.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.568+0000 7fc23e4e2b80 1 -- 172.21.15.60:0/3636989033 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612f303aee0 msgr2=0x5612f303b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:17.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.568+0000 7fc23e4e2b80 1 --2- 172.21.15.60:0/3636989033 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612f303aee0 0x5612f303b360 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc22c10abc0 tx=0x7fc218005c40 comp rx=0 tx=0).stop 2024-05-03T14:49:17.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.569+0000 7fc23e4e2b80 1 -- 172.21.15.60:0/3636989033 shutdown_connections 2024-05-03T14:49:17.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.569+0000 7fc23e4e2b80 1 --2- 172.21.15.60:0/3636989033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5612f303b8a0 0x5612f3047940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:17.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.569+0000 7fc23e4e2b80 1 --2- 172.21.15.60:0/3636989033 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612f3040b80 0x5612f3040f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:17.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.569+0000 7fc23e4e2b80 1 --2- 172.21.15.60:0/3636989033 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612f303aee0 0x5612f303b360 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:17.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.569+0000 7fc23e4e2b80 1 -- 172.21.15.60:0/3636989033 >> 172.21.15.60:0/3636989033 conn(0x5612f2fb3680 msgr2=0x5612f3035b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:49:17.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.569+0000 7fc23e4e2b80 1 -- 172.21.15.60:0/3636989033 shutdown_connections 2024-05-03T14:49:17.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.569+0000 7fc23e4e2b80 1 -- 172.21.15.60:0/3636989033 wait complete. 2024-05-03T14:49:17.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.569+0000 7fc23e4e2b80 1 -- start start 2024-05-03T14:49:17.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.570+0000 7fc23e4e2b80 1 Processor -- start 2024-05-03T14:49:17.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.570+0000 7fc23e4e2b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612f303aee0 0x5612f3047510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:17.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.570+0000 7fc23e4e2b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5612f303b8a0 0x5612f30448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:17.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.570+0000 7fc23e4e2b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612f3040b80 0x5612f3044df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:17.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.570+0000 7fc23e4e2b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5612f3047e80 con 0x5612f303b8a0 2024-05-03T14:49:17.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.570+0000 7fc23e4e2b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5612f2ef3ad0 con 0x5612f303aee0 2024-05-03T14:49:17.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.570+0000 7fc23e4e2b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5612f3048000 con 0x5612f3040b80 2024-05-03T14:49:17.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.570+0000 7fc23e4e2b80 10 client.0 _ll_register_callbacks cb 0x5612f2f34c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T14:49:17.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.570+0000 7fc231c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612f303aee0 0x5612f3047510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:17.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.570+0000 7fc231c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612f303aee0 0x5612f3047510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.60:38804/0 (socket says 172.21.15.60:38804) 2024-05-03T14:49:17.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.570+0000 7fc231c00700 1 -- 172.21.15.60:0/1644301399 learned_addr learned my addr 172.21.15.60:0/1644301399 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:49:17.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.571+0000 7fc231c00700 1 --2- 172.21.15.60:0/1644301399 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612f3040b80 0x5612f3044df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:17.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.571+0000 7fc22be00700 1 --2- 172.21.15.60:0/1644301399 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5612f303b8a0 0x5612f30448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:17.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.571+0000 7fc231c00700 1 -- 172.21.15.60:0/1644301399 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612f3040b80 msgr2=0x5612f3044df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:49:17.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.571+0000 7fc231c00700 1 --2- 172.21.15.60:0/1644301399 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612f3040b80 0x5612f3044df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:17.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.571+0000 7fc231c00700 1 -- 172.21.15.60:0/1644301399 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5612f303b8a0 msgr2=0x5612f30448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:49:17.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.571+0000 7fc231c00700 1 --2- 172.21.15.60:0/1644301399 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5612f303b8a0 0x5612f30448b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:17.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.571+0000 7fc231c00700 1 -- 172.21.15.60:0/1644301399 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612f2f533d0 con 0x5612f303aee0 2024-05-03T14:49:17.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.571+0000 7fc231c00700 1 --2- 172.21.15.60:0/1644301399 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612f303aee0 0x5612f3047510 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc22401c6c0 tx=0x7fc22400b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dab21176566c2092 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:49:17.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.571+0000 7fc212a00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-03T14:49:17.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.571+0000 7fc212a00700 1 -- 172.21.15.60:0/1644301399 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc224003a00 con 0x5612f303aee0 2024-05-03T14:49:17.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.572+0000 7fc212a00700 1 -- 172.21.15.60:0/1644301399 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7fc224003ba0 con 0x5612f303aee0 2024-05-03T14:49:17.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.572+0000 7fc23e4e2b80 1 -- 172.21.15.60:0/1644301399 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5612f3007fa0 con 0x5612f303aee0 2024-05-03T14:49:17.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.572+0000 7fc212a00700 1 -- 172.21.15.60:0/1644301399 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc224003da0 con 0x5612f303aee0 2024-05-03T14:49:17.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.572+0000 7fc23e4e2b80 10 client.25100 fetch_fsmap learned FSMap version 29 2024-05-03T14:49:17.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.572+0000 7fc23e4e2b80 1 -- 172.21.15.60:0/1644301399 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5612f30526b0 con 0x5612f303aee0 2024-05-03T14:49:17.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.572+0000 7fc212a00700 1 -- 172.21.15.60:0/1644301399 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x5612f3007fa0 con 0x5612f303aee0 2024-05-03T14:49:17.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.572+0000 7fc212a00700 1 -- 172.21.15.60:0/1644301399 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc22401a610 con 0x5612f303aee0 2024-05-03T14:49:17.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.572+0000 7fc23e4e2b80 10 client.25100 fetch_fsmap finished waiting for FSMap version 29 2024-05-03T14:49:17.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.573+0000 7fc23e4e2b80 10 client.25100 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-03T14:49:17.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.573+0000 7fc23e4e2b80 20 client.25100 populate_metadata read hostname 'smithi060' 2024-05-03T14:49:17.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.573+0000 7fc23e4e2b80 1 --2- 172.21.15.60:0/1644301399 >> [v2:172.21.15.90:6836/2243702844,v1:172.21.15.90:6837/2243702844] conn(0x5612f3054c90 0x5612f3056100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:17.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.573+0000 7fc23e4e2b80 4 client.25100 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T14:49:17.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.573+0000 7fc23e4e2b80 1 -- 172.21.15.60:0/1644301399 --> [v2:172.21.15.90:6836/2243702844,v1:172.21.15.90:6837/2243702844] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612f2f9b3b0 con 0x5612f3054c90 2024-05-03T14:49:17.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.573+0000 7fc22be00700 1 --2- 172.21.15.60:0/1644301399 >> [v2:172.21.15.90:6836/2243702844,v1:172.21.15.90:6837/2243702844] conn(0x5612f3054c90 0x5612f3056100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:17.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.573+0000 7fc22be00700 1 --2- 172.21.15.60:0/1644301399 >> [v2:172.21.15.90:6836/2243702844,v1:172.21.15.90:6837/2243702844] conn(0x5612f3054c90 0x5612f3056100 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e3b2cd8c50e1ca2a server_cookie=6c7bbf0d2e8a9847 in_seq=0 out_seq=0 2024-05-03T14:49:17.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.573+0000 7fc212a00700 10 client.25100 ms_handle_connect on v2:172.21.15.90:6836/2243702844 2024-05-03T14:49:17.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.574+0000 7fc212a00700 1 -- 172.21.15.60:0/1644301399 <== mds.4 v2:172.21.15.90:6836/2243702844 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5612f2f9b3b0 con 0x5612f3054c90 2024-05-03T14:49:17.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.574+0000 7fc212a00700 10 client.25100 handle_command_reply: tid=0 2024-05-03T14:49:17.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.574+0000 7fc23e4e2b80 1 client.25100 shutdown 2024-05-03T14:49:17.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.575+0000 7fc23e4e2b80 1 -- 172.21.15.60:0/1644301399 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612f303aee0 msgr2=0x5612f3047510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:17.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.575+0000 7fc23e4e2b80 1 --2- 172.21.15.60:0/1644301399 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612f303aee0 0x5612f3047510 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc22401c6c0 tx=0x7fc22400b040 comp rx=0 tx=0).stop 2024-05-03T14:49:17.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.575+0000 7fc23e4e2b80 1 -- 172.21.15.60:0/1644301399 shutdown_connections 2024-05-03T14:49:17.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.575+0000 7fc23e4e2b80 1 --2- 172.21.15.60:0/1644301399 >> [v2:172.21.15.90:6836/2243702844,v1:172.21.15.90:6837/2243702844] conn(0x5612f3054c90 0x5612f3056100 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:17.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.575+0000 7fc23e4e2b80 1 --2- 172.21.15.60:0/1644301399 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612f3040b80 0x5612f3044df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:17.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.575+0000 7fc23e4e2b80 1 --2- 172.21.15.60:0/1644301399 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612f303aee0 0x5612f3047510 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:17.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.575+0000 7fc23e4e2b80 1 --2- 172.21.15.60:0/1644301399 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5612f303b8a0 0x5612f30448b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:17.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.575+0000 7fc212a00700 0 client.25100 ms_handle_reset on v2:172.21.15.90:6836/2243702844 2024-05-03T14:49:17.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.575+0000 7fc23e4e2b80 1 -- 172.21.15.60:0/1644301399 >> 172.21.15.60:0/1644301399 conn(0x5612f2fb3680 msgr2=0x5612f3039ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:49:17.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.575+0000 7fc23e4e2b80 1 -- 172.21.15.60:0/1644301399 shutdown_connections 2024-05-03T14:49:17.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.575+0000 7fc23e4e2b80 1 -- 172.21.15.60:0/1644301399 wait complete. 2024-05-03T14:49:17.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.575+0000 7fc23e4e2b80 20 client.25100 trim_cache size 0 max 16384 2024-05-03T14:49:17.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.598+0000 7fc23e4e2b80 1 Processor -- start 2024-05-03T14:49:17.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.599+0000 7fc23e4e2b80 1 -- start start 2024-05-03T14:49:17.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.599+0000 7fc23e4e2b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612f303aee0 0x5612f3030520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:17.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.599+0000 7fc23e4e2b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5612f303b8a0 0x5612f3046d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:17.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.599+0000 7fc23e4e2b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612f3040b80 0x5612f3053b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:17.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.599+0000 7fc23e4e2b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5612f3054440 con 0x5612f303b8a0 2024-05-03T14:49:17.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.599+0000 7fc23e4e2b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5612f3054b10 con 0x5612f303aee0 2024-05-03T14:49:17.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.599+0000 7fc23e4e2b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5612f30551e0 con 0x5612f3040b80 2024-05-03T14:49:17.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.599+0000 7fc22be00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5612f303b8a0 0x5612f3046d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:17.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.599+0000 7fc22be00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5612f303b8a0 0x5612f3046d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44642/0 (socket says 172.21.15.60:44642) 2024-05-03T14:49:17.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.599+0000 7fc22be00700 1 -- 172.21.15.60:0/3735347758 learned_addr learned my addr 172.21.15.60:0/3735347758 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:49:17.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.599+0000 7fc231c00700 1 --2- 172.21.15.60:0/3735347758 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612f303aee0 0x5612f3030520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:17.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.600+0000 7fc231c00700 1 --2- 172.21.15.60:0/3735347758 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612f3040b80 0x5612f3053b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:17.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.600+0000 7fc22be00700 1 -- 172.21.15.60:0/3735347758 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612f3040b80 msgr2=0x5612f3053b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:49:17.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.600+0000 7fc22be00700 1 --2- 172.21.15.60:0/3735347758 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612f3040b80 0x5612f3053b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:17.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.600+0000 7fc22be00700 1 -- 172.21.15.60:0/3735347758 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612f303aee0 msgr2=0x5612f3030520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:49:17.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.600+0000 7fc22be00700 1 --2- 172.21.15.60:0/3735347758 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612f303aee0 0x5612f3030520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:17.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.600+0000 7fc22be00700 1 -- 172.21.15.60:0/3735347758 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612f3007f80 con 0x5612f303b8a0 2024-05-03T14:49:17.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.600+0000 7fc231c00700 1 --2- 172.21.15.60:0/3735347758 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612f303aee0 0x5612f3030520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T14:49:17.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.600+0000 7fc22be00700 1 --2- 172.21.15.60:0/3735347758 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5612f303b8a0 0x5612f3046d50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc218025040 tx=0x7fc218005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d402cf2f38d89c31 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:49:17.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.601+0000 7fc21d600700 1 -- 172.21.15.60:0/3735347758 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc218035070 con 0x5612f303b8a0 2024-05-03T14:49:17.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.601+0000 7fc21d600700 1 -- 172.21.15.60:0/3735347758 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7fc21800bda0 con 0x5612f303b8a0 2024-05-03T14:49:17.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.601+0000 7fc21d600700 1 -- 172.21.15.60:0/3735347758 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc218027040 con 0x5612f303b8a0 2024-05-03T14:49:17.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.601+0000 7fc23e4e2b80 1 -- 172.21.15.60:0/3735347758 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5612f303b8a0 msgr2=0x5612f3046d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:17.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.601+0000 7fc23e4e2b80 1 --2- 172.21.15.60:0/3735347758 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5612f303b8a0 0x5612f3046d50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc218025040 tx=0x7fc218005040 comp rx=0 tx=0).stop 2024-05-03T14:49:17.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.601+0000 7fc23e4e2b80 1 -- 172.21.15.60:0/3735347758 shutdown_connections 2024-05-03T14:49:17.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.601+0000 7fc23e4e2b80 1 --2- 172.21.15.60:0/3735347758 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612f3040b80 0x5612f3053b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:17.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.601+0000 7fc23e4e2b80 1 --2- 172.21.15.60:0/3735347758 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612f303aee0 0x5612f3030520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:17.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.601+0000 7fc23e4e2b80 1 --2- 172.21.15.60:0/3735347758 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5612f303b8a0 0x5612f3046d50 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:17.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.601+0000 7fc23e4e2b80 1 -- 172.21.15.60:0/3735347758 >> 172.21.15.60:0/3735347758 conn(0x5612f2fb3680 msgr2=0x5612f302f300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:49:17.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.602+0000 7fc23e4e2b80 1 -- 172.21.15.60:0/3735347758 shutdown_connections 2024-05-03T14:49:17.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.602+0000 7fc23e4e2b80 1 -- 172.21.15.60:0/3735347758 wait complete. 2024-05-03T14:49:17.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.602+0000 7fc23e4e2b80 1 -- start start 2024-05-03T14:49:17.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.602+0000 7fc23e4e2b80 1 Processor -- start 2024-05-03T14:49:17.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.603+0000 7fc23e4e2b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612f303aee0 0x5612f305fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:17.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.603+0000 7fc23e4e2b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5612f303b8a0 0x5612f3045d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:17.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.603+0000 7fc23e4e2b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612f3040b80 0x5612f30462a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:17.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.603+0000 7fc23e4e2b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5612f3055f80 con 0x5612f303b8a0 2024-05-03T14:49:17.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.603+0000 7fc23e4e2b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5612f30558b0 con 0x5612f303aee0 2024-05-03T14:49:17.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.603+0000 7fc23e4e2b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5612f3056650 con 0x5612f3040b80 2024-05-03T14:49:17.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.603+0000 7fc23e4e2b80 10 client.0 _ll_register_callbacks cb 0x5612f2f34c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T14:49:17.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.603+0000 7fc22be00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5612f303b8a0 0x5612f3045d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:17.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.603+0000 7fc231c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612f303aee0 0x5612f305fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:17.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.603+0000 7fc22be00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5612f303b8a0 0x5612f3045d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44658/0 (socket says 172.21.15.60:44658) 2024-05-03T14:49:17.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.603+0000 7fc22be00700 1 -- 172.21.15.60:0/1162351562 learned_addr learned my addr 172.21.15.60:0/1162351562 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:49:17.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.603+0000 7fc231c00700 1 --2- 172.21.15.60:0/1162351562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612f3040b80 0x5612f30462a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:17.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.604+0000 7fc22be00700 1 -- 172.21.15.60:0/1162351562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612f3040b80 msgr2=0x5612f30462a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:49:17.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.604+0000 7fc22be00700 1 --2- 172.21.15.60:0/1162351562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612f3040b80 0x5612f30462a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:17.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.604+0000 7fc22be00700 1 -- 172.21.15.60:0/1162351562 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612f303aee0 msgr2=0x5612f305fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:49:17.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.604+0000 7fc22be00700 1 --2- 172.21.15.60:0/1162351562 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612f303aee0 0x5612f305fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:17.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.604+0000 7fc22be00700 1 -- 172.21.15.60:0/1162351562 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612f3046ae0 con 0x5612f303b8a0 2024-05-03T14:49:17.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.604+0000 7fc231c00700 1 --2- 172.21.15.60:0/1162351562 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612f303aee0 0x5612f305fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:49:17.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.604+0000 7fc22be00700 1 --2- 172.21.15.60:0/1162351562 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5612f303b8a0 0x5612f3045d40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc2180098e0 tx=0x7fc218005700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f62f62c1695c9c server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:49:17.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.604+0000 7fc212a00700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-05-03T14:49:17.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.604+0000 7fc212a00700 1 -- 172.21.15.60:0/1162351562 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc218035070 con 0x5612f303b8a0 2024-05-03T14:49:17.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.605+0000 7fc23e4e2b80 1 -- 172.21.15.60:0/1162351562 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc21800bda0 con 0x5612f303b8a0 2024-05-03T14:49:17.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.605+0000 7fc212a00700 1 -- 172.21.15.60:0/1162351562 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7fc2180050b0 con 0x5612f303b8a0 2024-05-03T14:49:17.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.605+0000 7fc212a00700 1 -- 172.21.15.60:0/1162351562 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc218027040 con 0x5612f303b8a0 2024-05-03T14:49:17.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.605+0000 7fc23e4e2b80 10 client.15006 fetch_fsmap learned FSMap version 29 2024-05-03T14:49:17.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.605+0000 7fc23e4e2b80 1 -- 172.21.15.60:0/1162351562 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5612f30c01c0 con 0x5612f303b8a0 2024-05-03T14:49:17.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.605+0000 7fc212a00700 1 -- 172.21.15.60:0/1162351562 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7fc21800bda0 con 0x5612f303b8a0 2024-05-03T14:49:17.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.605+0000 7fc212a00700 1 -- 172.21.15.60:0/1162351562 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc218021030 con 0x5612f303b8a0 2024-05-03T14:49:17.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.605+0000 7fc23e4e2b80 10 client.15006 fetch_fsmap finished waiting for FSMap version 29 2024-05-03T14:49:17.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.606+0000 7fc23e4e2b80 10 client.15006 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-03T14:49:17.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.606+0000 7fc23e4e2b80 20 client.15006 populate_metadata read hostname 'smithi060' 2024-05-03T14:49:17.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.606+0000 7fc23e4e2b80 1 --2- 172.21.15.60:0/1162351562 >> [v2:172.21.15.90:6836/2243702844,v1:172.21.15.90:6837/2243702844] conn(0x5612f30c4630 0x5612f30c3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:17.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.606+0000 7fc23e4e2b80 4 client.15006 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-03T14:49:17.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.606+0000 7fc23e4e2b80 1 -- 172.21.15.60:0/1162351562 --> [v2:172.21.15.90:6836/2243702844,v1:172.21.15.90:6837/2243702844] -- command(tid 0: {"prefix": "status"}) -- 0x5612f2f95450 con 0x5612f30c4630 2024-05-03T14:49:17.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.606+0000 7fc231c00700 1 --2- 172.21.15.60:0/1162351562 >> [v2:172.21.15.90:6836/2243702844,v1:172.21.15.90:6837/2243702844] conn(0x5612f30c4630 0x5612f30c3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:17.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.607+0000 7fc231c00700 1 --2- 172.21.15.60:0/1162351562 >> [v2:172.21.15.90:6836/2243702844,v1:172.21.15.90:6837/2243702844] conn(0x5612f30c4630 0x5612f30c3340 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=91343a1232d25fb0 server_cookie=cf5058d28ab0c97d in_seq=0 out_seq=0 2024-05-03T14:49:17.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.607+0000 7fc212a00700 10 client.15006 ms_handle_connect on v2:172.21.15.90:6836/2243702844 2024-05-03T14:49:17.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.607+0000 7fc212a00700 1 -- 172.21.15.60:0/1162351562 <== mds.4 v2:172.21.15.90:6836/2243702844 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5612f2f95450 con 0x5612f30c4630 2024-05-03T14:49:17.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.607+0000 7fc212a00700 10 client.15006 handle_command_reply: tid=0 2024-05-03T14:49:17.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.607+0000 7fc23e4e2b80 1 client.15006 shutdown 2024-05-03T14:49:17.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.607+0000 7fc23e4e2b80 1 -- 172.21.15.60:0/1162351562 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5612f303b8a0 msgr2=0x5612f3045d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:17.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.607+0000 7fc23e4e2b80 1 --2- 172.21.15.60:0/1162351562 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5612f303b8a0 0x5612f3045d40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc2180098e0 tx=0x7fc218005700 comp rx=0 tx=0).stop 2024-05-03T14:49:17.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.607+0000 7fc23e4e2b80 1 -- 172.21.15.60:0/1162351562 shutdown_connections 2024-05-03T14:49:17.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.607+0000 7fc23e4e2b80 1 --2- 172.21.15.60:0/1162351562 >> [v2:172.21.15.90:6836/2243702844,v1:172.21.15.90:6837/2243702844] conn(0x5612f30c4630 0x5612f30c3340 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:17.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.607+0000 7fc23e4e2b80 1 --2- 172.21.15.60:0/1162351562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612f3040b80 0x5612f30462a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:17.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.607+0000 7fc23e4e2b80 1 --2- 172.21.15.60:0/1162351562 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612f303aee0 0x5612f305fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:17.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.607+0000 7fc23e4e2b80 1 --2- 172.21.15.60:0/1162351562 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5612f303b8a0 0x5612f3045d40 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:17.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.607+0000 7fc212a00700 0 client.15006 ms_handle_reset on v2:172.21.15.90:6836/2243702844 2024-05-03T14:49:17.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.607+0000 7fc23e4e2b80 1 -- 172.21.15.60:0/1162351562 >> 172.21.15.60:0/1162351562 conn(0x5612f2fb3680 msgr2=0x5612f3044870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:49:17.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.608+0000 7fc23e4e2b80 1 -- 172.21.15.60:0/1162351562 shutdown_connections 2024-05-03T14:49:17.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.608+0000 7fc23e4e2b80 1 -- 172.21.15.60:0/1162351562 wait complete. 2024-05-03T14:49:17.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.608+0000 7fc23e4e2b80 20 client.15006 trim_cache size 0 max 16384 2024-05-03T14:49:17.612 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-05-03T14:49:17.612 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "285d64b2-095a-11ef-bc97-c7b262605968", 2024-05-03T14:49:17.612 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 4, 2024-05-03T14:49:17.612 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 24470, 2024-05-03T14:49:17.612 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-05-03T14:49:17.612 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-05-03T14:49:17.612 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-05-03T14:49:17.612 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 201.07616837200001, 2024-05-03T14:49:17.612 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 29, 2024-05-03T14:49:17.613 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 89, 2024-05-03T14:49:17.613 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 87, 2024-05-03T14:49:17.613 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 320.556636932 2024-05-03T14:49:17.613 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-05-03T14:49:17.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.609+0000 7fc232600700 1 -- 172.21.15.60:0/1931960985 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fc2140b2990 msgr2=0x7fc2140b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:17.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.609+0000 7fc232600700 1 --2- 172.21.15.60:0/1931960985 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fc2140b2990 0x7fc2140b4e50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc22000f010 tx=0x7fc220009ef0 comp rx=0 tx=0).stop 2024-05-03T14:49:17.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.609+0000 7fc232600700 1 -- 172.21.15.60:0/1931960985 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc22c1055d0 msgr2=0x7fc22c19bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:17.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.609+0000 7fc232600700 1 --2- 172.21.15.60:0/1931960985 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc22c1055d0 0x7fc22c19bdb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc22000e9c0 tx=0x7fc22000ecd0 comp rx=0 tx=0).stop 2024-05-03T14:49:17.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.609+0000 7fc232600700 1 -- 172.21.15.60:0/1931960985 shutdown_connections 2024-05-03T14:49:17.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.609+0000 7fc232600700 1 --2- 172.21.15.60:0/1931960985 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fc2140b2990 0x7fc2140b4e50 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:17.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.609+0000 7fc232600700 1 --2- 172.21.15.60:0/1931960985 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc22c10b660 0x7fc22c1a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:17.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.609+0000 7fc232600700 1 --2- 172.21.15.60:0/1931960985 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc22c1055d0 0x7fc22c19bdb0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:17.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.609+0000 7fc232600700 1 --2- 172.21.15.60:0/1931960985 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc22c105f90 0x7fc22c19c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:17.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.609+0000 7fc232600700 1 -- 172.21.15.60:0/1931960985 >> 172.21.15.60:0/1931960985 conn(0x7fc22c1010c0 msgr2=0x7fc22c102520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:49:17.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.609+0000 7fc232600700 1 -- 172.21.15.60:0/1931960985 shutdown_connections 2024-05-03T14:49:17.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:17.609+0000 7fc232600700 1 -- 172.21.15.60:0/1931960985 wait complete. 2024-05-03T14:49:17.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:49:17 smithi060 ceph-mon[28873]: pgmap v475: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:49:17.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:49:17 smithi090 ceph-mon[33568]: pgmap v475: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:49:17.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:49:17 smithi149 ceph-mon[33332]: pgmap v475: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:49:18.577 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph tell mds.c status 2024-05-03T14:49:19.055 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:49:19.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:49:19 smithi060 ceph-mon[28873]: pgmap v476: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:49:19.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:49:19 smithi090 ceph-mon[33568]: pgmap v476: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:49:19.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:49:19 smithi149 ceph-mon[33332]: pgmap v476: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:49:21.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.278+0000 7ffa90c00700 1 -- 172.21.15.60:0/335244883 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa8c10b6e0 msgr2=0x7ffa8c10bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:21.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.278+0000 7ffa90c00700 1 --2- 172.21.15.60:0/335244883 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa8c10b6e0 0x7ffa8c10bac0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c009990 tx=0x7ffa7c009ca0 comp rx=0 tx=0).stop 2024-05-03T14:49:21.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.278+0000 7ffa90c00700 1 -- 172.21.15.60:0/335244883 shutdown_connections 2024-05-03T14:49:21.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.278+0000 7ffa90c00700 1 --2- 172.21.15.60:0/335244883 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa8c1060a0 0x7ffa8c1125b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:21.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.278+0000 7ffa90c00700 1 --2- 172.21.15.60:0/335244883 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa8c10b6e0 0x7ffa8c10bac0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:21.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.278+0000 7ffa90c00700 1 --2- 172.21.15.60:0/335244883 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffa8c1056e0 0x7ffa8c105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:21.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.278+0000 7ffa90c00700 1 -- 172.21.15.60:0/335244883 >> 172.21.15.60:0/335244883 conn(0x7ffa8c1011d0 msgr2=0x7ffa8c1035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:49:21.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.278+0000 7ffa90c00700 1 -- 172.21.15.60:0/335244883 shutdown_connections 2024-05-03T14:49:21.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.278+0000 7ffa90c00700 1 -- 172.21.15.60:0/335244883 wait complete. 2024-05-03T14:49:21.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.279+0000 7ffa90c00700 1 Processor -- start 2024-05-03T14:49:21.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.279+0000 7ffa90c00700 1 -- start start 2024-05-03T14:49:21.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.280+0000 7ffa90c00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa8c1056e0 0x7ffa8c19c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:21.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.280+0000 7ffa90c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffa8c1060a0 0x7ffa8c19c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:21.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.280+0000 7ffa90c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa8c10b6e0 0x7ffa8c1a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:21.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.280+0000 7ffa90c00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8c115b80 con 0x7ffa8c1056e0 2024-05-03T14:49:21.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.280+0000 7ffa90c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8c115a00 con 0x7ffa8c1060a0 2024-05-03T14:49:21.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.280+0000 7ffa90c00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8c115d00 con 0x7ffa8c10b6e0 2024-05-03T14:49:21.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.280+0000 7ffa89e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa8c1056e0 0x7ffa8c19c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:21.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.280+0000 7ffa89e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa8c1056e0 0x7ffa8c19c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44674/0 (socket says 172.21.15.60:44674) 2024-05-03T14:49:21.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.280+0000 7ffa89e00700 1 -- 172.21.15.60:0/2218357742 learned_addr learned my addr 172.21.15.60:0/2218357742 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:49:21.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.280+0000 7ffa8a800700 1 --2- 172.21.15.60:0/2218357742 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa8c10b6e0 0x7ffa8c1a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:21.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.281+0000 7ffa89400700 1 --2- 172.21.15.60:0/2218357742 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffa8c1060a0 0x7ffa8c19c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:21.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.281+0000 7ffa89e00700 1 -- 172.21.15.60:0/2218357742 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa8c10b6e0 msgr2=0x7ffa8c1a08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:21.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.281+0000 7ffa89e00700 1 --2- 172.21.15.60:0/2218357742 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa8c10b6e0 0x7ffa8c1a08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:21.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.281+0000 7ffa89e00700 1 -- 172.21.15.60:0/2218357742 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffa8c1060a0 msgr2=0x7ffa8c19c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:21.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.281+0000 7ffa89e00700 1 --2- 172.21.15.60:0/2218357742 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffa8c1060a0 0x7ffa8c19c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:21.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.281+0000 7ffa89e00700 1 -- 172.21.15.60:0/2218357742 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa8c1a0ff0 con 0x7ffa8c1056e0 2024-05-03T14:49:21.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.281+0000 7ffa89400700 1 --2- 172.21.15.60:0/2218357742 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffa8c1060a0 0x7ffa8c19c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T14:49:21.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.281+0000 7ffa89e00700 1 --2- 172.21.15.60:0/2218357742 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa8c1056e0 0x7ffa8c19c030 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c000c00 tx=0x7ffa7c0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:49:21.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.281+0000 7ffa82a00700 1 -- 172.21.15.60:0/2218357742 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c027070 con 0x7ffa8c1056e0 2024-05-03T14:49:21.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.281+0000 7ffa82a00700 1 -- 172.21.15.60:0/2218357742 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7ffa7c022470 con 0x7ffa8c1056e0 2024-05-03T14:49:21.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.281+0000 7ffa90c00700 1 -- 172.21.15.60:0/2218357742 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa8c1a1280 con 0x7ffa8c1056e0 2024-05-03T14:49:21.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.282+0000 7ffa82a00700 1 -- 172.21.15.60:0/2218357742 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c017650 con 0x7ffa8c1056e0 2024-05-03T14:49:21.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.282+0000 7ffa90c00700 1 -- 172.21.15.60:0/2218357742 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa8c1a8b60 con 0x7ffa8c1056e0 2024-05-03T14:49:21.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.285+0000 7ffa82a00700 1 -- 172.21.15.60:0/2218357742 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7ffa7c02a070 con 0x7ffa8c1056e0 2024-05-03T14:49:21.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.286+0000 7ffa9b4e9b80 1 Processor -- start 2024-05-03T14:49:21.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.287+0000 7ffa9b4e9b80 1 -- start start 2024-05-03T14:49:21.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.287+0000 7ffa82a00700 1 --2- 172.21.15.60:0/2218357742 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7ffa700b2910 0x7ffa700b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:21.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.287+0000 7ffa9b4e9b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560521262760 0x560521262b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:21.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.287+0000 7ffa9b4e9b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560521263080 0x56052125df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:21.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.287+0000 7ffa9b4e9b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56052125e580 0x56052125e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:21.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.287+0000 7ffa9b4e9b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5605211b09d0 con 0x56052125e580 2024-05-03T14:49:21.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.287+0000 7ffa9b4e9b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56052114e710 con 0x560521263080 2024-05-03T14:49:21.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.287+0000 7ffa9b4e9b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5605211ac990 con 0x560521262760 2024-05-03T14:49:21.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.287+0000 7ffa82a00700 1 -- 172.21.15.60:0/2218357742 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7ffa7c0e35d0 con 0x7ffa8c1056e0 2024-05-03T14:49:21.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.287+0000 7ffa8a800700 1 --2- 172.21.15.60:0/2218357742 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7ffa700b2910 0x7ffa700b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:21.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.287+0000 7ffa89e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560521262760 0x560521262b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:21.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.287+0000 7ffa8a800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56052125e580 0x56052125e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:21.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.287+0000 7ffa89e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560521262760 0x560521262b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.60:57276/0 (socket says 172.21.15.60:57276) 2024-05-03T14:49:21.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.287+0000 7ffa89e00700 1 -- 172.21.15.60:0/3430081459 learned_addr learned my addr 172.21.15.60:0/3430081459 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:49:21.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.287+0000 7ffa89400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560521263080 0x56052125df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:21.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.288+0000 7ffa89e00700 1 -- 172.21.15.60:0/3430081459 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560521263080 msgr2=0x56052125df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:49:21.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.288+0000 7ffa89e00700 1 --2- 172.21.15.60:0/3430081459 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560521263080 0x56052125df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:21.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.288+0000 7ffa89e00700 1 -- 172.21.15.60:0/3430081459 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56052125e580 msgr2=0x56052125e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:49:21.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.288+0000 7ffa89e00700 1 --2- 172.21.15.60:0/3430081459 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56052125e580 0x56052125e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:21.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.288+0000 7ffa89e00700 1 -- 172.21.15.60:0/3430081459 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605210fd880 con 0x560521262760 2024-05-03T14:49:21.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.288+0000 7ffa89e00700 1 --2- 172.21.15.60:0/3430081459 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560521262760 0x560521262b40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c0a67c0 tx=0x7ffa7c0aa1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a753740a12aa5c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:49:21.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.288+0000 7ffa77400700 1 -- 172.21.15.60:0/3430081459 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c0a4e60 con 0x560521262760 2024-05-03T14:49:21.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.288+0000 7ffa77400700 1 -- 172.21.15.60:0/3430081459 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7ffa7c02a360 con 0x560521262760 2024-05-03T14:49:21.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.289+0000 7ffa77400700 1 -- 172.21.15.60:0/3430081459 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c0aeaf0 con 0x560521262760 2024-05-03T14:49:21.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.289+0000 7ffa9b4e9b80 1 -- 172.21.15.60:0/3430081459 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560521262760 msgr2=0x560521262b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:21.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.289+0000 7ffa8a800700 1 --2- 172.21.15.60:0/2218357742 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7ffa700b2910 0x7ffa700b4dd0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ffa84009790 tx=0x7ffa8400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:49:21.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.289+0000 7ffa9b4e9b80 1 --2- 172.21.15.60:0/3430081459 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560521262760 0x560521262b40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c0a67c0 tx=0x7ffa7c0aa1c0 comp rx=0 tx=0).stop 2024-05-03T14:49:21.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.289+0000 7ffa9b4e9b80 1 -- 172.21.15.60:0/3430081459 shutdown_connections 2024-05-03T14:49:21.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.289+0000 7ffa9b4e9b80 1 --2- 172.21.15.60:0/3430081459 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56052125e580 0x56052125e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:21.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.289+0000 7ffa9b4e9b80 1 --2- 172.21.15.60:0/3430081459 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560521262760 0x560521262b40 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:21.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.289+0000 7ffa9b4e9b80 1 --2- 172.21.15.60:0/3430081459 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560521263080 0x56052125df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:21.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.289+0000 7ffa9b4e9b80 1 -- 172.21.15.60:0/3430081459 >> 172.21.15.60:0/3430081459 conn(0x56052118cb10 msgr2=0x5605212587a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:49:21.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.290+0000 7ffa9b4e9b80 1 -- 172.21.15.60:0/3430081459 shutdown_connections 2024-05-03T14:49:21.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.290+0000 7ffa9b4e9b80 1 -- 172.21.15.60:0/3430081459 wait complete. 2024-05-03T14:49:21.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.290+0000 7ffa9b4e9b80 1 -- start start 2024-05-03T14:49:21.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.290+0000 7ffa9b4e9b80 1 Processor -- start 2024-05-03T14:49:21.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.291+0000 7ffa9b4e9b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56052125e580 0x560521269e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:21.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.291+0000 7ffa9b4e9b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560521262760 0x5605212671d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:21.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.291+0000 7ffa9b4e9b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560521263080 0x560521267710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:21.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.291+0000 7ffa9b4e9b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x56052126aa10 con 0x560521263080 2024-05-03T14:49:21.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.291+0000 7ffa9b4e9b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x560521116ad0 con 0x56052125e580 2024-05-03T14:49:21.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.291+0000 7ffa9b4e9b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56052126ab90 con 0x560521262760 2024-05-03T14:49:21.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.291+0000 7ffa9b4e9b80 10 client.0 _ll_register_callbacks cb 0x560521157c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T14:49:21.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.291+0000 7ffa8a800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560521263080 0x560521267710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:21.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.291+0000 7ffa8a800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560521263080 0x560521267710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44684/0 (socket says 172.21.15.60:44684) 2024-05-03T14:49:21.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.291+0000 7ffa8a800700 1 -- 172.21.15.60:0/964805837 learned_addr learned my addr 172.21.15.60:0/964805837 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:49:21.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.291+0000 7ffa89400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560521262760 0x5605212671d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:21.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.292+0000 7ffa89e00700 1 --2- 172.21.15.60:0/964805837 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56052125e580 0x560521269e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:21.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.292+0000 7ffa89e00700 1 -- 172.21.15.60:0/964805837 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560521262760 msgr2=0x5605212671d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:49:21.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.292+0000 7ffa89e00700 1 --2- 172.21.15.60:0/964805837 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560521262760 0x5605212671d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:21.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.292+0000 7ffa89e00700 1 -- 172.21.15.60:0/964805837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560521263080 msgr2=0x560521267710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:49:21.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.292+0000 7ffa89e00700 1 --2- 172.21.15.60:0/964805837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560521263080 0x560521267710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:21.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.292+0000 7ffa89e00700 1 -- 172.21.15.60:0/964805837 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605211763d0 con 0x56052125e580 2024-05-03T14:49:21.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.292+0000 7ffa8a800700 1 --2- 172.21.15.60:0/964805837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560521263080 0x560521267710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:49:21.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.292+0000 7ffa89400700 1 --2- 172.21.15.60:0/964805837 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560521262760 0x5605212671d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:49:21.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.292+0000 7ffa89e00700 1 --2- 172.21.15.60:0/964805837 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56052125e580 0x560521269e30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c00c010 tx=0x7ffa7c00b4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68106c177c10f848 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:49:21.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.292+0000 7ffa6be00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-03T14:49:21.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.293+0000 7ffa6be00700 1 -- 172.21.15.60:0/964805837 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c0a4e60 con 0x56052125e580 2024-05-03T14:49:21.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.293+0000 7ffa6be00700 1 -- 172.21.15.60:0/964805837 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7ffa7c0e32b0 con 0x56052125e580 2024-05-03T14:49:21.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.293+0000 7ffa9b4e9b80 1 -- 172.21.15.60:0/964805837 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa7c02a360 con 0x56052125e580 2024-05-03T14:49:21.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.293+0000 7ffa6be00700 1 -- 172.21.15.60:0/964805837 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ffa7c02acf0 con 0x56052125e580 2024-05-03T14:49:21.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.293+0000 7ffa9b4e9b80 10 client.25118 fetch_fsmap learned FSMap version 29 2024-05-03T14:49:21.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.293+0000 7ffa9b4e9b80 1 -- 172.21.15.60:0/964805837 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56052126aa10 con 0x56052125e580 2024-05-03T14:49:21.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.293+0000 7ffa6be00700 1 -- 172.21.15.60:0/964805837 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7ffa7c02a360 con 0x56052125e580 2024-05-03T14:49:21.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.293+0000 7ffa6be00700 1 -- 172.21.15.60:0/964805837 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ffa7c0920f0 con 0x56052125e580 2024-05-03T14:49:21.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.294+0000 7ffa9b4e9b80 10 client.25118 fetch_fsmap finished waiting for FSMap version 29 2024-05-03T14:49:21.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.294+0000 7ffa9b4e9b80 10 client.25118 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-03T14:49:21.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.294+0000 7ffa9b4e9b80 20 client.25118 populate_metadata read hostname 'smithi060' 2024-05-03T14:49:21.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.294+0000 7ffa9b4e9b80 1 --2- 172.21.15.60:0/964805837 >> [v2:172.21.15.149:6836/2611278690,v1:172.21.15.149:6837/2611278690] conn(0x5605212775a0 0x560521278a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:21.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.294+0000 7ffa9b4e9b80 4 client.25118 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T14:49:21.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.294+0000 7ffa9b4e9b80 1 -- 172.21.15.60:0/964805837 --> [v2:172.21.15.149:6836/2611278690,v1:172.21.15.149:6837/2611278690] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56052122b180 con 0x5605212775a0 2024-05-03T14:49:21.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.294+0000 7ffa89400700 1 --2- 172.21.15.60:0/964805837 >> [v2:172.21.15.149:6836/2611278690,v1:172.21.15.149:6837/2611278690] conn(0x5605212775a0 0x560521278a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:21.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.295+0000 7ffa89400700 1 --2- 172.21.15.60:0/964805837 >> [v2:172.21.15.149:6836/2611278690,v1:172.21.15.149:6837/2611278690] conn(0x5605212775a0 0x560521278a10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6f96cbb3ebd5b8bd server_cookie=56c0ef55ce3f908 in_seq=0 out_seq=0 2024-05-03T14:49:21.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.295+0000 7ffa6be00700 10 client.25118 ms_handle_connect on v2:172.21.15.149:6836/2611278690 2024-05-03T14:49:21.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.296+0000 7ffa6be00700 1 -- 172.21.15.60:0/964805837 <== mds.3 v2:172.21.15.149:6836/2611278690 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56052122b180 con 0x5605212775a0 2024-05-03T14:49:21.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.296+0000 7ffa6be00700 10 client.25118 handle_command_reply: tid=0 2024-05-03T14:49:21.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.296+0000 7ffa9b4e9b80 1 client.25118 shutdown 2024-05-03T14:49:21.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.296+0000 7ffa9b4e9b80 1 -- 172.21.15.60:0/964805837 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56052125e580 msgr2=0x560521269e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:21.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.296+0000 7ffa9b4e9b80 1 --2- 172.21.15.60:0/964805837 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56052125e580 0x560521269e30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c00c010 tx=0x7ffa7c00b4b0 comp rx=0 tx=0).stop 2024-05-03T14:49:21.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.296+0000 7ffa9b4e9b80 1 -- 172.21.15.60:0/964805837 shutdown_connections 2024-05-03T14:49:21.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.296+0000 7ffa9b4e9b80 1 --2- 172.21.15.60:0/964805837 >> [v2:172.21.15.149:6836/2611278690,v1:172.21.15.149:6837/2611278690] conn(0x5605212775a0 0x560521278a10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:21.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.296+0000 7ffa9b4e9b80 1 --2- 172.21.15.60:0/964805837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560521263080 0x560521267710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:21.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.296+0000 7ffa9b4e9b80 1 --2- 172.21.15.60:0/964805837 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56052125e580 0x560521269e30 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:21.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.296+0000 7ffa9b4e9b80 1 --2- 172.21.15.60:0/964805837 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560521262760 0x5605212671d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:21.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.296+0000 7ffa6be00700 0 client.25118 ms_handle_reset on v2:172.21.15.149:6836/2611278690 2024-05-03T14:49:21.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.296+0000 7ffa9b4e9b80 1 -- 172.21.15.60:0/964805837 >> 172.21.15.60:0/964805837 conn(0x56052118cb10 msgr2=0x560521257600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:49:21.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.297+0000 7ffa9b4e9b80 1 -- 172.21.15.60:0/964805837 shutdown_connections 2024-05-03T14:49:21.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.297+0000 7ffa9b4e9b80 1 -- 172.21.15.60:0/964805837 wait complete. 2024-05-03T14:49:21.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.297+0000 7ffa9b4e9b80 20 client.25118 trim_cache size 0 max 16384 2024-05-03T14:49:21.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.320+0000 7ffa9b4e9b80 1 Processor -- start 2024-05-03T14:49:21.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.320+0000 7ffa9b4e9b80 1 -- start start 2024-05-03T14:49:21.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.320+0000 7ffa9b4e9b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56052125e580 0x560521265ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:21.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.320+0000 7ffa9b4e9b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560521262760 0x560521266000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:21.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.320+0000 7ffa9b4e9b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560521263080 0x560521264470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:21.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.320+0000 7ffa9b4e9b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x560521276d50 con 0x560521262760 2024-05-03T14:49:21.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.320+0000 7ffa9b4e9b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x560521277420 con 0x560521263080 2024-05-03T14:49:21.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.320+0000 7ffa9b4e9b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x560521277af0 con 0x56052125e580 2024-05-03T14:49:21.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.321+0000 7ffa89e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56052125e580 0x560521265ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:21.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.321+0000 7ffa89e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56052125e580 0x560521265ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.60:57290/0 (socket says 172.21.15.60:57290) 2024-05-03T14:49:21.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.321+0000 7ffa89e00700 1 -- 172.21.15.60:0/4269098457 learned_addr learned my addr 172.21.15.60:0/4269098457 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:49:21.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.321+0000 7ffa8a800700 1 --2- 172.21.15.60:0/4269098457 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560521263080 0x560521264470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:21.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.321+0000 7ffa89400700 1 --2- 172.21.15.60:0/4269098457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560521262760 0x560521266000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:21.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.321+0000 7ffa89e00700 1 -- 172.21.15.60:0/4269098457 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560521263080 msgr2=0x560521264470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:49:21.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.321+0000 7ffa89e00700 1 --2- 172.21.15.60:0/4269098457 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560521263080 0x560521264470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:21.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.321+0000 7ffa89e00700 1 -- 172.21.15.60:0/4269098457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560521262760 msgr2=0x560521266000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:49:21.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.321+0000 7ffa89e00700 1 --2- 172.21.15.60:0/4269098457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560521262760 0x560521266000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:21.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.321+0000 7ffa89e00700 1 -- 172.21.15.60:0/4269098457 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605212623b0 con 0x56052125e580 2024-05-03T14:49:21.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.321+0000 7ffa89400700 1 --2- 172.21.15.60:0/4269098457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560521262760 0x560521266000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T14:49:21.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.322+0000 7ffa89e00700 1 --2- 172.21.15.60:0/4269098457 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56052125e580 0x560521265ac0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c0aa6d0 tx=0x7ffa7c022e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9ff69e89709f665 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:49:21.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.322+0000 7ffa76a00700 1 -- 172.21.15.60:0/4269098457 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c0a4e60 con 0x56052125e580 2024-05-03T14:49:21.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.322+0000 7ffa76a00700 1 -- 172.21.15.60:0/4269098457 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7ffa7c02a360 con 0x56052125e580 2024-05-03T14:49:21.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.322+0000 7ffa76a00700 1 -- 172.21.15.60:0/4269098457 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c0aa740 con 0x56052125e580 2024-05-03T14:49:21.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.323+0000 7ffa9b4e9b80 1 -- 172.21.15.60:0/4269098457 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56052125e580 msgr2=0x560521265ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:21.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.323+0000 7ffa9b4e9b80 1 --2- 172.21.15.60:0/4269098457 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56052125e580 0x560521265ac0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c0aa6d0 tx=0x7ffa7c022e00 comp rx=0 tx=0).stop 2024-05-03T14:49:21.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.323+0000 7ffa9b4e9b80 1 -- 172.21.15.60:0/4269098457 shutdown_connections 2024-05-03T14:49:21.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.323+0000 7ffa9b4e9b80 1 --2- 172.21.15.60:0/4269098457 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560521263080 0x560521264470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:21.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.323+0000 7ffa9b4e9b80 1 --2- 172.21.15.60:0/4269098457 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56052125e580 0x560521265ac0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:21.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.323+0000 7ffa9b4e9b80 1 --2- 172.21.15.60:0/4269098457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560521262760 0x560521266000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:21.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.323+0000 7ffa9b4e9b80 1 -- 172.21.15.60:0/4269098457 >> 172.21.15.60:0/4269098457 conn(0x56052118cb10 msgr2=0x560521258710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:49:21.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.323+0000 7ffa9b4e9b80 1 -- 172.21.15.60:0/4269098457 shutdown_connections 2024-05-03T14:49:21.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.323+0000 7ffa9b4e9b80 1 -- 172.21.15.60:0/4269098457 wait complete. 2024-05-03T14:49:21.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.323+0000 7ffa9b4e9b80 1 -- start start 2024-05-03T14:49:21.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.324+0000 7ffa9b4e9b80 1 Processor -- start 2024-05-03T14:49:21.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.324+0000 7ffa9b4e9b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56052125e580 0x560521264160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:21.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.324+0000 7ffa9b4e9b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560521262760 0x5605212646a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:21.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.324+0000 7ffa9b4e9b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560521263080 0x560521264be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:21.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.324+0000 7ffa9b4e9b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x560521278890 con 0x56052125e580 2024-05-03T14:49:21.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.324+0000 7ffa9b4e9b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5605212781c0 con 0x560521263080 2024-05-03T14:49:21.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.324+0000 7ffa9b4e9b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x560521278f60 con 0x560521262760 2024-05-03T14:49:21.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.324+0000 7ffa9b4e9b80 10 client.0 _ll_register_callbacks cb 0x560521157c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T14:49:21.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.325+0000 7ffa89e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56052125e580 0x560521264160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:21.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.325+0000 7ffa8a800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560521263080 0x560521264be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:21.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.325+0000 7ffa89400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560521262760 0x5605212646a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:21.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.325+0000 7ffa8a800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560521263080 0x560521264be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.60:38868/0 (socket says 172.21.15.60:38868) 2024-05-03T14:49:21.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.325+0000 7ffa8a800700 1 -- 172.21.15.60:0/2544630109 learned_addr learned my addr 172.21.15.60:0/2544630109 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:49:21.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.325+0000 7ffa89400700 1 -- 172.21.15.60:0/2544630109 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560521263080 msgr2=0x560521264be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:49:21.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.325+0000 7ffa89400700 1 --2- 172.21.15.60:0/2544630109 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560521263080 0x560521264be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:21.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.325+0000 7ffa89400700 1 -- 172.21.15.60:0/2544630109 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56052125e580 msgr2=0x560521264160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:49:21.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.325+0000 7ffa89400700 1 --2- 172.21.15.60:0/2544630109 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56052125e580 0x560521264160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:21.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.325+0000 7ffa89400700 1 -- 172.21.15.60:0/2544630109 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560521280470 con 0x560521262760 2024-05-03T14:49:21.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.325+0000 7ffa89400700 1 --2- 172.21.15.60:0/2544630109 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560521262760 0x5605212646a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x560521267ca0 tx=0x7ffa78012b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa414e38a1ea41a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:49:21.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.325+0000 7ffa6be00700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-05-03T14:49:21.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.326+0000 7ffa6be00700 1 -- 172.21.15.60:0/2544630109 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa78021620 con 0x560521262760 2024-05-03T14:49:21.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.326+0000 7ffa6be00700 1 -- 172.21.15.60:0/2544630109 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7ffa780217c0 con 0x560521262760 2024-05-03T14:49:21.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.326+0000 7ffa9b4e9b80 1 -- 172.21.15.60:0/2544630109 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa7c02a360 con 0x560521262760 2024-05-03T14:49:21.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.326+0000 7ffa6be00700 1 -- 172.21.15.60:0/2544630109 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ffa78031050 con 0x560521262760 2024-05-03T14:49:21.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.326+0000 7ffa9b4e9b80 10 client.25132 fetch_fsmap learned FSMap version 29 2024-05-03T14:49:21.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.326+0000 7ffa9b4e9b80 1 -- 172.21.15.60:0/2544630109 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5605212e2eb0 con 0x560521262760 2024-05-03T14:49:21.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.327+0000 7ffa6be00700 1 -- 172.21.15.60:0/2544630109 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7ffa7c02a360 con 0x560521262760 2024-05-03T14:49:21.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.327+0000 7ffa6be00700 1 -- 172.21.15.60:0/2544630109 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ffa78011440 con 0x560521262760 2024-05-03T14:49:21.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.327+0000 7ffa9b4e9b80 10 client.25132 fetch_fsmap finished waiting for FSMap version 29 2024-05-03T14:49:21.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.327+0000 7ffa9b4e9b80 10 client.25132 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-03T14:49:21.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.327+0000 7ffa9b4e9b80 20 client.25132 populate_metadata read hostname 'smithi060' 2024-05-03T14:49:21.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.327+0000 7ffa9b4e9b80 1 --2- 172.21.15.60:0/2544630109 >> [v2:172.21.15.149:6836/2611278690,v1:172.21.15.149:6837/2611278690] conn(0x5605212e7a70 0x5605212e6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:21.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.327+0000 7ffa9b4e9b80 4 client.25132 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-03T14:49:21.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.327+0000 7ffa9b4e9b80 1 -- 172.21.15.60:0/2544630109 --> [v2:172.21.15.149:6836/2611278690,v1:172.21.15.149:6837/2611278690] -- command(tid 0: {"prefix": "status"}) -- 0x5605211be3b0 con 0x5605212e7a70 2024-05-03T14:49:21.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.327+0000 7ffa89e00700 1 --2- 172.21.15.60:0/2544630109 >> [v2:172.21.15.149:6836/2611278690,v1:172.21.15.149:6837/2611278690] conn(0x5605212e7a70 0x5605212e6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:21.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.328+0000 7ffa89e00700 1 --2- 172.21.15.60:0/2544630109 >> [v2:172.21.15.149:6836/2611278690,v1:172.21.15.149:6837/2611278690] conn(0x5605212e7a70 0x5605212e6780 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f18ab0ec4aa3f3c3 server_cookie=f83cf319f2086448 in_seq=0 out_seq=0 2024-05-03T14:49:21.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.328+0000 7ffa6be00700 10 client.25132 ms_handle_connect on v2:172.21.15.149:6836/2611278690 2024-05-03T14:49:21.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.328+0000 7ffa6be00700 1 -- 172.21.15.60:0/2544630109 <== mds.3 v2:172.21.15.149:6836/2611278690 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5605211be3b0 con 0x5605212e7a70 2024-05-03T14:49:21.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.328+0000 7ffa6be00700 10 client.25132 handle_command_reply: tid=0 2024-05-03T14:49:21.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.328+0000 7ffa9b4e9b80 1 client.25132 shutdown 2024-05-03T14:49:21.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.328+0000 7ffa9b4e9b80 1 -- 172.21.15.60:0/2544630109 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560521262760 msgr2=0x5605212646a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:21.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.328+0000 7ffa9b4e9b80 1 --2- 172.21.15.60:0/2544630109 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560521262760 0x5605212646a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x560521267ca0 tx=0x7ffa78012b00 comp rx=0 tx=0).stop 2024-05-03T14:49:21.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.328+0000 7ffa9b4e9b80 1 -- 172.21.15.60:0/2544630109 shutdown_connections 2024-05-03T14:49:21.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.328+0000 7ffa9b4e9b80 1 --2- 172.21.15.60:0/2544630109 >> [v2:172.21.15.149:6836/2611278690,v1:172.21.15.149:6837/2611278690] conn(0x5605212e7a70 0x5605212e6780 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:21.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.328+0000 7ffa9b4e9b80 1 --2- 172.21.15.60:0/2544630109 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560521263080 0x560521264be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:21.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.328+0000 7ffa9b4e9b80 1 --2- 172.21.15.60:0/2544630109 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56052125e580 0x560521264160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:21.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.328+0000 7ffa9b4e9b80 1 --2- 172.21.15.60:0/2544630109 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560521262760 0x5605212646a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:21.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.328+0000 7ffa6be00700 0 client.25132 ms_handle_reset on v2:172.21.15.149:6836/2611278690 2024-05-03T14:49:21.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.328+0000 7ffa9b4e9b80 1 -- 172.21.15.60:0/2544630109 >> 172.21.15.60:0/2544630109 conn(0x56052118cb10 msgr2=0x560521265a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:49:21.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.329+0000 7ffa9b4e9b80 1 -- 172.21.15.60:0/2544630109 shutdown_connections 2024-05-03T14:49:21.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.329+0000 7ffa9b4e9b80 1 -- 172.21.15.60:0/2544630109 wait complete. 2024-05-03T14:49:21.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.329+0000 7ffa9b4e9b80 20 client.25132 trim_cache size 0 max 16384 2024-05-03T14:49:21.333 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-05-03T14:49:21.333 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "285d64b2-095a-11ef-bc97-c7b262605968", 2024-05-03T14:49:21.333 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 3, 2024-05-03T14:49:21.333 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 24475, 2024-05-03T14:49:21.333 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-05-03T14:49:21.333 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-05-03T14:49:21.333 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-05-03T14:49:21.333 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 205.817218873, 2024-05-03T14:49:21.333 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 29, 2024-05-03T14:49:21.333 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 89, 2024-05-03T14:49:21.333 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 87, 2024-05-03T14:49:21.333 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 327.10608230999998 2024-05-03T14:49:21.333 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-05-03T14:49:21.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.330+0000 7ffa90c00700 1 -- 172.21.15.60:0/2218357742 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7ffa700b2910 msgr2=0x7ffa700b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:21.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.330+0000 7ffa90c00700 1 --2- 172.21.15.60:0/2218357742 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7ffa700b2910 0x7ffa700b4dd0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ffa84009790 tx=0x7ffa8400b040 comp rx=0 tx=0).stop 2024-05-03T14:49:21.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.330+0000 7ffa90c00700 1 -- 172.21.15.60:0/2218357742 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa8c1056e0 msgr2=0x7ffa8c19c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:21.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.330+0000 7ffa90c00700 1 --2- 172.21.15.60:0/2218357742 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa8c1056e0 0x7ffa8c19c030 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c000c00 tx=0x7ffa7c0057c0 comp rx=0 tx=0).stop 2024-05-03T14:49:21.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.330+0000 7ffa90c00700 1 -- 172.21.15.60:0/2218357742 shutdown_connections 2024-05-03T14:49:21.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.330+0000 7ffa90c00700 1 --2- 172.21.15.60:0/2218357742 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7ffa700b2910 0x7ffa700b4dd0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:21.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.330+0000 7ffa90c00700 1 --2- 172.21.15.60:0/2218357742 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa8c10b6e0 0x7ffa8c1a08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:21.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.330+0000 7ffa90c00700 1 --2- 172.21.15.60:0/2218357742 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa8c1056e0 0x7ffa8c19c030 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:21.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.330+0000 7ffa90c00700 1 --2- 172.21.15.60:0/2218357742 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffa8c1060a0 0x7ffa8c19c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:21.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.330+0000 7ffa90c00700 1 -- 172.21.15.60:0/2218357742 >> 172.21.15.60:0/2218357742 conn(0x7ffa8c1011d0 msgr2=0x7ffa8c101790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:49:21.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.330+0000 7ffa90c00700 1 -- 172.21.15.60:0/2218357742 shutdown_connections 2024-05-03T14:49:21.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:21.330+0000 7ffa90c00700 1 -- 172.21.15.60:0/2218357742 wait complete. 2024-05-03T14:49:21.738 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:49:21 smithi060 ceph-mon[28873]: pgmap v477: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:49:21.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:49:21 smithi090 ceph-mon[33568]: pgmap v477: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:49:21.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:49:21 smithi149 ceph-mon[33332]: pgmap v477: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:49:22.232 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph tell mds.i status 2024-05-03T14:49:22.684 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:49:23.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:49:23 smithi060 ceph-mon[28873]: pgmap v478: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T14:49:23.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:49:23 smithi090 ceph-mon[33568]: pgmap v478: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T14:49:23.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:49:23 smithi149 ceph-mon[33332]: pgmap v478: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T14:49:24.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:24.995+0000 7fc76aa00700 1 -- 172.21.15.60:0/3063186496 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75800bc80 con 0x7fc76c075ff0 2024-05-03T14:49:24.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:24.996+0000 7fc773200700 1 -- 172.21.15.60:0/3063186496 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc76c075ff0 msgr2=0x7fc76c10f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:24.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:24.996+0000 7fc773200700 1 --2- 172.21.15.60:0/3063186496 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc76c075ff0 0x7fc76c10f9c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc758009990 tx=0x7fc758009ca0 comp rx=0 tx=0).stop 2024-05-03T14:49:24.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:24.996+0000 7fc773200700 1 -- 172.21.15.60:0/3063186496 shutdown_connections 2024-05-03T14:49:24.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:24.996+0000 7fc773200700 1 --2- 172.21.15.60:0/3063186496 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc76c10ff00 0x7fc76c1122f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:24.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:24.996+0000 7fc773200700 1 --2- 172.21.15.60:0/3063186496 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc76c0756d0 0x7fc76c075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:24.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:24.996+0000 7fc773200700 1 --2- 172.21.15.60:0/3063186496 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc76c075ff0 0x7fc76c10f9c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:24.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:24.996+0000 7fc773200700 1 -- 172.21.15.60:0/3063186496 >> 172.21.15.60:0/3063186496 conn(0x7fc76c07a600 msgr2=0x7fc76c07aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:49:24.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:24.996+0000 7fc773200700 1 -- 172.21.15.60:0/3063186496 shutdown_connections 2024-05-03T14:49:24.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:24.996+0000 7fc773200700 1 -- 172.21.15.60:0/3063186496 wait complete. 2024-05-03T14:49:24.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:24.997+0000 7fc773200700 1 Processor -- start 2024-05-03T14:49:25.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:24.997+0000 7fc773200700 1 -- start start 2024-05-03T14:49:25.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:24.998+0000 7fc773200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc76c0756d0 0x7fc76c19c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:25.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:24.998+0000 7fc773200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc76c075ff0 0x7fc76c19c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:25.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:24.998+0000 7fc773200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc76c10ff00 0x7fc76c1a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:25.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:24.998+0000 7fc773200700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc76c115b50 con 0x7fc76c0756d0 2024-05-03T14:49:25.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:24.998+0000 7fc773200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc76c1159d0 con 0x7fc76c10ff00 2024-05-03T14:49:25.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:24.998+0000 7fc773200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc76c115cd0 con 0x7fc76c075ff0 2024-05-03T14:49:25.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:24.998+0000 7fc76be00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc76c0756d0 0x7fc76c19c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:25.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:24.998+0000 7fc76be00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc76c0756d0 0x7fc76c19c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36460/0 (socket says 172.21.15.60:36460) 2024-05-03T14:49:25.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:24.998+0000 7fc76be00700 1 -- 172.21.15.60:0/4170190590 learned_addr learned my addr 172.21.15.60:0/4170190590 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:49:25.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:24.998+0000 7fc76b400700 1 --2- 172.21.15.60:0/4170190590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc76c075ff0 0x7fc76c19c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:25.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:24.998+0000 7fc771200700 1 --2- 172.21.15.60:0/4170190590 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc76c10ff00 0x7fc76c1a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:25.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:24.999+0000 7fc76be00700 1 -- 172.21.15.60:0/4170190590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc76c075ff0 msgr2=0x7fc76c19c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:25.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:24.999+0000 7fc76be00700 1 --2- 172.21.15.60:0/4170190590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc76c075ff0 0x7fc76c19c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:25.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:24.999+0000 7fc76be00700 1 -- 172.21.15.60:0/4170190590 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc76c10ff00 msgr2=0x7fc76c1a08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:25.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:24.999+0000 7fc76be00700 1 --2- 172.21.15.60:0/4170190590 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc76c10ff00 0x7fc76c1a08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:25.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:24.999+0000 7fc76be00700 1 -- 172.21.15.60:0/4170190590 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc76c1a0fe0 con 0x7fc76c0756d0 2024-05-03T14:49:25.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:24.999+0000 7fc771200700 1 --2- 172.21.15.60:0/4170190590 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc76c10ff00 0x7fc76c1a08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T14:49:25.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:24.999+0000 7fc76b400700 1 --2- 172.21.15.60:0/4170190590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc76c075ff0 0x7fc76c19c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:49:25.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:24.999+0000 7fc76be00700 1 --2- 172.21.15.60:0/4170190590 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc76c0756d0 0x7fc76c19c020 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc75c00e9a0 tx=0x7fc75c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:49:25.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:24.999+0000 7fc768c00700 1 -- 172.21.15.60:0/4170190590 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75c00cbd0 con 0x7fc76c0756d0 2024-05-03T14:49:25.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:24.999+0000 7fc768c00700 1 -- 172.21.15.60:0/4170190590 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7fc75c00cd70 con 0x7fc76c0756d0 2024-05-03T14:49:25.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:24.999+0000 7fc773200700 1 -- 172.21.15.60:0/4170190590 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc76c1a12d0 con 0x7fc76c0756d0 2024-05-03T14:49:25.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:24.999+0000 7fc768c00700 1 -- 172.21.15.60:0/4170190590 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75c018820 con 0x7fc76c0756d0 2024-05-03T14:49:25.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.000+0000 7fc773200700 1 -- 172.21.15.60:0/4170190590 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc76c1a8b50 con 0x7fc76c0756d0 2024-05-03T14:49:25.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.003+0000 7fc768c00700 1 -- 172.21.15.60:0/4170190590 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fc75c010430 con 0x7fc76c0756d0 2024-05-03T14:49:25.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.004+0000 7fc77dac1b80 1 Processor -- start 2024-05-03T14:49:25.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.005+0000 7fc77dac1b80 1 -- start start 2024-05-03T14:49:25.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.005+0000 7fc768c00700 1 --2- 172.21.15.60:0/4170190590 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fc7500b2910 0x7fc7500b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:25.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.005+0000 7fc77dac1b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fa1056760 0x557fa1056b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:25.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.005+0000 7fc77dac1b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fa1057080 0x557fa1051f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:25.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.005+0000 7fc77dac1b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fa1052580 0x557fa10529e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:25.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.005+0000 7fc77dac1b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x557fa0fa49d0 con 0x557fa1057080 2024-05-03T14:49:25.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.005+0000 7fc77dac1b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x557fa0f42710 con 0x557fa1056760 2024-05-03T14:49:25.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.005+0000 7fc77dac1b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x557fa0fa0990 con 0x557fa1052580 2024-05-03T14:49:25.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.005+0000 7fc768c00700 1 -- 172.21.15.60:0/4170190590 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7fc75c0e29d0 con 0x7fc76c0756d0 2024-05-03T14:49:25.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.005+0000 7fc76b400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fa1057080 0x557fa1051f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:25.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.005+0000 7fc771200700 1 --2- 172.21.15.60:0/4170190590 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fc7500b2910 0x7fc7500b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:25.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.006+0000 7fc771200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fa1052580 0x557fa10529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:25.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.006+0000 7fc76be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fa1056760 0x557fa1056b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:25.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.006+0000 7fc771200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fa1052580 0x557fa10529e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.60:59018/0 (socket says 172.21.15.60:59018) 2024-05-03T14:49:25.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.006+0000 7fc771200700 1 -- 172.21.15.60:0/3592790338 learned_addr learned my addr 172.21.15.60:0/3592790338 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:49:25.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.006+0000 7fc771200700 1 -- 172.21.15.60:0/3592790338 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fa1056760 msgr2=0x557fa1056b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:49:25.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.006+0000 7fc771200700 1 --2- 172.21.15.60:0/3592790338 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fa1056760 0x557fa1056b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:25.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.006+0000 7fc771200700 1 -- 172.21.15.60:0/3592790338 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fa1057080 msgr2=0x557fa1051f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:49:25.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.006+0000 7fc771200700 1 --2- 172.21.15.60:0/3592790338 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fa1057080 0x557fa1051f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:25.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.006+0000 7fc771200700 1 -- 172.21.15.60:0/3592790338 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fa0ef1880 con 0x557fa1052580 2024-05-03T14:49:25.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.006+0000 7fc76be00700 1 --2- 172.21.15.60:0/3592790338 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fa1056760 0x557fa1056b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:49:25.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.007+0000 7fc771200700 1 --2- 172.21.15.60:0/3592790338 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fa1052580 0x557fa10529e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc764019550 tx=0x7fc764019860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb204db21f1ca0e server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:49:25.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.007+0000 7fc771200700 1 --2- 172.21.15.60:0/4170190590 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fc7500b2910 0x7fc7500b4dd0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc764019bb0 tx=0x7fc76400f000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:49:25.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.007+0000 7fc761600700 1 -- 172.21.15.60:0/3592790338 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc764004db0 con 0x557fa1052580 2024-05-03T14:49:25.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.007+0000 7fc761600700 1 -- 172.21.15.60:0/3592790338 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7fc76401e070 con 0x557fa1052580 2024-05-03T14:49:25.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.007+0000 7fc761600700 1 -- 172.21.15.60:0/3592790338 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc764007760 con 0x557fa1052580 2024-05-03T14:49:25.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.008+0000 7fc77dac1b80 1 -- 172.21.15.60:0/3592790338 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fa1052580 msgr2=0x557fa10529e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:25.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.008+0000 7fc77dac1b80 1 --2- 172.21.15.60:0/3592790338 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fa1052580 0x557fa10529e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc764019550 tx=0x7fc764019860 comp rx=0 tx=0).stop 2024-05-03T14:49:25.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.008+0000 7fc77dac1b80 1 -- 172.21.15.60:0/3592790338 shutdown_connections 2024-05-03T14:49:25.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.008+0000 7fc77dac1b80 1 --2- 172.21.15.60:0/3592790338 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fa1052580 0x557fa10529e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:25.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.008+0000 7fc77dac1b80 1 --2- 172.21.15.60:0/3592790338 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fa1056760 0x557fa1056b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:25.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.008+0000 7fc77dac1b80 1 --2- 172.21.15.60:0/3592790338 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fa1057080 0x557fa1051f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:25.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.008+0000 7fc77dac1b80 1 -- 172.21.15.60:0/3592790338 >> 172.21.15.60:0/3592790338 conn(0x557fa0f80b10 msgr2=0x557fa104c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:49:25.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.008+0000 7fc77dac1b80 1 -- 172.21.15.60:0/3592790338 shutdown_connections 2024-05-03T14:49:25.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.008+0000 7fc77dac1b80 1 -- 172.21.15.60:0/3592790338 wait complete. 2024-05-03T14:49:25.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.009+0000 7fc77dac1b80 1 -- start start 2024-05-03T14:49:25.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.009+0000 7fc77dac1b80 1 Processor -- start 2024-05-03T14:49:25.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.009+0000 7fc77dac1b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fa1052580 0x557fa105df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:25.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.009+0000 7fc77dac1b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fa1056760 0x557fa105b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:25.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.009+0000 7fc77dac1b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fa1057080 0x557fa105b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:25.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.009+0000 7fc77dac1b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x557fa105eb90 con 0x557fa1057080 2024-05-03T14:49:25.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.009+0000 7fc77dac1b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x557fa0f0aad0 con 0x557fa1056760 2024-05-03T14:49:25.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.009+0000 7fc77dac1b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x557fa105ed10 con 0x557fa1052580 2024-05-03T14:49:25.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.009+0000 7fc77dac1b80 10 client.0 _ll_register_callbacks cb 0x557fa0f4bc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T14:49:25.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.010+0000 7fc76b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fa1056760 0x557fa105b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:25.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.010+0000 7fc76b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fa1056760 0x557fa105b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.60:53732/0 (socket says 172.21.15.60:53732) 2024-05-03T14:49:25.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.010+0000 7fc76b400700 1 -- 172.21.15.60:0/1410631263 learned_addr learned my addr 172.21.15.60:0/1410631263 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:49:25.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.010+0000 7fc76be00700 1 --2- 172.21.15.60:0/1410631263 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fa1052580 0x557fa105df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:25.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.010+0000 7fc771200700 1 --2- 172.21.15.60:0/1410631263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fa1057080 0x557fa105b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:25.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.010+0000 7fc76b400700 1 -- 172.21.15.60:0/1410631263 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fa1052580 msgr2=0x557fa105df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:49:25.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.010+0000 7fc76b400700 1 --2- 172.21.15.60:0/1410631263 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fa1052580 0x557fa105df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:25.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.010+0000 7fc76b400700 1 -- 172.21.15.60:0/1410631263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fa1057080 msgr2=0x557fa105b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:49:25.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.010+0000 7fc76b400700 1 --2- 172.21.15.60:0/1410631263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fa1057080 0x557fa105b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:25.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.010+0000 7fc76b400700 1 -- 172.21.15.60:0/1410631263 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fa0f6a3d0 con 0x557fa1056760 2024-05-03T14:49:25.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.011+0000 7fc76b400700 1 --2- 172.21.15.60:0/1410631263 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fa1056760 0x557fa105b2b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc758009990 tx=0x7fc758005040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edccc1ab044e427d server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:49:25.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.011+0000 7fc756000700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-03T14:49:25.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.011+0000 7fc756000700 1 -- 172.21.15.60:0/1410631263 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc758029060 con 0x557fa1056760 2024-05-03T14:49:25.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.011+0000 7fc77dac1b80 1 -- 172.21.15.60:0/1410631263 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc76401e070 con 0x557fa1056760 2024-05-03T14:49:25.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.011+0000 7fc756000700 1 -- 172.21.15.60:0/1410631263 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7fc758003880 con 0x557fa1056760 2024-05-03T14:49:25.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.011+0000 7fc756000700 1 -- 172.21.15.60:0/1410631263 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc758030050 con 0x557fa1056760 2024-05-03T14:49:25.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.011+0000 7fc77dac1b80 10 client.25136 fetch_fsmap learned FSMap version 29 2024-05-03T14:49:25.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.011+0000 7fc77dac1b80 1 -- 172.21.15.60:0/1410631263 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557fa1069460 con 0x557fa1056760 2024-05-03T14:49:25.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.012+0000 7fc756000700 1 -- 172.21.15.60:0/1410631263 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7fc76401e070 con 0x557fa1056760 2024-05-03T14:49:25.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.012+0000 7fc756000700 1 -- 172.21.15.60:0/1410631263 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc75800f460 con 0x557fa1056760 2024-05-03T14:49:25.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.012+0000 7fc77dac1b80 10 client.25136 fetch_fsmap finished waiting for FSMap version 29 2024-05-03T14:49:25.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.012+0000 7fc77dac1b80 10 client.25136 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-03T14:49:25.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.012+0000 7fc77dac1b80 20 client.25136 populate_metadata read hostname 'smithi060' 2024-05-03T14:49:25.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.012+0000 7fc77dac1b80 1 --2- 172.21.15.60:0/1410631263 >> [v2:172.21.15.149:6838/2527622212,v1:172.21.15.149:6839/2527622212] conn(0x557fa106ba60 0x557fa106ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:25.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.012+0000 7fc77dac1b80 4 client.25136 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T14:49:25.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.012+0000 7fc77dac1b80 1 -- 172.21.15.60:0/1410631263 --> [v2:172.21.15.149:6838/2527622212,v1:172.21.15.149:6839/2527622212] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557fa101f180 con 0x557fa106ba60 2024-05-03T14:49:25.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.012+0000 7fc76be00700 1 --2- 172.21.15.60:0/1410631263 >> [v2:172.21.15.149:6838/2527622212,v1:172.21.15.149:6839/2527622212] conn(0x557fa106ba60 0x557fa106ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:25.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.013+0000 7fc76be00700 1 --2- 172.21.15.60:0/1410631263 >> [v2:172.21.15.149:6838/2527622212,v1:172.21.15.149:6839/2527622212] conn(0x557fa106ba60 0x557fa106ced0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cefd34836713dd52 server_cookie=28c7054d213e9c62 in_seq=0 out_seq=0 2024-05-03T14:49:25.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.013+0000 7fc756000700 10 client.25136 ms_handle_connect on v2:172.21.15.149:6838/2527622212 2024-05-03T14:49:25.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.014+0000 7fc756000700 1 -- 172.21.15.60:0/1410631263 <== mds.2 v2:172.21.15.149:6838/2527622212 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557fa101f180 con 0x557fa106ba60 2024-05-03T14:49:25.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.014+0000 7fc756000700 10 client.25136 handle_command_reply: tid=0 2024-05-03T14:49:25.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.014+0000 7fc77dac1b80 1 client.25136 shutdown 2024-05-03T14:49:25.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.014+0000 7fc77dac1b80 1 -- 172.21.15.60:0/1410631263 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fa1056760 msgr2=0x557fa105b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:25.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.014+0000 7fc77dac1b80 1 --2- 172.21.15.60:0/1410631263 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fa1056760 0x557fa105b2b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc758009990 tx=0x7fc758005040 comp rx=0 tx=0).stop 2024-05-03T14:49:25.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.014+0000 7fc77dac1b80 1 -- 172.21.15.60:0/1410631263 shutdown_connections 2024-05-03T14:49:25.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.014+0000 7fc77dac1b80 1 --2- 172.21.15.60:0/1410631263 >> [v2:172.21.15.149:6838/2527622212,v1:172.21.15.149:6839/2527622212] conn(0x557fa106ba60 0x557fa106ced0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:25.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.014+0000 7fc77dac1b80 1 --2- 172.21.15.60:0/1410631263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fa1057080 0x557fa105b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:25.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.014+0000 7fc77dac1b80 1 --2- 172.21.15.60:0/1410631263 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fa1052580 0x557fa105df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:25.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.014+0000 7fc77dac1b80 1 --2- 172.21.15.60:0/1410631263 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fa1056760 0x557fa105b2b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:25.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.014+0000 7fc756000700 0 client.25136 ms_handle_reset on v2:172.21.15.149:6838/2527622212 2024-05-03T14:49:25.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.014+0000 7fc77dac1b80 1 -- 172.21.15.60:0/1410631263 >> 172.21.15.60:0/1410631263 conn(0x557fa0f80b10 msgr2=0x557fa1057d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:49:25.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.014+0000 7fc77dac1b80 1 -- 172.21.15.60:0/1410631263 shutdown_connections 2024-05-03T14:49:25.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.014+0000 7fc77dac1b80 1 -- 172.21.15.60:0/1410631263 wait complete. 2024-05-03T14:49:25.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.015+0000 7fc77dac1b80 20 client.25136 trim_cache size 0 max 16384 2024-05-03T14:49:25.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.038+0000 7fc77dac1b80 1 Processor -- start 2024-05-03T14:49:25.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.038+0000 7fc77dac1b80 1 -- start start 2024-05-03T14:49:25.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.039+0000 7fc77dac1b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fa0f80b10 0x557fa104c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:25.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.039+0000 7fc77dac1b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fa1052580 0x557fa1051a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:25.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.039+0000 7fc77dac1b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fa1056760 0x557fa104ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:25.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.039+0000 7fc77dac1b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x557fa106b210 con 0x557fa0f80b10 2024-05-03T14:49:25.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.039+0000 7fc77dac1b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x557fa106b8e0 con 0x557fa1056760 2024-05-03T14:49:25.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.039+0000 7fc77dac1b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x557fa106bfb0 con 0x557fa1052580 2024-05-03T14:49:25.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.039+0000 7fc76be00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fa0f80b10 0x557fa104c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:25.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.039+0000 7fc76b400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fa1052580 0x557fa1051a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:25.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.039+0000 7fc76b400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fa1052580 0x557fa1051a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.60:59040/0 (socket says 172.21.15.60:59040) 2024-05-03T14:49:25.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.039+0000 7fc76b400700 1 -- 172.21.15.60:0/856904274 learned_addr learned my addr 172.21.15.60:0/856904274 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:49:25.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.039+0000 7fc771200700 1 --2- 172.21.15.60:0/856904274 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fa1056760 0x557fa104ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:25.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.040+0000 7fc76b400700 1 -- 172.21.15.60:0/856904274 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fa1056760 msgr2=0x557fa104ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:49:25.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.040+0000 7fc76b400700 1 --2- 172.21.15.60:0/856904274 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fa1056760 0x557fa104ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:25.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.040+0000 7fc76b400700 1 -- 172.21.15.60:0/856904274 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fa0f80b10 msgr2=0x557fa104c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:49:25.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.040+0000 7fc76b400700 1 --2- 172.21.15.60:0/856904274 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fa0f80b10 0x557fa104c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:25.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.040+0000 7fc76b400700 1 -- 172.21.15.60:0/856904274 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fa10563b0 con 0x557fa1052580 2024-05-03T14:49:25.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.040+0000 7fc76b400700 1 --2- 172.21.15.60:0/856904274 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fa1052580 0x557fa1051a70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc7580098e0 tx=0x7fc7580059d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=711c254d193aeb86 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:49:25.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.040+0000 7fc760c00700 1 -- 172.21.15.60:0/856904274 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc758029030 con 0x557fa1052580 2024-05-03T14:49:25.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.040+0000 7fc760c00700 1 -- 172.21.15.60:0/856904274 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7fc75800fa70 con 0x557fa1052580 2024-05-03T14:49:25.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.040+0000 7fc760c00700 1 -- 172.21.15.60:0/856904274 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7580246e0 con 0x557fa1052580 2024-05-03T14:49:25.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.041+0000 7fc77dac1b80 1 -- 172.21.15.60:0/856904274 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fa1052580 msgr2=0x557fa1051a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:25.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.041+0000 7fc77dac1b80 1 --2- 172.21.15.60:0/856904274 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fa1052580 0x557fa1051a70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc7580098e0 tx=0x7fc7580059d0 comp rx=0 tx=0).stop 2024-05-03T14:49:25.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.041+0000 7fc77dac1b80 1 -- 172.21.15.60:0/856904274 shutdown_connections 2024-05-03T14:49:25.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.041+0000 7fc77dac1b80 1 --2- 172.21.15.60:0/856904274 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fa1056760 0x557fa104ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:25.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.041+0000 7fc77dac1b80 1 --2- 172.21.15.60:0/856904274 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fa0f80b10 0x557fa104c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:25.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.041+0000 7fc77dac1b80 1 --2- 172.21.15.60:0/856904274 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fa1052580 0x557fa1051a70 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:25.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.041+0000 7fc77dac1b80 1 -- 172.21.15.60:0/856904274 >> 172.21.15.60:0/856904274 conn(0x557fa10493e0 msgr2=0x557fa1051360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:49:25.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.041+0000 7fc77dac1b80 1 -- 172.21.15.60:0/856904274 shutdown_connections 2024-05-03T14:49:25.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.041+0000 7fc77dac1b80 1 -- 172.21.15.60:0/856904274 wait complete. 2024-05-03T14:49:25.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.042+0000 7fc77dac1b80 1 -- start start 2024-05-03T14:49:25.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.042+0000 7fc77dac1b80 1 Processor -- start 2024-05-03T14:49:25.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.042+0000 7fc77dac1b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fa0f80b10 0x557fa10747f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:25.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.042+0000 7fc77dac1b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fa1056760 0x557fa1074d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:25.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.042+0000 7fc77dac1b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fa1057080 0x557fa1075270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:25.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.042+0000 7fc77dac1b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x557fa106cd50 con 0x557fa0f80b10 2024-05-03T14:49:25.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.042+0000 7fc77dac1b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x557fa106c680 con 0x557fa1057080 2024-05-03T14:49:25.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.042+0000 7fc77dac1b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x557fa106d420 con 0x557fa1056760 2024-05-03T14:49:25.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.042+0000 7fc77dac1b80 10 client.0 _ll_register_callbacks cb 0x557fa0f4bc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T14:49:25.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.043+0000 7fc76b400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fa1056760 0x557fa1074d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:25.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.043+0000 7fc76b400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fa1056760 0x557fa1074d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.60:59056/0 (socket says 172.21.15.60:59056) 2024-05-03T14:49:25.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.043+0000 7fc76b400700 1 -- 172.21.15.60:0/1052305244 learned_addr learned my addr 172.21.15.60:0/1052305244 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:49:25.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.043+0000 7fc76be00700 1 --2- 172.21.15.60:0/1052305244 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fa0f80b10 0x557fa10747f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:25.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.043+0000 7fc771200700 1 --2- 172.21.15.60:0/1052305244 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fa1057080 0x557fa1075270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:25.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.043+0000 7fc76b400700 1 -- 172.21.15.60:0/1052305244 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fa1057080 msgr2=0x557fa1075270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:49:25.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.043+0000 7fc76b400700 1 --2- 172.21.15.60:0/1052305244 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fa1057080 0x557fa1075270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:25.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.043+0000 7fc76b400700 1 -- 172.21.15.60:0/1052305244 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fa0f80b10 msgr2=0x557fa10747f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:49:25.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.043+0000 7fc76b400700 1 --2- 172.21.15.60:0/1052305244 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fa0f80b10 0x557fa10747f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:25.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.043+0000 7fc76b400700 1 -- 172.21.15.60:0/1052305244 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fa1073600 con 0x557fa1056760 2024-05-03T14:49:25.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.044+0000 7fc76b400700 1 --2- 172.21.15.60:0/1052305244 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fa1056760 0x557fa1074d30 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc758000c00 tx=0x7fc758005c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40640fe9d1125ea4 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:49:25.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.044+0000 7fc756000700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-05-03T14:49:25.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.044+0000 7fc756000700 1 -- 172.21.15.60:0/1052305244 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc758003880 con 0x557fa1056760 2024-05-03T14:49:25.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.044+0000 7fc77dac1b80 1 -- 172.21.15.60:0/1052305244 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc75800fa70 con 0x557fa1056760 2024-05-03T14:49:25.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.044+0000 7fc756000700 1 -- 172.21.15.60:0/1052305244 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7fc758003a20 con 0x557fa1056760 2024-05-03T14:49:25.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.044+0000 7fc756000700 1 -- 172.21.15.60:0/1052305244 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc75804b050 con 0x557fa1056760 2024-05-03T14:49:25.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.044+0000 7fc77dac1b80 10 client.25150 fetch_fsmap learned FSMap version 29 2024-05-03T14:49:25.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.044+0000 7fc77dac1b80 1 -- 172.21.15.60:0/1052305244 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557fa106f5a0 con 0x557fa1056760 2024-05-03T14:49:25.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.045+0000 7fc756000700 1 -- 172.21.15.60:0/1052305244 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7fc75800fa70 con 0x557fa1056760 2024-05-03T14:49:25.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.045+0000 7fc756000700 1 -- 172.21.15.60:0/1052305244 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc758046030 con 0x557fa1056760 2024-05-03T14:49:25.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.045+0000 7fc77dac1b80 10 client.25150 fetch_fsmap finished waiting for FSMap version 29 2024-05-03T14:49:25.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.045+0000 7fc77dac1b80 10 client.25150 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-03T14:49:25.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.045+0000 7fc77dac1b80 20 client.25150 populate_metadata read hostname 'smithi060' 2024-05-03T14:49:25.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.045+0000 7fc77dac1b80 1 --2- 172.21.15.60:0/1052305244 >> [v2:172.21.15.149:6838/2527622212,v1:172.21.15.149:6839/2527622212] conn(0x557fa10dc960 0x557fa10db670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:25.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.045+0000 7fc77dac1b80 4 client.25150 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-03T14:49:25.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.045+0000 7fc77dac1b80 1 -- 172.21.15.60:0/1052305244 --> [v2:172.21.15.149:6838/2527622212,v1:172.21.15.149:6839/2527622212] -- command(tid 0: {"prefix": "status"}) -- 0x557fa0fb23b0 con 0x557fa10dc960 2024-05-03T14:49:25.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.045+0000 7fc76be00700 1 --2- 172.21.15.60:0/1052305244 >> [v2:172.21.15.149:6838/2527622212,v1:172.21.15.149:6839/2527622212] conn(0x557fa10dc960 0x557fa10db670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:25.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.046+0000 7fc76be00700 1 --2- 172.21.15.60:0/1052305244 >> [v2:172.21.15.149:6838/2527622212,v1:172.21.15.149:6839/2527622212] conn(0x557fa10dc960 0x557fa10db670 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ee16af9dee706407 server_cookie=71d5aeaf402a0a3e in_seq=0 out_seq=0 2024-05-03T14:49:25.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.046+0000 7fc756000700 10 client.25150 ms_handle_connect on v2:172.21.15.149:6838/2527622212 2024-05-03T14:49:25.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.046+0000 7fc756000700 1 -- 172.21.15.60:0/1052305244 <== mds.2 v2:172.21.15.149:6838/2527622212 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x557fa0fb23b0 con 0x557fa10dc960 2024-05-03T14:49:25.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.046+0000 7fc756000700 10 client.25150 handle_command_reply: tid=0 2024-05-03T14:49:25.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.046+0000 7fc77dac1b80 1 client.25150 shutdown 2024-05-03T14:49:25.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.046+0000 7fc77dac1b80 1 -- 172.21.15.60:0/1052305244 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fa1056760 msgr2=0x557fa1074d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:25.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.046+0000 7fc77dac1b80 1 --2- 172.21.15.60:0/1052305244 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fa1056760 0x557fa1074d30 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc758000c00 tx=0x7fc758005c30 comp rx=0 tx=0).stop 2024-05-03T14:49:25.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.046+0000 7fc77dac1b80 1 -- 172.21.15.60:0/1052305244 shutdown_connections 2024-05-03T14:49:25.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.047+0000 7fc77dac1b80 1 --2- 172.21.15.60:0/1052305244 >> [v2:172.21.15.149:6838/2527622212,v1:172.21.15.149:6839/2527622212] conn(0x557fa10dc960 0x557fa10db670 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:25.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.047+0000 7fc77dac1b80 1 --2- 172.21.15.60:0/1052305244 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fa1057080 0x557fa1075270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:25.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.047+0000 7fc77dac1b80 1 --2- 172.21.15.60:0/1052305244 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fa0f80b10 0x557fa10747f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:25.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.047+0000 7fc77dac1b80 1 --2- 172.21.15.60:0/1052305244 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fa1056760 0x557fa1074d30 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:25.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.047+0000 7fc77dac1b80 1 -- 172.21.15.60:0/1052305244 >> 172.21.15.60:0/1052305244 conn(0x557fa10493e0 msgr2=0x557fa1059760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:49:25.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.047+0000 7fc756000700 0 client.25150 ms_handle_reset on v2:172.21.15.149:6838/2527622212 2024-05-03T14:49:25.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.047+0000 7fc77dac1b80 1 -- 172.21.15.60:0/1052305244 shutdown_connections 2024-05-03T14:49:25.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.047+0000 7fc77dac1b80 1 -- 172.21.15.60:0/1052305244 wait complete. 2024-05-03T14:49:25.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.047+0000 7fc77dac1b80 20 client.25150 trim_cache size 0 max 16384 2024-05-03T14:49:25.052 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-05-03T14:49:25.052 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "285d64b2-095a-11ef-bc97-c7b262605968", 2024-05-03T14:49:25.052 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 2, 2024-05-03T14:49:25.052 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 24487, 2024-05-03T14:49:25.052 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-05-03T14:49:25.052 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-05-03T14:49:25.052 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-05-03T14:49:25.052 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 210.54723806000001, 2024-05-03T14:49:25.052 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 29, 2024-05-03T14:49:25.052 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 89, 2024-05-03T14:49:25.052 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 87, 2024-05-03T14:49:25.053 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 325.16887559700001 2024-05-03T14:49:25.053 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-05-03T14:49:25.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.047+0000 7fc773200700 1 -- 172.21.15.60:0/4170190590 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fc7500b2910 msgr2=0x7fc7500b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:25.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.048+0000 7fc773200700 1 --2- 172.21.15.60:0/4170190590 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fc7500b2910 0x7fc7500b4dd0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc764019bb0 tx=0x7fc76400f000 comp rx=0 tx=0).stop 2024-05-03T14:49:25.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.048+0000 7fc773200700 1 -- 172.21.15.60:0/4170190590 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc76c0756d0 msgr2=0x7fc76c19c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:25.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.048+0000 7fc773200700 1 --2- 172.21.15.60:0/4170190590 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc76c0756d0 0x7fc76c19c020 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc75c00e9a0 tx=0x7fc75c00ecb0 comp rx=0 tx=0).stop 2024-05-03T14:49:25.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.048+0000 7fc773200700 1 -- 172.21.15.60:0/4170190590 shutdown_connections 2024-05-03T14:49:25.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.048+0000 7fc773200700 1 --2- 172.21.15.60:0/4170190590 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fc7500b2910 0x7fc7500b4dd0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:25.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.048+0000 7fc773200700 1 --2- 172.21.15.60:0/4170190590 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc76c10ff00 0x7fc76c1a08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:25.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.048+0000 7fc773200700 1 --2- 172.21.15.60:0/4170190590 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc76c0756d0 0x7fc76c19c020 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:25.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.048+0000 7fc773200700 1 --2- 172.21.15.60:0/4170190590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc76c075ff0 0x7fc76c19c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:25.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.048+0000 7fc773200700 1 -- 172.21.15.60:0/4170190590 >> 172.21.15.60:0/4170190590 conn(0x7fc76c07a600 msgr2=0x7fc76c101870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:49:25.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.048+0000 7fc773200700 1 -- 172.21.15.60:0/4170190590 shutdown_connections 2024-05-03T14:49:25.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:25.048+0000 7fc773200700 1 -- 172.21.15.60:0/4170190590 wait complete. 2024-05-03T14:49:25.774 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:49:25 smithi060 ceph-mon[28873]: pgmap v479: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:49:25.850 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph tell mds.b status 2024-05-03T14:49:25.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:49:25 smithi090 ceph-mon[33568]: pgmap v479: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:49:25.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:49:25 smithi149 ceph-mon[33332]: pgmap v479: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:49:26.328 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:49:27.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:49:27 smithi060 ceph-mon[28873]: pgmap v480: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:49:27.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:49:27 smithi090 ceph-mon[33568]: pgmap v480: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:49:27.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:49:27 smithi149 ceph-mon[33332]: pgmap v480: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:49:28.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.567+0000 7fb1a8a00700 1 -- 172.21.15.60:0/1812910208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1a410a3a0 msgr2=0x7fb1a4116840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:28.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.567+0000 7fb1a8a00700 1 --2- 172.21.15.60:0/1812910208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1a410a3a0 0x7fb1a4116840 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb19c00b0a0 tx=0x7fb19c00b3b0 comp rx=0 tx=0).stop 2024-05-03T14:49:28.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.567+0000 7fb1a8a00700 1 -- 172.21.15.60:0/1812910208 shutdown_connections 2024-05-03T14:49:28.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.567+0000 7fb1a8a00700 1 --2- 172.21.15.60:0/1812910208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1a410a3a0 0x7fb1a4116840 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:28.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.567+0000 7fb1a8a00700 1 --2- 172.21.15.60:0/1812910208 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb1a410e8b0 0x7fb1a410ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:28.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.567+0000 7fb1a8a00700 1 --2- 172.21.15.60:0/1812910208 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb1a41099e0 0x7fb1a4109e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:28.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.567+0000 7fb1a8a00700 1 -- 172.21.15.60:0/1812910208 >> 172.21.15.60:0/1812910208 conn(0x7fb1a4079230 msgr2=0x7fb1a41082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:49:28.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.567+0000 7fb1a8a00700 1 -- 172.21.15.60:0/1812910208 shutdown_connections 2024-05-03T14:49:28.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.567+0000 7fb1a8a00700 1 -- 172.21.15.60:0/1812910208 wait complete. 2024-05-03T14:49:28.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.568+0000 7fb1a8a00700 1 Processor -- start 2024-05-03T14:49:28.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.568+0000 7fb1a8a00700 1 -- start start 2024-05-03T14:49:28.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.569+0000 7fb1a8a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1a41099e0 0x7fb1a41a0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:28.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.569+0000 7fb1a8a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb1a410a3a0 0x7fb1a41a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:28.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.569+0000 7fb1a8a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb1a410e8b0 0x7fb1a41a4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:28.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.569+0000 7fb1a8a00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a4119e70 con 0x7fb1a41099e0 2024-05-03T14:49:28.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.569+0000 7fb1a8a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a4119cf0 con 0x7fb1a410a3a0 2024-05-03T14:49:28.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.569+0000 7fb1a8a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a4119ff0 con 0x7fb1a410e8b0 2024-05-03T14:49:28.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.569+0000 7fb1a2800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1a41099e0 0x7fb1a41a0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:28.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.569+0000 7fb1a2800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1a41099e0 0x7fb1a41a0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36528/0 (socket says 172.21.15.60:36528) 2024-05-03T14:49:28.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.569+0000 7fb1a3200700 1 --2- 172.21.15.60:0/3301256718 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb1a410e8b0 0x7fb1a41a4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:28.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.569+0000 7fb1a2800700 1 -- 172.21.15.60:0/3301256718 learned_addr learned my addr 172.21.15.60:0/3301256718 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:49:28.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.569+0000 7fb1a1e00700 1 --2- 172.21.15.60:0/3301256718 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb1a410a3a0 0x7fb1a41a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:28.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.570+0000 7fb1a3200700 1 -- 172.21.15.60:0/3301256718 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb1a410a3a0 msgr2=0x7fb1a41a0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:28.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.570+0000 7fb1a3200700 1 --2- 172.21.15.60:0/3301256718 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb1a410a3a0 0x7fb1a41a0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:28.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.570+0000 7fb1a3200700 1 -- 172.21.15.60:0/3301256718 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1a41099e0 msgr2=0x7fb1a41a0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:28.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.570+0000 7fb1a3200700 1 --2- 172.21.15.60:0/3301256718 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1a41099e0 0x7fb1a41a0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:28.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.570+0000 7fb1a3200700 1 -- 172.21.15.60:0/3301256718 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1a41a51c0 con 0x7fb1a410e8b0 2024-05-03T14:49:28.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.570+0000 7fb1a1e00700 1 --2- 172.21.15.60:0/3301256718 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb1a410a3a0 0x7fb1a41a0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:49:28.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.570+0000 7fb1a3200700 1 --2- 172.21.15.60:0/3301256718 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb1a410e8b0 0x7fb1a41a4a80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb19c00b070 tx=0x7fb19c004c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:49:28.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.571+0000 7fb19b400700 1 -- 172.21.15.60:0/3301256718 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19c021070 con 0x7fb1a410e8b0 2024-05-03T14:49:28.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.571+0000 7fb1a8a00700 1 -- 172.21.15.60:0/3301256718 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1a41a5450 con 0x7fb1a410e8b0 2024-05-03T14:49:28.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.571+0000 7fb19b400700 1 -- 172.21.15.60:0/3301256718 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7fb19c007b80 con 0x7fb1a410e8b0 2024-05-03T14:49:28.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.571+0000 7fb19b400700 1 -- 172.21.15.60:0/3301256718 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19c0052a0 con 0x7fb1a410e8b0 2024-05-03T14:49:28.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.571+0000 7fb1a8a00700 1 -- 172.21.15.60:0/3301256718 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1a41acd30 con 0x7fb1a410e8b0 2024-05-03T14:49:28.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.574+0000 7fb19b400700 1 -- 172.21.15.60:0/3301256718 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fb19c024070 con 0x7fb1a410e8b0 2024-05-03T14:49:28.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.576+0000 7fb1b3d22b80 1 Processor -- start 2024-05-03T14:49:28.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.576+0000 7fb19b400700 1 --2- 172.21.15.60:0/3301256718 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fb1880b2910 0x7fb1880b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:28.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.576+0000 7fb1b3d22b80 1 -- start start 2024-05-03T14:49:28.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.576+0000 7fb19b400700 1 -- 172.21.15.60:0/3301256718 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7fb19c0e3610 con 0x7fb1a410e8b0 2024-05-03T14:49:28.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.576+0000 7fb1a1e00700 1 --2- 172.21.15.60:0/3301256718 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fb1880b2910 0x7fb1880b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:28.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.576+0000 7fb1b3d22b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561115cceb80 0x561115ccef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:28.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.576+0000 7fb1b3d22b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561115cc8ee0 0x561115cc9360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:28.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.576+0000 7fb1b3d22b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561115cc98a0 0x561115cd5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:28.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.577+0000 7fb1b3d22b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x561115c1b9d0 con 0x561115cc98a0 2024-05-03T14:49:28.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.577+0000 7fb1b3d22b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561115bb9710 con 0x561115cc8ee0 2024-05-03T14:49:28.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.577+0000 7fb1b3d22b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x561115c17990 con 0x561115cceb80 2024-05-03T14:49:28.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.577+0000 7fb1a2800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561115cceb80 0x561115ccef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:28.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.577+0000 7fb1a3200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561115cc98a0 0x561115cd5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:28.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.577+0000 7fb1a2800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561115cceb80 0x561115ccef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.60:59084/0 (socket says 172.21.15.60:59084) 2024-05-03T14:49:28.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.577+0000 7fb1a2800700 1 -- 172.21.15.60:0/2185695268 learned_addr learned my addr 172.21.15.60:0/2185695268 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:49:28.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.577+0000 7fb1a1e00700 1 --2- 172.21.15.60:0/2185695268 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561115cc8ee0 0x561115cc9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:28.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.577+0000 7fb1a3200700 1 -- 172.21.15.60:0/2185695268 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561115cceb80 msgr2=0x561115ccef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:49:28.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.577+0000 7fb1a3200700 1 --2- 172.21.15.60:0/2185695268 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561115cceb80 0x561115ccef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:28.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.577+0000 7fb1a3200700 1 -- 172.21.15.60:0/2185695268 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561115cc8ee0 msgr2=0x561115cc9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:49:28.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.577+0000 7fb1a3200700 1 --2- 172.21.15.60:0/2185695268 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561115cc8ee0 0x561115cc9360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:28.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.577+0000 7fb1a3200700 1 -- 172.21.15.60:0/2185695268 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561115b68880 con 0x561115cc98a0 2024-05-03T14:49:28.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.577+0000 7fb1a1e00700 1 --2- 172.21.15.60:0/2185695268 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561115cc8ee0 0x561115cc9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T14:49:28.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.578+0000 7fb1a1e00700 1 --2- 172.21.15.60:0/3301256718 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fb1880b2910 0x7fb1880b4dd0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb1a41093e0 tx=0x7fb190009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:49:28.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.578+0000 7fb1a3200700 1 --2- 172.21.15.60:0/2185695268 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561115cc98a0 0x561115cd5940 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb19c0e0be0 tx=0x7fb19c004e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5e3515f865cbf94 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:49:28.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.578+0000 7fb18fe00700 1 -- 172.21.15.60:0/2185695268 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19c02b070 con 0x561115cc98a0 2024-05-03T14:49:28.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.578+0000 7fb18fe00700 1 -- 172.21.15.60:0/2185695268 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7fb19c0aa290 con 0x561115cc98a0 2024-05-03T14:49:28.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.578+0000 7fb18fe00700 1 -- 172.21.15.60:0/2185695268 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19c007d30 con 0x561115cc98a0 2024-05-03T14:49:28.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.579+0000 7fb1b3d22b80 1 -- 172.21.15.60:0/2185695268 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561115cc98a0 msgr2=0x561115cd5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:28.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.579+0000 7fb1b3d22b80 1 --2- 172.21.15.60:0/2185695268 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561115cc98a0 0x561115cd5940 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb19c0e0be0 tx=0x7fb19c004e70 comp rx=0 tx=0).stop 2024-05-03T14:49:28.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.579+0000 7fb1b3d22b80 1 -- 172.21.15.60:0/2185695268 shutdown_connections 2024-05-03T14:49:28.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.579+0000 7fb1b3d22b80 1 --2- 172.21.15.60:0/2185695268 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561115cc98a0 0x561115cd5940 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:28.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.579+0000 7fb1b3d22b80 1 --2- 172.21.15.60:0/2185695268 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561115cceb80 0x561115ccef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:28.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.579+0000 7fb1b3d22b80 1 --2- 172.21.15.60:0/2185695268 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561115cc8ee0 0x561115cc9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:28.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.579+0000 7fb1b3d22b80 1 -- 172.21.15.60:0/2185695268 >> 172.21.15.60:0/2185695268 conn(0x561115c41680 msgr2=0x561115cc3b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:49:28.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.579+0000 7fb1b3d22b80 1 -- 172.21.15.60:0/2185695268 shutdown_connections 2024-05-03T14:49:28.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.579+0000 7fb1b3d22b80 1 -- 172.21.15.60:0/2185695268 wait complete. 2024-05-03T14:49:28.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.580+0000 7fb1b3d22b80 1 -- start start 2024-05-03T14:49:28.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.580+0000 7fb1b3d22b80 1 Processor -- start 2024-05-03T14:49:28.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.580+0000 7fb1b3d22b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561115cc8ee0 0x561115cd55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:28.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.580+0000 7fb1b3d22b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561115cc98a0 0x561115cd29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:28.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.580+0000 7fb1b3d22b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561115cceb80 0x561115cd2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:28.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.580+0000 7fb1b3d22b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x561115ccf650 con 0x561115cc8ee0 2024-05-03T14:49:28.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.580+0000 7fb1b3d22b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561115b81ad0 con 0x561115cceb80 2024-05-03T14:49:28.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.580+0000 7fb1b3d22b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x561115cd5e80 con 0x561115cc98a0 2024-05-03T14:49:28.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.581+0000 7fb1b3d22b80 10 client.0 _ll_register_callbacks cb 0x561115bc2c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T14:49:28.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.581+0000 7fb1a3200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561115cceb80 0x561115cd2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:28.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.581+0000 7fb1a3200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561115cceb80 0x561115cd2ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.60:53792/0 (socket says 172.21.15.60:53792) 2024-05-03T14:49:28.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.581+0000 7fb1a3200700 1 -- 172.21.15.60:0/1950048823 learned_addr learned my addr 172.21.15.60:0/1950048823 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:49:28.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.581+0000 7fb1a2800700 1 --2- 172.21.15.60:0/1950048823 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561115cc8ee0 0x561115cd55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:28.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.581+0000 7fb1a1e00700 1 --2- 172.21.15.60:0/1950048823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561115cc98a0 0x561115cd29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:28.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.581+0000 7fb1a3200700 1 -- 172.21.15.60:0/1950048823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561115cc98a0 msgr2=0x561115cd29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:49:28.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.581+0000 7fb1a3200700 1 --2- 172.21.15.60:0/1950048823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561115cc98a0 0x561115cd29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:28.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.581+0000 7fb1a3200700 1 -- 172.21.15.60:0/1950048823 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561115cc8ee0 msgr2=0x561115cd55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:49:28.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.581+0000 7fb1a3200700 1 --2- 172.21.15.60:0/1950048823 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561115cc8ee0 0x561115cd55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:28.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.581+0000 7fb1a3200700 1 -- 172.21.15.60:0/1950048823 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561115be13d0 con 0x561115cceb80 2024-05-03T14:49:28.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.582+0000 7fb1a3200700 1 --2- 172.21.15.60:0/1950048823 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561115cceb80 0x561115cd2ee0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb19c00b780 tx=0x7fb19c00b3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2da49d086df17013 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:49:28.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.582+0000 7fb18cc00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-03T14:49:28.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.582+0000 7fb18cc00700 1 -- 172.21.15.60:0/1950048823 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19c02b070 con 0x561115cceb80 2024-05-03T14:49:28.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.582+0000 7fb1b3d22b80 1 -- 172.21.15.60:0/1950048823 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb19c0aa290 con 0x561115cceb80 2024-05-03T14:49:28.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.582+0000 7fb18cc00700 1 -- 172.21.15.60:0/1950048823 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7fb19c0e0e30 con 0x561115cceb80 2024-05-03T14:49:28.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.582+0000 7fb18cc00700 1 -- 172.21.15.60:0/1950048823 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb19c0aae80 con 0x561115cceb80 2024-05-03T14:49:28.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.582+0000 7fb1b3d22b80 10 client.25154 fetch_fsmap learned FSMap version 29 2024-05-03T14:49:28.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.582+0000 7fb1b3d22b80 1 -- 172.21.15.60:0/1950048823 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561115ce0410 con 0x561115cceb80 2024-05-03T14:49:28.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.582+0000 7fb18cc00700 1 -- 172.21.15.60:0/1950048823 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7fb19c0aa290 con 0x561115cceb80 2024-05-03T14:49:28.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.583+0000 7fb18cc00700 1 -- 172.21.15.60:0/1950048823 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb19c011070 con 0x561115cceb80 2024-05-03T14:49:28.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.583+0000 7fb1b3d22b80 10 client.25154 fetch_fsmap finished waiting for FSMap version 29 2024-05-03T14:49:28.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.583+0000 7fb1b3d22b80 10 client.25154 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-03T14:49:28.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.583+0000 7fb1b3d22b80 20 client.25154 populate_metadata read hostname 'smithi060' 2024-05-03T14:49:28.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.583+0000 7fb1b3d22b80 1 --2- 172.21.15.60:0/1950048823 >> [v2:172.21.15.90:6838/3512381172,v1:172.21.15.90:6839/3512381172] conn(0x561115ce27f0 0x561115ce3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:28.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.583+0000 7fb1b3d22b80 4 client.25154 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T14:49:28.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.583+0000 7fb1b3d22b80 1 -- 172.21.15.60:0/1950048823 --> [v2:172.21.15.90:6838/3512381172,v1:172.21.15.90:6839/3512381172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561115c95fa0 con 0x561115ce27f0 2024-05-03T14:49:28.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.583+0000 7fb1a2800700 1 --2- 172.21.15.60:0/1950048823 >> [v2:172.21.15.90:6838/3512381172,v1:172.21.15.90:6839/3512381172] conn(0x561115ce27f0 0x561115ce3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:28.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.584+0000 7fb1a2800700 1 --2- 172.21.15.60:0/1950048823 >> [v2:172.21.15.90:6838/3512381172,v1:172.21.15.90:6839/3512381172] conn(0x561115ce27f0 0x561115ce3c60 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c9122529e8469ae1 server_cookie=a56b216c2314a416 in_seq=0 out_seq=0 2024-05-03T14:49:28.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.584+0000 7fb18cc00700 10 client.25154 ms_handle_connect on v2:172.21.15.90:6838/3512381172 2024-05-03T14:49:28.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.585+0000 7fb18cc00700 1 -- 172.21.15.60:0/1950048823 <== mds.1 v2:172.21.15.90:6838/3512381172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561115c95fa0 con 0x561115ce27f0 2024-05-03T14:49:28.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.585+0000 7fb18cc00700 10 client.25154 handle_command_reply: tid=0 2024-05-03T14:49:28.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.585+0000 7fb1b3d22b80 1 client.25154 shutdown 2024-05-03T14:49:28.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.585+0000 7fb1b3d22b80 1 -- 172.21.15.60:0/1950048823 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561115cceb80 msgr2=0x561115cd2ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:28.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.585+0000 7fb1b3d22b80 1 --2- 172.21.15.60:0/1950048823 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561115cceb80 0x561115cd2ee0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb19c00b780 tx=0x7fb19c00b3b0 comp rx=0 tx=0).stop 2024-05-03T14:49:28.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.585+0000 7fb1b3d22b80 1 -- 172.21.15.60:0/1950048823 shutdown_connections 2024-05-03T14:49:28.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.585+0000 7fb1b3d22b80 1 --2- 172.21.15.60:0/1950048823 >> [v2:172.21.15.90:6838/3512381172,v1:172.21.15.90:6839/3512381172] conn(0x561115ce27f0 0x561115ce3c60 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:28.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.585+0000 7fb1b3d22b80 1 --2- 172.21.15.60:0/1950048823 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561115cceb80 0x561115cd2ee0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:28.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.585+0000 7fb1b3d22b80 1 --2- 172.21.15.60:0/1950048823 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561115cc8ee0 0x561115cd55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:28.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.585+0000 7fb1b3d22b80 1 --2- 172.21.15.60:0/1950048823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561115cc98a0 0x561115cd29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:28.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.585+0000 7fb1b3d22b80 1 -- 172.21.15.60:0/1950048823 >> 172.21.15.60:0/1950048823 conn(0x561115c41680 msgr2=0x561115cc7ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:49:28.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.585+0000 7fb18cc00700 0 client.25154 ms_handle_reset on v2:172.21.15.90:6838/3512381172 2024-05-03T14:49:28.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.585+0000 7fb1b3d22b80 1 -- 172.21.15.60:0/1950048823 shutdown_connections 2024-05-03T14:49:28.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.585+0000 7fb1b3d22b80 1 -- 172.21.15.60:0/1950048823 wait complete. 2024-05-03T14:49:28.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.586+0000 7fb1b3d22b80 20 client.25154 trim_cache size 0 max 16384 2024-05-03T14:49:28.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.609+0000 7fb1b3d22b80 1 Processor -- start 2024-05-03T14:49:28.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.609+0000 7fb1b3d22b80 1 -- start start 2024-05-03T14:49:28.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.609+0000 7fb1b3d22b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561115cc8ee0 0x561115cc5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:28.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.609+0000 7fb1b3d22b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561115cc98a0 0x561115ce1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:28.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.609+0000 7fb1b3d22b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561115cceb80 0x561115cd1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:28.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.609+0000 7fb1b3d22b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x561115ce1fa0 con 0x561115cc8ee0 2024-05-03T14:49:28.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.609+0000 7fb1b3d22b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561115ce2670 con 0x561115cc98a0 2024-05-03T14:49:28.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.609+0000 7fb1b3d22b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x561115ce2d40 con 0x561115cceb80 2024-05-03T14:49:28.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.609+0000 7fb1a2800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561115cc8ee0 0x561115cc5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:28.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.609+0000 7fb1a2800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561115cc8ee0 0x561115cc5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36558/0 (socket says 172.21.15.60:36558) 2024-05-03T14:49:28.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.609+0000 7fb1a2800700 1 -- 172.21.15.60:0/4101099066 learned_addr learned my addr 172.21.15.60:0/4101099066 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:49:28.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.610+0000 7fb1a1e00700 1 --2- 172.21.15.60:0/4101099066 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561115cc98a0 0x561115ce1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:28.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.610+0000 7fb1a3200700 1 --2- 172.21.15.60:0/4101099066 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561115cceb80 0x561115cd1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:28.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.610+0000 7fb1a2800700 1 -- 172.21.15.60:0/4101099066 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561115cceb80 msgr2=0x561115cd1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:49:28.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.610+0000 7fb1a2800700 1 --2- 172.21.15.60:0/4101099066 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561115cceb80 0x561115cd1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:28.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.610+0000 7fb1a2800700 1 -- 172.21.15.60:0/4101099066 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561115cc98a0 msgr2=0x561115ce1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:49:28.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.610+0000 7fb1a2800700 1 --2- 172.21.15.60:0/4101099066 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561115cc98a0 0x561115ce1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:28.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.610+0000 7fb1a2800700 1 -- 172.21.15.60:0/4101099066 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561115c95f80 con 0x561115cc8ee0 2024-05-03T14:49:28.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.610+0000 7fb1a2800700 1 --2- 172.21.15.60:0/4101099066 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561115cc8ee0 0x561115cc5540 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x561115cd2220 tx=0x7fb194009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78f7cf5dc6c0cf80 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:49:28.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.611+0000 7fb18f400700 1 -- 172.21.15.60:0/4101099066 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb194023070 con 0x561115cc8ee0 2024-05-03T14:49:28.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.611+0000 7fb18f400700 1 -- 172.21.15.60:0/4101099066 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7fb194010e50 con 0x561115cc8ee0 2024-05-03T14:49:28.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.611+0000 7fb18f400700 1 -- 172.21.15.60:0/4101099066 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19401f110 con 0x561115cc8ee0 2024-05-03T14:49:28.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.611+0000 7fb1b3d22b80 1 -- 172.21.15.60:0/4101099066 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561115cc8ee0 msgr2=0x561115cc5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:28.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.611+0000 7fb1b3d22b80 1 --2- 172.21.15.60:0/4101099066 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561115cc8ee0 0x561115cc5540 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x561115cd2220 tx=0x7fb194009260 comp rx=0 tx=0).stop 2024-05-03T14:49:28.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.611+0000 7fb1b3d22b80 1 -- 172.21.15.60:0/4101099066 shutdown_connections 2024-05-03T14:49:28.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.611+0000 7fb1b3d22b80 1 --2- 172.21.15.60:0/4101099066 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561115cceb80 0x561115cd1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:28.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.611+0000 7fb1b3d22b80 1 --2- 172.21.15.60:0/4101099066 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561115cc8ee0 0x561115cc5540 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:28.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.611+0000 7fb1b3d22b80 1 --2- 172.21.15.60:0/4101099066 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561115cc98a0 0x561115ce1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:28.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.611+0000 7fb1b3d22b80 1 -- 172.21.15.60:0/4101099066 >> 172.21.15.60:0/4101099066 conn(0x561115c41680 msgr2=0x561115cbda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:49:28.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.612+0000 7fb1b3d22b80 1 -- 172.21.15.60:0/4101099066 shutdown_connections 2024-05-03T14:49:28.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.612+0000 7fb1b3d22b80 1 -- 172.21.15.60:0/4101099066 wait complete. 2024-05-03T14:49:28.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.612+0000 7fb1b3d22b80 1 -- start start 2024-05-03T14:49:28.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.612+0000 7fb1b3d22b80 1 Processor -- start 2024-05-03T14:49:28.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.613+0000 7fb1b3d22b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561115cc8ee0 0x561115ced0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:28.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.613+0000 7fb1b3d22b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561115cc98a0 0x561115ced620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:28.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.613+0000 7fb1b3d22b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561115cceb80 0x561115ce8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:28.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.613+0000 7fb1b3d22b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x561115ce3ae0 con 0x561115cc8ee0 2024-05-03T14:49:28.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.613+0000 7fb1b3d22b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561115ce3410 con 0x561115cc98a0 2024-05-03T14:49:28.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.613+0000 7fb1b3d22b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x561115ce41b0 con 0x561115cceb80 2024-05-03T14:49:28.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.613+0000 7fb1b3d22b80 10 client.0 _ll_register_callbacks cb 0x561115bc2c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T14:49:28.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.613+0000 7fb1a2800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561115cc8ee0 0x561115ced0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:28.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.613+0000 7fb1a2800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561115cc8ee0 0x561115ced0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36570/0 (socket says 172.21.15.60:36570) 2024-05-03T14:49:28.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.613+0000 7fb1a2800700 1 -- 172.21.15.60:0/2906062071 learned_addr learned my addr 172.21.15.60:0/2906062071 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:49:28.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.613+0000 7fb1a3200700 1 --2- 172.21.15.60:0/2906062071 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561115cceb80 0x561115ce8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:28.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.613+0000 7fb1a1e00700 1 --2- 172.21.15.60:0/2906062071 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561115cc98a0 0x561115ced620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:28.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.613+0000 7fb1a2800700 1 -- 172.21.15.60:0/2906062071 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561115cceb80 msgr2=0x561115ce8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:49:28.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.613+0000 7fb1a2800700 1 --2- 172.21.15.60:0/2906062071 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561115cceb80 0x561115ce8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:28.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.613+0000 7fb1a2800700 1 -- 172.21.15.60:0/2906062071 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561115cc98a0 msgr2=0x561115ced620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:49:28.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.613+0000 7fb1a2800700 1 --2- 172.21.15.60:0/2906062071 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561115cc98a0 0x561115ced620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:28.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.613+0000 7fb1a2800700 1 -- 172.21.15.60:0/2906062071 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561115cc0450 con 0x561115cc8ee0 2024-05-03T14:49:28.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.614+0000 7fb1a1e00700 1 --2- 172.21.15.60:0/2906062071 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561115cc98a0 0x561115ced620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T14:49:28.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.614+0000 7fb1a2800700 1 --2- 172.21.15.60:0/2906062071 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561115cc8ee0 0x561115ced0e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb194005280 tx=0x7fb1940052b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba9b9a5990f936f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:49:28.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.614+0000 7fb18cc00700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-05-03T14:49:28.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.614+0000 7fb18cc00700 1 -- 172.21.15.60:0/2906062071 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19402b030 con 0x561115cc8ee0 2024-05-03T14:49:28.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.614+0000 7fb18cc00700 1 -- 172.21.15.60:0/2906062071 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7fb1940047c0 con 0x561115cc8ee0 2024-05-03T14:49:28.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.614+0000 7fb1b3d22b80 1 -- 172.21.15.60:0/2906062071 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb194010e50 con 0x561115cc8ee0 2024-05-03T14:49:28.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.615+0000 7fb18cc00700 1 -- 172.21.15.60:0/2906062071 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb194035050 con 0x561115cc8ee0 2024-05-03T14:49:28.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.615+0000 7fb1b3d22b80 10 client.15060 fetch_fsmap learned FSMap version 29 2024-05-03T14:49:28.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.615+0000 7fb1b3d22b80 1 -- 172.21.15.60:0/2906062071 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561115d4e540 con 0x561115cc8ee0 2024-05-03T14:49:28.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.615+0000 7fb18cc00700 1 -- 172.21.15.60:0/2906062071 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7fb194010e50 con 0x561115cc8ee0 2024-05-03T14:49:28.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.615+0000 7fb18cc00700 1 -- 172.21.15.60:0/2906062071 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb19401f7f0 con 0x561115cc8ee0 2024-05-03T14:49:28.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.615+0000 7fb1b3d22b80 10 client.15060 fetch_fsmap finished waiting for FSMap version 29 2024-05-03T14:49:28.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.615+0000 7fb1b3d22b80 10 client.15060 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-03T14:49:28.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.615+0000 7fb1b3d22b80 20 client.15060 populate_metadata read hostname 'smithi060' 2024-05-03T14:49:28.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.615+0000 7fb1b3d22b80 1 --2- 172.21.15.60:0/2906062071 >> [v2:172.21.15.90:6838/3512381172,v1:172.21.15.90:6839/3512381172] conn(0x561115d52de0 0x561115d51af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:28.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.615+0000 7fb1b3d22b80 4 client.15060 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-03T14:49:28.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.615+0000 7fb1b3d22b80 1 -- 172.21.15.60:0/2906062071 --> [v2:172.21.15.90:6838/3512381172,v1:172.21.15.90:6839/3512381172] -- command(tid 0: {"prefix": "status"}) -- 0x561115c293b0 con 0x561115d52de0 2024-05-03T14:49:28.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.616+0000 7fb1a1e00700 1 --2- 172.21.15.60:0/2906062071 >> [v2:172.21.15.90:6838/3512381172,v1:172.21.15.90:6839/3512381172] conn(0x561115d52de0 0x561115d51af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:28.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.616+0000 7fb1a1e00700 1 --2- 172.21.15.60:0/2906062071 >> [v2:172.21.15.90:6838/3512381172,v1:172.21.15.90:6839/3512381172] conn(0x561115d52de0 0x561115d51af0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3d15729e21daff46 server_cookie=16544f8e9915433f in_seq=0 out_seq=0 2024-05-03T14:49:28.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.616+0000 7fb18cc00700 10 client.15060 ms_handle_connect on v2:172.21.15.90:6838/3512381172 2024-05-03T14:49:28.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.617+0000 7fb18cc00700 1 -- 172.21.15.60:0/2906062071 <== mds.1 v2:172.21.15.90:6838/3512381172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x561115c293b0 con 0x561115d52de0 2024-05-03T14:49:28.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.617+0000 7fb18cc00700 10 client.15060 handle_command_reply: tid=0 2024-05-03T14:49:28.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.617+0000 7fb1b3d22b80 1 client.15060 shutdown 2024-05-03T14:49:28.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.617+0000 7fb1b3d22b80 1 -- 172.21.15.60:0/2906062071 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561115cc8ee0 msgr2=0x561115ced0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:28.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.617+0000 7fb1b3d22b80 1 --2- 172.21.15.60:0/2906062071 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561115cc8ee0 0x561115ced0e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb194005280 tx=0x7fb1940052b0 comp rx=0 tx=0).stop 2024-05-03T14:49:28.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.617+0000 7fb1b3d22b80 1 -- 172.21.15.60:0/2906062071 shutdown_connections 2024-05-03T14:49:28.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.617+0000 7fb1b3d22b80 1 --2- 172.21.15.60:0/2906062071 >> [v2:172.21.15.90:6838/3512381172,v1:172.21.15.90:6839/3512381172] conn(0x561115d52de0 0x561115d51af0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:28.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.617+0000 7fb1b3d22b80 1 --2- 172.21.15.60:0/2906062071 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561115cceb80 0x561115ce8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:28.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.617+0000 7fb1b3d22b80 1 --2- 172.21.15.60:0/2906062071 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561115cc8ee0 0x561115ced0e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:28.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.617+0000 7fb1b3d22b80 1 --2- 172.21.15.60:0/2906062071 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561115cc98a0 0x561115ced620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:28.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.617+0000 7fb18cc00700 0 client.15060 ms_handle_reset on v2:172.21.15.90:6838/3512381172 2024-05-03T14:49:28.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.617+0000 7fb1b3d22b80 1 -- 172.21.15.60:0/2906062071 >> 172.21.15.60:0/2906062071 conn(0x561115c41680 msgr2=0x561115cbda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:49:28.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.617+0000 7fb1b3d22b80 1 -- 172.21.15.60:0/2906062071 shutdown_connections 2024-05-03T14:49:28.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.617+0000 7fb1b3d22b80 1 -- 172.21.15.60:0/2906062071 wait complete. 2024-05-03T14:49:28.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.617+0000 7fb1b3d22b80 20 client.15060 trim_cache size 0 max 16384 2024-05-03T14:49:28.622 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-05-03T14:49:28.623 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "285d64b2-095a-11ef-bc97-c7b262605968", 2024-05-03T14:49:28.623 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 1, 2024-05-03T14:49:28.623 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 24509, 2024-05-03T14:49:28.623 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-05-03T14:49:28.623 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-05-03T14:49:28.623 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-05-03T14:49:28.623 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 215.14217170800001, 2024-05-03T14:49:28.623 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 29, 2024-05-03T14:49:28.623 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 89, 2024-05-03T14:49:28.623 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 87, 2024-05-03T14:49:28.623 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 314.27864015 2024-05-03T14:49:28.623 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-05-03T14:49:28.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.618+0000 7fb1a8a00700 1 -- 172.21.15.60:0/3301256718 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fb1880b2910 msgr2=0x7fb1880b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:28.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.618+0000 7fb1a8a00700 1 --2- 172.21.15.60:0/3301256718 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fb1880b2910 0x7fb1880b4dd0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb1a41093e0 tx=0x7fb190009260 comp rx=0 tx=0).stop 2024-05-03T14:49:28.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.618+0000 7fb1a8a00700 1 -- 172.21.15.60:0/3301256718 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb1a410e8b0 msgr2=0x7fb1a41a4a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:28.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.618+0000 7fb1a8a00700 1 --2- 172.21.15.60:0/3301256718 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb1a410e8b0 0x7fb1a41a4a80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb19c00b070 tx=0x7fb19c004c80 comp rx=0 tx=0).stop 2024-05-03T14:49:28.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.618+0000 7fb1a8a00700 1 -- 172.21.15.60:0/3301256718 shutdown_connections 2024-05-03T14:49:28.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.618+0000 7fb1a8a00700 1 --2- 172.21.15.60:0/3301256718 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fb1880b2910 0x7fb1880b4dd0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:28.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.618+0000 7fb1a8a00700 1 --2- 172.21.15.60:0/3301256718 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb1a410e8b0 0x7fb1a41a4a80 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:28.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.618+0000 7fb1a8a00700 1 --2- 172.21.15.60:0/3301256718 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1a41099e0 0x7fb1a41a0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:28.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.618+0000 7fb1a8a00700 1 --2- 172.21.15.60:0/3301256718 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb1a410a3a0 0x7fb1a41a0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:28.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.618+0000 7fb1a8a00700 1 -- 172.21.15.60:0/3301256718 >> 172.21.15.60:0/3301256718 conn(0x7fb1a4079230 msgr2=0x7fb1a4079750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:49:28.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.619+0000 7fb1a8a00700 1 -- 172.21.15.60:0/3301256718 shutdown_connections 2024-05-03T14:49:28.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:28.619+0000 7fb1a8a00700 1 -- 172.21.15.60:0/3301256718 wait complete. 2024-05-03T14:49:29.465 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph tell mds.a status 2024-05-03T14:49:29.836 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:49:29 smithi060 ceph-mon[28873]: pgmap v481: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:49:29.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:49:29 smithi090 ceph-mon[33568]: pgmap v481: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:49:29.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:49:29 smithi149 ceph-mon[33332]: pgmap v481: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:49:29.947 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:49:31.690 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:49:31 smithi060 ceph-mon[28873]: pgmap v482: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:49:31.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:49:31 smithi090 ceph-mon[33568]: pgmap v482: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:49:31.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:49:31 smithi149 ceph-mon[33332]: pgmap v482: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:49:32.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.192+0000 7f7e6a800700 1 -- 172.21.15.60:0/61060361 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e640761f0 msgr2=0x7f7e6410f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:32.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.192+0000 7f7e6a800700 1 --2- 172.21.15.60:0/61060361 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e640761f0 0x7f7e6410f6d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7e50009990 tx=0x7f7e50009ca0 comp rx=0 tx=0).stop 2024-05-03T14:49:32.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.192+0000 7f7e6a800700 1 -- 172.21.15.60:0/61060361 shutdown_connections 2024-05-03T14:49:32.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.192+0000 7f7e6a800700 1 --2- 172.21.15.60:0/61060361 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e6410fc10 0x7f7e64112080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:32.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.192+0000 7f7e6a800700 1 --2- 172.21.15.60:0/61060361 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e640758d0 0x7f7e64075cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:32.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.192+0000 7f7e6a800700 1 --2- 172.21.15.60:0/61060361 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e640761f0 0x7f7e6410f6d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:32.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.192+0000 7f7e6a800700 1 -- 172.21.15.60:0/61060361 >> 172.21.15.60:0/61060361 conn(0x7f7e640fef20 msgr2=0x7f7e64101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:49:32.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.192+0000 7f7e6a800700 1 -- 172.21.15.60:0/61060361 shutdown_connections 2024-05-03T14:49:32.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.192+0000 7f7e6a800700 1 -- 172.21.15.60:0/61060361 wait complete. 2024-05-03T14:49:32.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.194+0000 7f7e6a800700 1 Processor -- start 2024-05-03T14:49:32.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.194+0000 7f7e6a800700 1 -- start start 2024-05-03T14:49:32.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.195+0000 7f7e6a800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e640758d0 0x7f7e6419be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:32.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.195+0000 7f7e6a800700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e640761f0 0x7f7e6419c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:32.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.195+0000 7f7e6a800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e6410fc10 0x7f7e641a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:32.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.195+0000 7f7e6a800700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f7e64115a40 con 0x7f7e640761f0 2024-05-03T14:49:32.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.195+0000 7f7e6a800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7e641158c0 con 0x7f7e6410fc10 2024-05-03T14:49:32.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.195+0000 7f7e6a800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7e64115bc0 con 0x7f7e640758d0 2024-05-03T14:49:32.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.195+0000 7f7e68a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e640761f0 0x7f7e6419c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:32.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.195+0000 7f7e69400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e640758d0 0x7f7e6419be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:32.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.196+0000 7f7e69400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e640758d0 0x7f7e6419be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.60:48020/0 (socket says 172.21.15.60:48020) 2024-05-03T14:49:32.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.196+0000 7f7e69400700 1 -- 172.21.15.60:0/2605151309 learned_addr learned my addr 172.21.15.60:0/2605151309 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:49:32.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.196+0000 7f7e69e00700 1 --2- 172.21.15.60:0/2605151309 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e6410fc10 0x7f7e641a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:32.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.196+0000 7f7e68a00700 1 -- 172.21.15.60:0/2605151309 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e640758d0 msgr2=0x7f7e6419be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:32.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.196+0000 7f7e68a00700 1 --2- 172.21.15.60:0/2605151309 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e640758d0 0x7f7e6419be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:32.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.196+0000 7f7e68a00700 1 -- 172.21.15.60:0/2605151309 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e6410fc10 msgr2=0x7f7e641a06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:32.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.196+0000 7f7e68a00700 1 --2- 172.21.15.60:0/2605151309 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e6410fc10 0x7f7e641a06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:32.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.196+0000 7f7e68a00700 1 -- 172.21.15.60:0/2605151309 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e641a0e20 con 0x7f7e640761f0 2024-05-03T14:49:32.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.196+0000 7f7e68a00700 1 --2- 172.21.15.60:0/2605151309 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e640761f0 0x7f7e6419c350 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7e50006010 tx=0x7f7e500057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:49:32.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.197+0000 7f7e62000700 1 -- 172.21.15.60:0/2605151309 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e50027070 con 0x7f7e640761f0 2024-05-03T14:49:32.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.197+0000 7f7e62000700 1 -- 172.21.15.60:0/2605151309 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f7e50022470 con 0x7f7e640761f0 2024-05-03T14:49:32.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.197+0000 7f7e62000700 1 -- 172.21.15.60:0/2605151309 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e50017650 con 0x7f7e640761f0 2024-05-03T14:49:32.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.197+0000 7f7e6a800700 1 -- 172.21.15.60:0/2605151309 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e641a10b0 con 0x7f7e640761f0 2024-05-03T14:49:32.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.197+0000 7f7e6a800700 1 -- 172.21.15.60:0/2605151309 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e641a8990 con 0x7f7e640761f0 2024-05-03T14:49:32.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.202+0000 7f7e62000700 1 -- 172.21.15.60:0/2605151309 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f7e5002a070 con 0x7f7e640761f0 2024-05-03T14:49:32.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.203+0000 7f7e766ccb80 1 Processor -- start 2024-05-03T14:49:32.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.203+0000 7f7e766ccb80 1 -- start start 2024-05-03T14:49:32.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.204+0000 7f7e766ccb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6091f79f0 0x55d6091f7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:32.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.204+0000 7f7e766ccb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d6091f8310 0x55d6091f1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:32.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.204+0000 7f7e766ccb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d6091f2580 0x55d6091f29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:32.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.204+0000 7f7e766ccb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55d6091449d0 con 0x55d6091f8310 2024-05-03T14:49:32.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.204+0000 7f7e766ccb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55d6090e2710 con 0x55d6091f2580 2024-05-03T14:49:32.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.204+0000 7f7e766ccb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d609140990 con 0x55d6091f79f0 2024-05-03T14:49:32.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.204+0000 7f7e69400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d6091f8310 0x55d6091f1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:32.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.204+0000 7f7e69400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d6091f8310 0x55d6091f1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34266/0 (socket says 172.21.15.60:34266) 2024-05-03T14:49:32.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.204+0000 7f7e69400700 1 -- 172.21.15.60:0/3605867910 learned_addr learned my addr 172.21.15.60:0/3605867910 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:49:32.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.204+0000 7f7e69e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6091f79f0 0x55d6091f7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:32.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.204+0000 7f7e68a00700 1 --2- 172.21.15.60:0/3605867910 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d6091f2580 0x55d6091f29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:32.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.204+0000 7f7e62000700 1 --2- 172.21.15.60:0/2605151309 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f7e4c0b2910 0x7f7e4c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:32.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.205+0000 7f7e62000700 1 -- 172.21.15.60:0/2605151309 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f7e500e34b0 con 0x7f7e640761f0 2024-05-03T14:49:32.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.205+0000 7f7e69400700 1 -- 172.21.15.60:0/3605867910 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6091f79f0 msgr2=0x55d6091f7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:49:32.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.205+0000 7f7e69400700 1 --2- 172.21.15.60:0/3605867910 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6091f79f0 0x55d6091f7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:32.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.205+0000 7f7e69400700 1 -- 172.21.15.60:0/3605867910 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d6091f2580 msgr2=0x55d6091f29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:49:32.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.205+0000 7f7e69400700 1 --2- 172.21.15.60:0/3605867910 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d6091f2580 0x55d6091f29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:32.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.205+0000 7f7e69400700 1 -- 172.21.15.60:0/3605867910 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d609091880 con 0x55d6091f8310 2024-05-03T14:49:32.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.205+0000 7f7e69e00700 1 --2- 172.21.15.60:0/2605151309 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f7e4c0b2910 0x7f7e4c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:32.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.205+0000 7f7e69400700 1 --2- 172.21.15.60:0/3605867910 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d6091f8310 0x55d6091f1f60 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7e5800e660 tx=0x7f7e58009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=487d68e4a0ee9081 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:49:32.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.206+0000 7f7e56a00700 1 -- 172.21.15.60:0/3605867910 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e58003a00 con 0x55d6091f8310 2024-05-03T14:49:32.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.206+0000 7f7e69e00700 1 --2- 172.21.15.60:0/2605151309 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f7e4c0b2910 0x7f7e4c0b4dd0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c005950 tx=0x7f7e5c004750 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:49:32.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.206+0000 7f7e56a00700 1 -- 172.21.15.60:0/3605867910 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f7e5801a070 con 0x55d6091f8310 2024-05-03T14:49:32.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.206+0000 7f7e56a00700 1 -- 172.21.15.60:0/3605867910 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e58007870 con 0x55d6091f8310 2024-05-03T14:49:32.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.207+0000 7f7e766ccb80 1 -- 172.21.15.60:0/3605867910 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d6091f8310 msgr2=0x55d6091f1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:32.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.207+0000 7f7e766ccb80 1 --2- 172.21.15.60:0/3605867910 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d6091f8310 0x55d6091f1f60 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7e5800e660 tx=0x7f7e58009920 comp rx=0 tx=0).stop 2024-05-03T14:49:32.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.207+0000 7f7e766ccb80 1 -- 172.21.15.60:0/3605867910 shutdown_connections 2024-05-03T14:49:32.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.207+0000 7f7e766ccb80 1 --2- 172.21.15.60:0/3605867910 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d6091f2580 0x55d6091f29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:32.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.207+0000 7f7e766ccb80 1 --2- 172.21.15.60:0/3605867910 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6091f79f0 0x55d6091f7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:32.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.207+0000 7f7e766ccb80 1 --2- 172.21.15.60:0/3605867910 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d6091f8310 0x55d6091f1f60 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:32.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.207+0000 7f7e766ccb80 1 -- 172.21.15.60:0/3605867910 >> 172.21.15.60:0/3605867910 conn(0x55d609120b10 msgr2=0x55d6091ec980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:49:32.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.207+0000 7f7e766ccb80 1 -- 172.21.15.60:0/3605867910 shutdown_connections 2024-05-03T14:49:32.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.207+0000 7f7e766ccb80 1 -- 172.21.15.60:0/3605867910 wait complete. 2024-05-03T14:49:32.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.207+0000 7f7e766ccb80 1 -- start start 2024-05-03T14:49:32.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.208+0000 7f7e766ccb80 1 Processor -- start 2024-05-03T14:49:32.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.208+0000 7f7e766ccb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d6091f2580 0x55d6091fdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:32.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.208+0000 7f7e766ccb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d6091f79f0 0x55d6091fb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:32.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.208+0000 7f7e766ccb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6091f8310 0x55d6091fb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:32.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.208+0000 7f7e766ccb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55d6091fed20 con 0x55d6091f2580 2024-05-03T14:49:32.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.208+0000 7f7e766ccb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55d6090aaad0 con 0x55d6091f79f0 2024-05-03T14:49:32.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.208+0000 7f7e766ccb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d6091feea0 con 0x55d6091f8310 2024-05-03T14:49:32.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.208+0000 7f7e766ccb80 10 client.0 _ll_register_callbacks cb 0x55d6090ebc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T14:49:32.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.209+0000 7f7e69400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d6091f2580 0x55d6091fdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:32.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.209+0000 7f7e69400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d6091f2580 0x55d6091fdfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34270/0 (socket says 172.21.15.60:34270) 2024-05-03T14:49:32.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.209+0000 7f7e69400700 1 -- 172.21.15.60:0/3948354679 learned_addr learned my addr 172.21.15.60:0/3948354679 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:49:32.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.209+0000 7f7e68a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d6091f79f0 0x55d6091fb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:32.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.209+0000 7f7e69e00700 1 --2- 172.21.15.60:0/3948354679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6091f8310 0x55d6091fb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:32.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.209+0000 7f7e68a00700 1 -- 172.21.15.60:0/3948354679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6091f8310 msgr2=0x55d6091fb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:49:32.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.209+0000 7f7e68a00700 1 --2- 172.21.15.60:0/3948354679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6091f8310 0x55d6091fb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:32.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.209+0000 7f7e68a00700 1 -- 172.21.15.60:0/3948354679 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d6091f2580 msgr2=0x55d6091fdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:49:32.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.209+0000 7f7e68a00700 1 --2- 172.21.15.60:0/3948354679 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d6091f2580 0x55d6091fdfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:32.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.209+0000 7f7e68a00700 1 -- 172.21.15.60:0/3948354679 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d60910a3d0 con 0x55d6091f79f0 2024-05-03T14:49:32.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.209+0000 7f7e68a00700 1 --2- 172.21.15.60:0/3948354679 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d6091f79f0 0x55d6091fb360 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7e500e0a80 tx=0x7f7e50022900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58b92b7ce6c8eabd server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:49:32.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.209+0000 7f7e4b400700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-03T14:49:32.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.210+0000 7f7e4b400700 1 -- 172.21.15.60:0/3948354679 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e500aa3f0 con 0x55d6091f79f0 2024-05-03T14:49:32.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.210+0000 7f7e4b400700 1 -- 172.21.15.60:0/3948354679 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f7e500b2f60 con 0x55d6091f79f0 2024-05-03T14:49:32.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.210+0000 7f7e766ccb80 1 -- 172.21.15.60:0/3948354679 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e5801a070 con 0x55d6091f79f0 2024-05-03T14:49:32.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.210+0000 7f7e4b400700 1 -- 172.21.15.60:0/3948354679 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7e500a9d90 con 0x55d6091f79f0 2024-05-03T14:49:32.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.210+0000 7f7e766ccb80 10 client.25172 fetch_fsmap learned FSMap version 29 2024-05-03T14:49:32.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.210+0000 7f7e766ccb80 1 -- 172.21.15.60:0/3948354679 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d609209500 con 0x55d6091f79f0 2024-05-03T14:49:32.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.210+0000 7f7e4b400700 1 -- 172.21.15.60:0/3948354679 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f7e5801a070 con 0x55d6091f79f0 2024-05-03T14:49:32.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.210+0000 7f7e4b400700 1 -- 172.21.15.60:0/3948354679 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7e50090b90 con 0x55d6091f79f0 2024-05-03T14:49:32.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.211+0000 7f7e766ccb80 10 client.25172 fetch_fsmap finished waiting for FSMap version 29 2024-05-03T14:49:32.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.211+0000 7f7e766ccb80 10 client.25172 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-03T14:49:32.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.211+0000 7f7e766ccb80 20 client.25172 populate_metadata read hostname 'smithi060' 2024-05-03T14:49:32.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.211+0000 7f7e766ccb80 1 --2- 172.21.15.60:0/3948354679 >> [v2:172.21.15.60:6840/924163182,v1:172.21.15.60:6841/924163182] conn(0x55d60920bba0 0x55d60920d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:32.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.211+0000 7f7e766ccb80 4 client.25172 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T14:49:32.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.211+0000 7f7e766ccb80 1 -- 172.21.15.60:0/3948354679 --> [v2:172.21.15.60:6840/924163182,v1:172.21.15.60:6841/924163182] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d6091bf180 con 0x55d60920bba0 2024-05-03T14:49:32.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.211+0000 7f7e69400700 1 --2- 172.21.15.60:0/3948354679 >> [v2:172.21.15.60:6840/924163182,v1:172.21.15.60:6841/924163182] conn(0x55d60920bba0 0x55d60920d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:32.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.212+0000 7f7e69400700 1 --2- 172.21.15.60:0/3948354679 >> [v2:172.21.15.60:6840/924163182,v1:172.21.15.60:6841/924163182] conn(0x55d60920bba0 0x55d60920d010 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8948dc73f81913a server_cookie=103afd25e097f149 in_seq=0 out_seq=0 2024-05-03T14:49:32.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.212+0000 7f7e4b400700 10 client.25172 ms_handle_connect on v2:172.21.15.60:6840/924163182 2024-05-03T14:49:32.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.212+0000 7f7e4b400700 1 -- 172.21.15.60:0/3948354679 <== mds.0 v2:172.21.15.60:6840/924163182 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d6091bf180 con 0x55d60920bba0 2024-05-03T14:49:32.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.212+0000 7f7e4b400700 10 client.25172 handle_command_reply: tid=0 2024-05-03T14:49:32.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.213+0000 7f7e766ccb80 1 client.25172 shutdown 2024-05-03T14:49:32.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.213+0000 7f7e766ccb80 1 -- 172.21.15.60:0/3948354679 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d6091f79f0 msgr2=0x55d6091fb360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:32.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.213+0000 7f7e766ccb80 1 --2- 172.21.15.60:0/3948354679 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d6091f79f0 0x55d6091fb360 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7e500e0a80 tx=0x7f7e50022900 comp rx=0 tx=0).stop 2024-05-03T14:49:32.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.213+0000 7f7e766ccb80 1 -- 172.21.15.60:0/3948354679 shutdown_connections 2024-05-03T14:49:32.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.213+0000 7f7e766ccb80 1 --2- 172.21.15.60:0/3948354679 >> [v2:172.21.15.60:6840/924163182,v1:172.21.15.60:6841/924163182] conn(0x55d60920bba0 0x55d60920d010 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:32.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.213+0000 7f7e766ccb80 1 --2- 172.21.15.60:0/3948354679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6091f8310 0x55d6091fb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:32.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.213+0000 7f7e766ccb80 1 --2- 172.21.15.60:0/3948354679 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d6091f2580 0x55d6091fdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:32.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.213+0000 7f7e766ccb80 1 --2- 172.21.15.60:0/3948354679 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d6091f79f0 0x55d6091fb360 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:32.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.213+0000 7f7e766ccb80 1 -- 172.21.15.60:0/3948354679 >> 172.21.15.60:0/3948354679 conn(0x55d609120b10 msgr2=0x55d6091ec030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:49:32.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.213+0000 7f7e4b400700 0 client.25172 ms_handle_reset on v2:172.21.15.60:6840/924163182 2024-05-03T14:49:32.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.213+0000 7f7e766ccb80 1 -- 172.21.15.60:0/3948354679 shutdown_connections 2024-05-03T14:49:32.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.213+0000 7f7e766ccb80 1 -- 172.21.15.60:0/3948354679 wait complete. 2024-05-03T14:49:32.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.213+0000 7f7e766ccb80 20 client.25172 trim_cache size 0 max 16384 2024-05-03T14:49:32.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.237+0000 7f7e766ccb80 1 Processor -- start 2024-05-03T14:49:32.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.237+0000 7f7e766ccb80 1 -- start start 2024-05-03T14:49:32.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.237+0000 7f7e766ccb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d6091f2580 0x55d60920b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:32.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.237+0000 7f7e766ccb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6091f79f0 0x55d6091fb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:32.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.237+0000 7f7e766ccb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d6091f8310 0x55d6091f5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:32.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.237+0000 7f7e766ccb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55d60920b350 con 0x55d6091f2580 2024-05-03T14:49:32.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.237+0000 7f7e766ccb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55d60920ba20 con 0x55d6091f8310 2024-05-03T14:49:32.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.237+0000 7f7e766ccb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d60920c0f0 con 0x55d6091f79f0 2024-05-03T14:49:32.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.238+0000 7f7e68a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6091f79f0 0x55d6091fb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:32.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.238+0000 7f7e69400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d6091f2580 0x55d60920b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:32.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.238+0000 7f7e68a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6091f79f0 0x55d6091fb320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.60:48054/0 (socket says 172.21.15.60:48054) 2024-05-03T14:49:32.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.238+0000 7f7e68a00700 1 -- 172.21.15.60:0/164979870 learned_addr learned my addr 172.21.15.60:0/164979870 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:49:32.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.238+0000 7f7e69e00700 1 --2- 172.21.15.60:0/164979870 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d6091f8310 0x55d6091f5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:32.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.238+0000 7f7e68a00700 1 -- 172.21.15.60:0/164979870 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d6091f8310 msgr2=0x55d6091f5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:49:32.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.238+0000 7f7e68a00700 1 --2- 172.21.15.60:0/164979870 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d6091f8310 0x55d6091f5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:32.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.238+0000 7f7e68a00700 1 -- 172.21.15.60:0/164979870 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d6091f2580 msgr2=0x55d60920b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:49:32.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.238+0000 7f7e68a00700 1 --2- 172.21.15.60:0/164979870 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d6091f2580 0x55d60920b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:32.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.238+0000 7f7e68a00700 1 -- 172.21.15.60:0/164979870 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6091f98f0 con 0x55d6091f79f0 2024-05-03T14:49:32.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.238+0000 7f7e69400700 1 --2- 172.21.15.60:0/164979870 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d6091f2580 0x55d60920b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T14:49:32.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.239+0000 7f7e68a00700 1 --2- 172.21.15.60:0/164979870 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6091f79f0 0x55d6091fb320 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7e500098e0 tx=0x7f7e50005c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc1b6308639bceb1 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:49:32.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.239+0000 7f7e56000700 1 -- 172.21.15.60:0/164979870 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e500a9f70 con 0x55d6091f79f0 2024-05-03T14:49:32.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.239+0000 7f7e56000700 1 -- 172.21.15.60:0/164979870 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f7e500aad20 con 0x55d6091f79f0 2024-05-03T14:49:32.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.239+0000 7f7e56000700 1 -- 172.21.15.60:0/164979870 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e500b2f60 con 0x55d6091f79f0 2024-05-03T14:49:32.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.239+0000 7f7e766ccb80 1 -- 172.21.15.60:0/164979870 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6091f79f0 msgr2=0x55d6091fb320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:32.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.239+0000 7f7e766ccb80 1 --2- 172.21.15.60:0/164979870 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6091f79f0 0x55d6091fb320 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7e500098e0 tx=0x7f7e50005c60 comp rx=0 tx=0).stop 2024-05-03T14:49:32.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.240+0000 7f7e766ccb80 1 -- 172.21.15.60:0/164979870 shutdown_connections 2024-05-03T14:49:32.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.240+0000 7f7e766ccb80 1 --2- 172.21.15.60:0/164979870 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d6091f8310 0x55d6091f5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:32.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.240+0000 7f7e766ccb80 1 --2- 172.21.15.60:0/164979870 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d6091f2580 0x55d60920b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:32.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.240+0000 7f7e766ccb80 1 --2- 172.21.15.60:0/164979870 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6091f79f0 0x55d6091fb320 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:32.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.240+0000 7f7e766ccb80 1 -- 172.21.15.60:0/164979870 >> 172.21.15.60:0/164979870 conn(0x55d609120b10 msgr2=0x55d6091e9070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:49:32.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.240+0000 7f7e766ccb80 1 -- 172.21.15.60:0/164979870 shutdown_connections 2024-05-03T14:49:32.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.240+0000 7f7e766ccb80 1 -- 172.21.15.60:0/164979870 wait complete. 2024-05-03T14:49:32.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.240+0000 7f7e766ccb80 1 -- start start 2024-05-03T14:49:32.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.240+0000 7f7e766ccb80 1 Processor -- start 2024-05-03T14:49:32.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.241+0000 7f7e766ccb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d6091f2580 0x55d609216490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:32.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.241+0000 7f7e766ccb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6091f79f0 0x55d6092169d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:32.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.241+0000 7f7e766ccb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d6091f8310 0x55d609211440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:32.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.241+0000 7f7e766ccb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55d60920ce90 con 0x55d6091f8310 2024-05-03T14:49:32.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.241+0000 7f7e766ccb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55d60920c7c0 con 0x55d6091f2580 2024-05-03T14:49:32.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.241+0000 7f7e766ccb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d60920d560 con 0x55d6091f79f0 2024-05-03T14:49:32.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.241+0000 7f7e766ccb80 10 client.0 _ll_register_callbacks cb 0x55d6090ebc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T14:49:32.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.241+0000 7f7e69400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d6091f2580 0x55d609216490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:32.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.241+0000 7f7e68a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6091f79f0 0x55d6092169d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:32.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.241+0000 7f7e69400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d6091f2580 0x55d609216490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.60:36372/0 (socket says 172.21.15.60:36372) 2024-05-03T14:49:32.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.241+0000 7f7e69400700 1 -- 172.21.15.60:0/2825732116 learned_addr learned my addr 172.21.15.60:0/2825732116 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:49:32.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.241+0000 7f7e69e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d6091f8310 0x55d609211440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:32.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.242+0000 7f7e69400700 1 -- 172.21.15.60:0/2825732116 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6091f79f0 msgr2=0x55d6092169d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:49:32.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.242+0000 7f7e69400700 1 --2- 172.21.15.60:0/2825732116 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6091f79f0 0x55d6092169d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:32.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.242+0000 7f7e69400700 1 -- 172.21.15.60:0/2825732116 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d6091f8310 msgr2=0x55d609211440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T14:49:32.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.242+0000 7f7e69400700 1 --2- 172.21.15.60:0/2825732116 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d6091f8310 0x55d609211440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:32.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.242+0000 7f7e69400700 1 -- 172.21.15.60:0/2825732116 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d609211cb0 con 0x55d6091f2580 2024-05-03T14:49:32.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.242+0000 7f7e68a00700 1 --2- 172.21.15.60:0/2825732116 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6091f79f0 0x55d6092169d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:49:32.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.242+0000 7f7e69400700 1 --2- 172.21.15.60:0/2825732116 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d6091f2580 0x55d609216490 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7e58004990 tx=0x7f7e58004830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46cbfa7c58252b0f server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:49:32.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.242+0000 7f7e4b400700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-03T14:49:32.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.242+0000 7f7e4b400700 1 -- 172.21.15.60:0/2825732116 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e5802f060 con 0x55d6091f2580 2024-05-03T14:49:32.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.242+0000 7f7e766ccb80 1 -- 172.21.15.60:0/2825732116 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e500aad20 con 0x55d6091f2580 2024-05-03T14:49:32.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.242+0000 7f7e4b400700 1 -- 172.21.15.60:0/2825732116 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f7e58029340 con 0x55d6091f2580 2024-05-03T14:49:32.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.243+0000 7f7e4b400700 1 -- 172.21.15.60:0/2825732116 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7e58036050 con 0x55d6091f2580 2024-05-03T14:49:32.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.243+0000 7f7e766ccb80 10 client.25178 fetch_fsmap learned FSMap version 29 2024-05-03T14:49:32.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.243+0000 7f7e766ccb80 1 -- 172.21.15.60:0/2825732116 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d609277af0 con 0x55d6091f2580 2024-05-03T14:49:32.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.243+0000 7f7e4b400700 1 -- 172.21.15.60:0/2825732116 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f7e500aad20 con 0x55d6091f2580 2024-05-03T14:49:32.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.243+0000 7f7e4b400700 1 -- 172.21.15.60:0/2825732116 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7e58007840 con 0x55d6091f2580 2024-05-03T14:49:32.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.243+0000 7f7e766ccb80 10 client.25178 fetch_fsmap finished waiting for FSMap version 29 2024-05-03T14:49:32.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.243+0000 7f7e766ccb80 10 client.25178 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-03T14:49:32.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.243+0000 7f7e766ccb80 20 client.25178 populate_metadata read hostname 'smithi060' 2024-05-03T14:49:32.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.244+0000 7f7e766ccb80 1 --2- 172.21.15.60:0/2825732116 >> [v2:172.21.15.60:6840/924163182,v1:172.21.15.60:6841/924163182] conn(0x55d60927c2c0 0x55d60927afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:32.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.244+0000 7f7e766ccb80 4 client.25178 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-03T14:49:32.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.244+0000 7f7e766ccb80 1 -- 172.21.15.60:0/2825732116 --> [v2:172.21.15.60:6840/924163182,v1:172.21.15.60:6841/924163182] -- command(tid 0: {"prefix": "status"}) -- 0x55d6091523b0 con 0x55d60927c2c0 2024-05-03T14:49:32.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.244+0000 7f7e68a00700 1 --2- 172.21.15.60:0/2825732116 >> [v2:172.21.15.60:6840/924163182,v1:172.21.15.60:6841/924163182] conn(0x55d60927c2c0 0x55d60927afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:32.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.244+0000 7f7e68a00700 1 --2- 172.21.15.60:0/2825732116 >> [v2:172.21.15.60:6840/924163182,v1:172.21.15.60:6841/924163182] conn(0x55d60927c2c0 0x55d60927afd0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5946ccca0943f290 server_cookie=d6df7bb978ed1d04 in_seq=0 out_seq=0 2024-05-03T14:49:32.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.244+0000 7f7e4b400700 10 client.25178 ms_handle_connect on v2:172.21.15.60:6840/924163182 2024-05-03T14:49:32.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.245+0000 7f7e4b400700 1 -- 172.21.15.60:0/2825732116 <== mds.0 v2:172.21.15.60:6840/924163182 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55d6091523b0 con 0x55d60927c2c0 2024-05-03T14:49:32.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.245+0000 7f7e4b400700 10 client.25178 handle_command_reply: tid=0 2024-05-03T14:49:32.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.245+0000 7f7e766ccb80 1 client.25178 shutdown 2024-05-03T14:49:32.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.245+0000 7f7e766ccb80 1 -- 172.21.15.60:0/2825732116 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d6091f2580 msgr2=0x55d609216490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:32.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.245+0000 7f7e766ccb80 1 --2- 172.21.15.60:0/2825732116 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d6091f2580 0x55d609216490 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7e58004990 tx=0x7f7e58004830 comp rx=0 tx=0).stop 2024-05-03T14:49:32.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.245+0000 7f7e766ccb80 1 -- 172.21.15.60:0/2825732116 shutdown_connections 2024-05-03T14:49:32.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.245+0000 7f7e766ccb80 1 --2- 172.21.15.60:0/2825732116 >> [v2:172.21.15.60:6840/924163182,v1:172.21.15.60:6841/924163182] conn(0x55d60927c2c0 0x55d60927afd0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:32.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.245+0000 7f7e766ccb80 1 --2- 172.21.15.60:0/2825732116 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d6091f8310 0x55d609211440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:32.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.245+0000 7f7e766ccb80 1 --2- 172.21.15.60:0/2825732116 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d6091f2580 0x55d609216490 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:32.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.245+0000 7f7e766ccb80 1 --2- 172.21.15.60:0/2825732116 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6091f79f0 0x55d6092169d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:32.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.245+0000 7f7e4b400700 0 client.25178 ms_handle_reset on v2:172.21.15.60:6840/924163182 2024-05-03T14:49:32.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.245+0000 7f7e766ccb80 1 -- 172.21.15.60:0/2825732116 >> 172.21.15.60:0/2825732116 conn(0x55d609120b10 msgr2=0x55d6091fb320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:49:32.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.245+0000 7f7e766ccb80 1 -- 172.21.15.60:0/2825732116 shutdown_connections 2024-05-03T14:49:32.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.245+0000 7f7e766ccb80 1 -- 172.21.15.60:0/2825732116 wait complete. 2024-05-03T14:49:32.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.245+0000 7f7e766ccb80 20 client.25178 trim_cache size 0 max 16384 2024-05-03T14:49:32.250 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-05-03T14:49:32.251 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "285d64b2-095a-11ef-bc97-c7b262605968", 2024-05-03T14:49:32.251 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2024-05-03T14:49:32.251 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 24511, 2024-05-03T14:49:32.251 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-05-03T14:49:32.251 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-05-03T14:49:32.251 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-05-03T14:49:32.251 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 297.78176873400002, 2024-05-03T14:49:32.251 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 29, 2024-05-03T14:49:32.251 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 89, 2024-05-03T14:49:32.251 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 87, 2024-05-03T14:49:32.251 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 320.73053766100003 2024-05-03T14:49:32.251 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-05-03T14:49:32.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.246+0000 7f7e6a800700 1 -- 172.21.15.60:0/2605151309 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f7e4c0b2910 msgr2=0x7f7e4c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:32.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.246+0000 7f7e6a800700 1 --2- 172.21.15.60:0/2605151309 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f7e4c0b2910 0x7f7e4c0b4dd0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c005950 tx=0x7f7e5c004750 comp rx=0 tx=0).stop 2024-05-03T14:49:32.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.246+0000 7f7e6a800700 1 -- 172.21.15.60:0/2605151309 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e640761f0 msgr2=0x7f7e6419c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:32.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.246+0000 7f7e6a800700 1 --2- 172.21.15.60:0/2605151309 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e640761f0 0x7f7e6419c350 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7e50006010 tx=0x7f7e500057c0 comp rx=0 tx=0).stop 2024-05-03T14:49:32.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.246+0000 7f7e6a800700 1 -- 172.21.15.60:0/2605151309 shutdown_connections 2024-05-03T14:49:32.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.246+0000 7f7e6a800700 1 --2- 172.21.15.60:0/2605151309 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f7e4c0b2910 0x7f7e4c0b4dd0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:32.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.246+0000 7f7e6a800700 1 --2- 172.21.15.60:0/2605151309 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e6410fc10 0x7f7e641a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:32.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.246+0000 7f7e6a800700 1 --2- 172.21.15.60:0/2605151309 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e640758d0 0x7f7e6419be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:32.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.246+0000 7f7e6a800700 1 --2- 172.21.15.60:0/2605151309 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e640761f0 0x7f7e6419c350 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:32.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.246+0000 7f7e6a800700 1 -- 172.21.15.60:0/2605151309 >> 172.21.15.60:0/2605151309 conn(0x7f7e640fef20 msgr2=0x7f7e64111e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:49:32.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.246+0000 7f7e6a800700 1 -- 172.21.15.60:0/2605151309 shutdown_connections 2024-05-03T14:49:32.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:32.247+0000 7f7e6a800700 1 -- 172.21.15.60:0/2605151309 wait complete. 2024-05-03T14:49:33.308 INFO:teuthology.run_tasks:Running task sequential... 2024-05-03T14:49:33.353 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-03T14:49:33.353 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-03T14:49:33.353 INFO:teuthology.task.print:Enabling mgr modules 2024-05-03T14:49:33.353 INFO:teuthology.run_tasks:Running task kclient... 2024-05-03T14:49:33.366 INFO:tasks.kclient:Mounting kernel clients... 2024-05-03T14:49:33.366 INFO:tasks.kclient:config is {'client.0': {}} 2024-05-03T14:49:33.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T14:49:33.366 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-05-03T14:49:33.428 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-05-03T14:49:33.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T14:49:33.431 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-03T14:49:33.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T14:49:33.528 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link delete ceph-brx 2024-05-03T14:49:33.650 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-05-03T14:49:33.650 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-03T14:49:33.650 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-03T14:49:33.651 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph fs ls 2024-05-03T14:49:33.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:49:33 smithi060 ceph-mon[28873]: pgmap v483: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T14:49:33.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:49:33 smithi090 ceph-mon[33568]: pgmap v483: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T14:49:33.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:49:33 smithi149 ceph-mon[33332]: pgmap v483: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T14:49:34.089 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T14:49:35.719 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:49:35 smithi060 ceph-mon[28873]: pgmap v484: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:49:35.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:49:35 smithi090 ceph-mon[33568]: pgmap v484: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:49:35.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:49:35 smithi149 ceph-mon[33332]: pgmap v484: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:49:36.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:36.265+0000 7f5e95200700 1 -- 172.21.15.60:0/3825751816 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e900756d0 msgr2=0x7f5e90075ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:36.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:36.265+0000 7f5e95200700 1 --2- 172.21.15.60:0/3825751816 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e900756d0 0x7f5e90075ab0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5e80009990 tx=0x7f5e80009ca0 comp rx=0 tx=0).stop 2024-05-03T14:49:36.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:36.265+0000 7f5e95200700 1 -- 172.21.15.60:0/3825751816 shutdown_connections 2024-05-03T14:49:36.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:36.265+0000 7f5e95200700 1 --2- 172.21.15.60:0/3825751816 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5e90114670 0x7f5e90116a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:36.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:36.265+0000 7f5e95200700 1 --2- 172.21.15.60:0/3825751816 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e900756d0 0x7f5e90075ab0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:36.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:36.265+0000 7f5e95200700 1 --2- 172.21.15.60:0/3825751816 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e90075ff0 0x7f5e90114130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:36.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:36.265+0000 7f5e95200700 1 -- 172.21.15.60:0/3825751816 >> 172.21.15.60:0/3825751816 conn(0x7f5e9007a600 msgr2=0x7f5e9007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:49:36.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:36.265+0000 7f5e95200700 1 -- 172.21.15.60:0/3825751816 shutdown_connections 2024-05-03T14:49:36.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:36.265+0000 7f5e95200700 1 -- 172.21.15.60:0/3825751816 wait complete. 2024-05-03T14:49:36.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:36.267+0000 7f5e95200700 1 Processor -- start 2024-05-03T14:49:36.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:36.267+0000 7f5e95200700 1 -- start start 2024-05-03T14:49:36.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:36.267+0000 7f5e95200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e900756d0 0x7f5e901a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:36.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:36.268+0000 7f5e95200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e90075ff0 0x7f5e901a0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:36.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:36.268+0000 7f5e95200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5e90114670 0x7f5e901a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:36.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:36.268+0000 7f5e95200700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5e90119f30 con 0x7f5e90075ff0 2024-05-03T14:49:36.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:36.268+0000 7f5e95200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5e90119db0 con 0x7f5e90114670 2024-05-03T14:49:36.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:36.268+0000 7f5e95200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5e9011a0b0 con 0x7f5e900756d0 2024-05-03T14:49:36.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:36.268+0000 7f5e8de00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e900756d0 0x7f5e901a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:36.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:36.268+0000 7f5e8d400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e90075ff0 0x7f5e901a0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:36.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:36.268+0000 7f5e8de00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e900756d0 0x7f5e901a03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.60:48078/0 (socket says 172.21.15.60:48078) 2024-05-03T14:49:36.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:36.268+0000 7f5e8de00700 1 -- 172.21.15.60:0/2791938147 learned_addr learned my addr 172.21.15.60:0/2791938147 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:49:36.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:36.268+0000 7f5e8e800700 1 --2- 172.21.15.60:0/2791938147 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5e90114670 0x7f5e901a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:36.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:36.268+0000 7f5e8de00700 1 -- 172.21.15.60:0/2791938147 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5e90114670 msgr2=0x7f5e901a4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:36.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:36.268+0000 7f5e8de00700 1 --2- 172.21.15.60:0/2791938147 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5e90114670 0x7f5e901a4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:36.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:36.268+0000 7f5e8de00700 1 -- 172.21.15.60:0/2791938147 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e90075ff0 msgr2=0x7f5e901a0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:36.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:36.268+0000 7f5e8de00700 1 --2- 172.21.15.60:0/2791938147 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e90075ff0 0x7f5e901a0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:36.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:36.268+0000 7f5e8de00700 1 -- 172.21.15.60:0/2791938147 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e901a53f0 con 0x7f5e900756d0 2024-05-03T14:49:36.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:36.269+0000 7f5e8de00700 1 --2- 172.21.15.60:0/2791938147 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e900756d0 0x7f5e901a03e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5e80000c00 tx=0x7f5e80005590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:49:36.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:36.269+0000 7f5e86a00700 1 -- 172.21.15.60:0/2791938147 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e80027070 con 0x7f5e900756d0 2024-05-03T14:49:36.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:36.269+0000 7f5e86a00700 1 -- 172.21.15.60:0/2791938147 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f5e8000fdc0 con 0x7f5e900756d0 2024-05-03T14:49:36.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:36.269+0000 7f5e86a00700 1 -- 172.21.15.60:0/2791938147 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e800173f0 con 0x7f5e900756d0 2024-05-03T14:49:36.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:36.269+0000 7f5e95200700 1 -- 172.21.15.60:0/2791938147 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e901a5680 con 0x7f5e900756d0 2024-05-03T14:49:36.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:36.269+0000 7f5e95200700 1 -- 172.21.15.60:0/2791938147 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e901acf60 con 0x7f5e900756d0 2024-05-03T14:49:36.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:36.270+0000 7f5e95200700 1 -- 172.21.15.60:0/2791938147 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e90076dc0 con 0x7f5e900756d0 2024-05-03T14:49:36.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:36.273+0000 7f5e86a00700 1 -- 172.21.15.60:0/2791938147 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f5e8002a070 con 0x7f5e900756d0 2024-05-03T14:49:36.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:36.274+0000 7f5e86a00700 1 --2- 172.21.15.60:0/2791938147 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f5e740b2910 0x7f5e740b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:36.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:36.275+0000 7f5e86a00700 1 -- 172.21.15.60:0/2791938147 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f5e800e34d0 con 0x7f5e900756d0 2024-05-03T14:49:36.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:36.275+0000 7f5e8d400700 1 --2- 172.21.15.60:0/2791938147 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f5e740b2910 0x7f5e740b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:36.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:36.276+0000 7f5e8d400700 1 --2- 172.21.15.60:0/2791938147 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f5e740b2910 0x7f5e740b4dd0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5e90110f00 tx=0x7f5e7c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:49:36.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:36.276+0000 7f5e86a00700 1 -- 172.21.15.60:0/2791938147 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f5e80013070 con 0x7f5e900756d0 2024-05-03T14:49:36.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:36.564+0000 7f5e95200700 1 -- 172.21.15.60:0/2791938147 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5e901ad130 con 0x7f5e900756d0 2024-05-03T14:49:36.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:36.564+0000 7f5e86a00700 1 -- 172.21.15.60:0/2791938147 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v29) ==== 53+0+88 (secure 0 0 0) 0x7f5e800aad40 con 0x7f5e900756d0 2024-05-03T14:49:36.567 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-03T14:49:36.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:36.569+0000 7f5e95200700 1 -- 172.21.15.60:0/2791938147 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f5e740b2910 msgr2=0x7f5e740b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:36.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:36.569+0000 7f5e95200700 1 --2- 172.21.15.60:0/2791938147 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f5e740b2910 0x7f5e740b4dd0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5e90110f00 tx=0x7f5e7c009260 comp rx=0 tx=0).stop 2024-05-03T14:49:36.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:36.569+0000 7f5e95200700 1 -- 172.21.15.60:0/2791938147 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e900756d0 msgr2=0x7f5e901a03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:36.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:36.569+0000 7f5e95200700 1 --2- 172.21.15.60:0/2791938147 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e900756d0 0x7f5e901a03e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5e80000c00 tx=0x7f5e80005590 comp rx=0 tx=0).stop 2024-05-03T14:49:36.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:36.569+0000 7f5e95200700 1 -- 172.21.15.60:0/2791938147 shutdown_connections 2024-05-03T14:49:36.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:36.569+0000 7f5e95200700 1 --2- 172.21.15.60:0/2791938147 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f5e740b2910 0x7f5e740b4dd0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:36.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:36.569+0000 7f5e95200700 1 --2- 172.21.15.60:0/2791938147 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5e90114670 0x7f5e901a4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:36.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:36.569+0000 7f5e95200700 1 --2- 172.21.15.60:0/2791938147 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e900756d0 0x7f5e901a03e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:36.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:36.569+0000 7f5e95200700 1 --2- 172.21.15.60:0/2791938147 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e90075ff0 0x7f5e901a0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:36.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:36.569+0000 7f5e95200700 1 -- 172.21.15.60:0/2791938147 >> 172.21.15.60:0/2791938147 conn(0x7f5e9007a600 msgr2=0x7f5e90113430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:49:36.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:36.569+0000 7f5e95200700 1 -- 172.21.15.60:0/2791938147 shutdown_connections 2024-05-03T14:49:36.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:36.570+0000 7f5e95200700 1 -- 172.21.15.60:0/2791938147 wait complete. 2024-05-03T14:49:36.836 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:49:36 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/2791938147' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-03T14:49:36.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:49:36 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/2791938147' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-03T14:49:36.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:49:36 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/2791938147' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-03T14:49:37.469 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-03T14:49:37.469 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-03T14:49:37.469 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2024-05-03T14:49:37.469 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-03T14:49:37.469 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-03T14:49:37.469 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-03T14:49:37.469 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-03T14:49:37.470 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-03T14:49:37.470 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-03T14:49:37.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T14:49:37.470 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2024-05-03T14:49:37.523 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-03T14:49:37.523 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-03T14:49:37.523 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2024-05-03T14:49:37.523 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-05-03T14:49:37.523 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2024-05-03T14:49:37.523 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-05-03T14:49:37.523 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-03T14:49:37.523 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2024-05-03T14:49:37.524 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2024-05-03T14:49:37.524 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-03T14:49:37.524 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2024-05-03T14:49:37.524 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2024-05-03T14:49:37.524 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-03T14:49:37.524 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2024-05-03T14:49:37.524 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2024-05-03T14:49:37.524 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-05-03T14:49:37.524 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 41750sec preferred_lft 41750sec 2024-05-03T14:49:37.524 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link noprefixroute 2024-05-03T14:49:37.524 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-05-03T14:49:37.524 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-03T14:49:37.524 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2024-05-03T14:49:37.524 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2024-05-03T14:49:37.525 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-03T14:49:37.526 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-03T14:49:37.526 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-05-03T14:49:37.526 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link add name ceph-brx type bridge 2024-05-03T14:49:37.526 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip addr flush dev ceph-brx 2024-05-03T14:49:37.526 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set ceph-brx up 2024-05-03T14:49:37.526 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-03T14:49:37.526 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-05-03T14:49:37.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-03T14:49:37.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-03T14:49:37.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T14:49:37.755 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-03T14:49:37.823 INFO:teuthology.orchestra.run.smithi060.stdout:1 2024-05-03T14:49:37.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T14:49:37.828 DEBUG:teuthology.orchestra.run.smithi060:> route 2024-05-03T14:49:37.874 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2024-05-03T14:49:37.874 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-03T14:49:37.874 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-03T14:49:37.874 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-03T14:49:37.874 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-03T14:49:37.875 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-03T14:49:37.875 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-05-03T14:49:37.875 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-03T14:49:37.875 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-03T14:49:37.875 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-03T14:49:37.875 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-05-03T14:49:37.875 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:49:37 smithi090 ceph-mon[33568]: pgmap v485: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:49:37.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:49:37 smithi149 ceph-mon[33332]: pgmap v485: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:49:38.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-03T14:49:38.046 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:49:37 smithi060 ceph-mon[28873]: pgmap v485: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:49:38.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-03T14:49:38.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T14:49:38.126 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-05-03T14:49:38.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T14:49:38.170 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list-id 2024-05-03T14:49:38.261 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-03T14:49:38.262 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-05-03T14:49:38.262 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-03T14:49:38.262 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-03T14:49:38.262 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-05-03T14:49:38.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-03T14:49:38.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-03T14:49:38.423 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-03T14:49:38.423 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-03T14:49:38.423 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-05-03T14:49:38.423 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-03T14:49:38.423 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-03T14:49:38.423 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-03T14:49:38.423 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-03T14:49:38.424 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-03T14:49:38.424 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-05-03T14:49:38.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-03T14:49:38.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-03T14:49:38.697 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-03T14:49:38.697 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-05-03T14:49:38.697 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2024-05-03T14:49:38.697 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set dev brx.0 master ceph-brx 2024-05-03T14:49:38.697 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-05-03T14:49:38.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-03T14:49:38.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-03T14:49:38.895 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-03T14:49:38.895 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-03T14:49:38.938 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-03T14:49:38.938 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-03T14:49:38.938 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-03T14:49:39.031 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-05-03T14:49:39.031 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-05-03T14:49:39.031 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/65eceeb5-e868-44a6-8a80-e9d8bf9f5757 2024-05-03T14:49:39.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T14:49:39.031 DEBUG:teuthology.orchestra.run.smithi060:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/65eceeb5-e868-44a6-8a80-e9d8bf9f5757 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2024-05-03T14:49:39.173 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2024-05-03T14:49:39.174 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync". 2024-05-03T14:49:39.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:49:39 smithi060 ceph-mon[28873]: pgmap v486: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:49:39.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:49:39 smithi090 ceph-mon[33568]: pgmap v486: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:49:39.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:49:39 smithi149 ceph-mon[33332]: pgmap v486: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:49:39.935 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2024-05-03T14:49:39.935 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync". 2024-05-03T14:49:39.936 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: resolved to: "172.21.15.60:6789,172.21.15.90:6789,172.21.15.149:6789" 2024-05-03T14:49:39.936 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: trying mount with new device syntax: 0@285d64b2-095a-11ef-bc97-c7b262605968.cephfs=/volumes/qa/sv_1/65eceeb5-e868-44a6-8a80-e9d8bf9f5757 2024-05-03T14:49:39.936 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync,name=0,key=0,mon_addr=172.21.15.60:6789/172.21.15.90:6789/172.21.15.149:6789" will pass to kernel 2024-05-03T14:49:39.941 INFO:teuthology.orchestra.run.smithi060.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-03T14:49:39.941 INFO:teuthology.orchestra.run.smithi060.stdout: You just mounted a file system that supports labels which does not 2024-05-03T14:49:39.941 INFO:teuthology.orchestra.run.smithi060.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-03T14:49:39.941 INFO:teuthology.orchestra.run.smithi060.stdout: applications will generate AVC messages and not be allowed access to 2024-05-03T14:49:39.941 INFO:teuthology.orchestra.run.smithi060.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-03T14:49:39.945 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-03T14:49:39.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T14:49:39.946 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-03T14:49:40.031 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-03T14:49:40.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:40 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-03T14:49:40.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-03T14:49:40.157 INFO:teuthology.orchestra.run.smithi060.stdout:client25195 2024-05-03T14:49:40.157 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-05-03T14:49:40.157 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-03T14:49:40.210 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-03T14:49:40.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:40 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-03T14:49:40.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-03T14:49:40.334 INFO:teuthology.orchestra.run.smithi060.stdout:client25195 2024-05-03T14:49:40.335 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-03T14:49:40.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:40 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-03T14:49:40.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-03T14:49:40.464 INFO:teuthology.orchestra.run.smithi060.stdout:client25195 2024-05-03T14:49:40.464 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/285d64b2-095a-11ef-bc97-c7b262605968.client25195/status') 2024-05-03T14:49:40.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-03T14:49:40.609 INFO:teuthology.orchestra.run.smithi060.stdout:instance: client.25195 (3)172.21.15.60:0/3766869073 2024-05-03T14:49:40.609 INFO:teuthology.orchestra.run.smithi060.stdout:blocklisted: false 2024-05-03T14:49:40.610 INFO:teuthology.orchestra.run.smithi060.stderr:0+1 records in 2024-05-03T14:49:40.610 INFO:teuthology.orchestra.run.smithi060.stderr:0+1 records out 2024-05-03T14:49:40.610 INFO:teuthology.orchestra.run.smithi060.stderr:71 bytes copied, 8.7528e-05 s, 811 kB/s 2024-05-03T14:49:40.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-03T14:49:40.623 INFO:teuthology.run_tasks:Running task exec... 2024-05-03T14:49:40.632 INFO:teuthology.task.exec:Executing custom commands... 2024-05-03T14:49:40.632 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi060.front.sepia.ceph.com 2024-05-03T14:49:40.633 DEBUG:teuthology.orchestra.run.smithi060:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-05-03T14:49:40.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:40.838+0000 7f8501a00640 1 -- 172.21.15.60:0/3540503037 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f84fc1045c0 msgr2=0x7f84fc10ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:40.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:40.838+0000 7f8501a00640 1 --2- 172.21.15.60:0/3540503037 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f84fc1045c0 0x7f84fc10ae50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f84f4009ca0 tx=0x7f84f402a8c0 comp rx=0 tx=0).stop 2024-05-03T14:49:40.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:40.839+0000 7f8501a00640 1 -- 172.21.15.60:0/3540503037 shutdown_connections 2024-05-03T14:49:40.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:40.839+0000 7f8501a00640 1 --2- 172.21.15.60:0/3540503037 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f84fc1045c0 0x7f84fc10ae50 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:40.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:40.839+0000 7f8501a00640 1 --2- 172.21.15.60:0/3540503037 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84fc103c00 0x7f84fc104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:40.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:40.839+0000 7f8501a00640 1 --2- 172.21.15.60:0/3540503037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84fc102a10 0x7f84fc102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:40.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:40.839+0000 7f8501a00640 1 -- 172.21.15.60:0/3540503037 >> 172.21.15.60:0/3540503037 conn(0x7f84fc0fdfa0 msgr2=0x7f84fc1003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:49:40.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:40.839+0000 7f8501a00640 1 -- 172.21.15.60:0/3540503037 shutdown_connections 2024-05-03T14:49:40.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:40.839+0000 7f8501a00640 1 -- 172.21.15.60:0/3540503037 wait complete. 2024-05-03T14:49:40.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:40.840+0000 7f8501a00640 1 Processor -- start 2024-05-03T14:49:40.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:40.840+0000 7f8501a00640 1 -- start start 2024-05-03T14:49:40.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:40.840+0000 7f8501a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84fc102a10 0x7f84fc198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:40.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:40.840+0000 7f8501a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84fc103c00 0x7f84fc1991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:40.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:40.840+0000 7f8501a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f84fc1045c0 0x7f84fc1a0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:40.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:40.840+0000 7f8501a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f84fc10e820 con 0x7f84fc102a10 2024-05-03T14:49:40.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:40.840+0000 7f8501a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f84fc10e6a0 con 0x7f84fc1045c0 2024-05-03T14:49:40.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:40.840+0000 7f8501a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f84fc10e9a0 con 0x7f84fc103c00 2024-05-03T14:49:40.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:40.840+0000 7f84faa00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84fc102a10 0x7f84fc198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:40.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:40.840+0000 7f84faa00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84fc102a10 0x7f84fc198cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34312/0 (socket says 172.21.15.60:34312) 2024-05-03T14:49:40.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:40.840+0000 7f84faa00640 1 -- 172.21.15.60:0/3684311849 learned_addr learned my addr 172.21.15.60:0/3684311849 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:49:40.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:40.841+0000 7f84fb400640 1 --2- 172.21.15.60:0/3684311849 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f84fc1045c0 0x7f84fc1a0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:40.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:40.841+0000 7f84fa000640 1 --2- 172.21.15.60:0/3684311849 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84fc103c00 0x7f84fc1991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:40.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:40.841+0000 7f84faa00640 1 -- 172.21.15.60:0/3684311849 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84fc103c00 msgr2=0x7f84fc1991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:40.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:40.841+0000 7f84faa00640 1 --2- 172.21.15.60:0/3684311849 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84fc103c00 0x7f84fc1991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:40.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:40.841+0000 7f84faa00640 1 -- 172.21.15.60:0/3684311849 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f84fc1045c0 msgr2=0x7f84fc1a0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:40.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:40.841+0000 7f84faa00640 1 --2- 172.21.15.60:0/3684311849 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f84fc1045c0 0x7f84fc1a0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:40.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:40.841+0000 7f84faa00640 1 -- 172.21.15.60:0/3684311849 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84fc1a08f0 con 0x7f84fc102a10 2024-05-03T14:49:40.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:40.841+0000 7f84fb400640 1 --2- 172.21.15.60:0/3684311849 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f84fc1045c0 0x7f84fc1a0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:49:40.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:40.841+0000 7f84faa00640 1 --2- 172.21.15.60:0/3684311849 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84fc102a10 0x7f84fc198cb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f84ec00e9c0 tx=0x7f84ec00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:49:40.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:40.842+0000 7f84f3400640 1 -- 172.21.15.60:0/3684311849 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84ec00cd80 con 0x7f84fc102a10 2024-05-03T14:49:40.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:40.842+0000 7f8501a00640 1 -- 172.21.15.60:0/3684311849 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84fc1a0be0 con 0x7f84fc102a10 2024-05-03T14:49:40.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:40.842+0000 7f84f3400640 1 -- 172.21.15.60:0/3684311849 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f84ec0035f0 con 0x7f84fc102a10 2024-05-03T14:49:40.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:40.842+0000 7f84f3400640 1 -- 172.21.15.60:0/3684311849 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84ec018630 con 0x7f84fc102a10 2024-05-03T14:49:40.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:40.842+0000 7f8501a00640 1 -- 172.21.15.60:0/3684311849 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84fc1a1180 con 0x7f84fc102a10 2024-05-03T14:49:40.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:40.843+0000 7f8501a00640 1 -- 172.21.15.60:0/3684311849 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84fc109c60 con 0x7f84fc102a10 2024-05-03T14:49:40.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:40.846+0000 7f84f3400640 1 -- 172.21.15.60:0/3684311849 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f84ec0037a0 con 0x7f84fc102a10 2024-05-03T14:49:40.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:40.847+0000 7f84f3400640 1 --2- 172.21.15.60:0/3684311849 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f84e00b2920 0x7f84e00b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:40.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:40.847+0000 7f84f3400640 1 -- 172.21.15.60:0/3684311849 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f84ec0e2cd0 con 0x7f84fc102a10 2024-05-03T14:49:40.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:40.848+0000 7f84fa000640 1 --2- 172.21.15.60:0/3684311849 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f84e00b2920 0x7f84e00b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:40.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:40.848+0000 7f84fa000640 1 --2- 172.21.15.60:0/3684311849 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f84e00b2920 0x7f84e00b4de0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f84fc19a1b0 tx=0x7f84e8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:49:40.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:40.850+0000 7f84f3400640 1 -- 172.21.15.60:0/3684311849 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f84ec054d70 con 0x7f84fc102a10 2024-05-03T14:49:41.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:41.104+0000 7f8501a00640 1 -- 172.21.15.60:0/3684311849 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f84fc02dd70 con 0x7f84fc102a10 2024-05-03T14:49:41.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:41.597+0000 7f84f3400640 1 -- 172.21.15.60:0/3684311849 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v30) ==== 118+0+0 (secure 0 0 0) 0x7f84ec0aa4b0 con 0x7f84fc102a10 2024-05-03T14:49:41.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:41.602+0000 7f8501a00640 1 -- 172.21.15.60:0/3684311849 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f84e00b2920 msgr2=0x7f84e00b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:41.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:41.602+0000 7f8501a00640 1 --2- 172.21.15.60:0/3684311849 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f84e00b2920 0x7f84e00b4de0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f84fc19a1b0 tx=0x7f84e8005f90 comp rx=0 tx=0).stop 2024-05-03T14:49:41.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:41.602+0000 7f8501a00640 1 -- 172.21.15.60:0/3684311849 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84fc102a10 msgr2=0x7f84fc198cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:41.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:41.602+0000 7f8501a00640 1 --2- 172.21.15.60:0/3684311849 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84fc102a10 0x7f84fc198cb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f84ec00e9c0 tx=0x7f84ec00ee80 comp rx=0 tx=0).stop 2024-05-03T14:49:41.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:41.602+0000 7f8501a00640 1 -- 172.21.15.60:0/3684311849 shutdown_connections 2024-05-03T14:49:41.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:41.602+0000 7f8501a00640 1 --2- 172.21.15.60:0/3684311849 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f84e00b2920 0x7f84e00b4de0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:41.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:41.602+0000 7f8501a00640 1 --2- 172.21.15.60:0/3684311849 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f84fc1045c0 0x7f84fc1a0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:41.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:41.602+0000 7f8501a00640 1 --2- 172.21.15.60:0/3684311849 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84fc103c00 0x7f84fc1991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:41.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:41.602+0000 7f8501a00640 1 --2- 172.21.15.60:0/3684311849 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84fc102a10 0x7f84fc198cb0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:41.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:41.602+0000 7f8501a00640 1 -- 172.21.15.60:0/3684311849 >> 172.21.15.60:0/3684311849 conn(0x7f84fc0fdfa0 msgr2=0x7f84fc106e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:49:41.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:41.602+0000 7f8501a00640 1 -- 172.21.15.60:0/3684311849 shutdown_connections 2024-05-03T14:49:41.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:41.602+0000 7f8501a00640 1 -- 172.21.15.60:0/3684311849 wait complete. 2024-05-03T14:49:41.627 DEBUG:teuthology.orchestra.run.smithi060:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-05-03T14:49:41.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:41.821+0000 7efe07200640 1 -- 172.21.15.60:0/2826034283 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe0810a970 msgr2=0x7efe08111290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:41.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:41.821+0000 7efe07200640 1 --2- 172.21.15.60:0/2826034283 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe0810a970 0x7efe08111290 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7efe00009ca0 tx=0x7efe0002a8c0 comp rx=0 tx=0).stop 2024-05-03T14:49:41.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:41.822+0000 7efe07200640 1 -- 172.21.15.60:0/2826034283 shutdown_connections 2024-05-03T14:49:41.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:41.822+0000 7efe07200640 1 --2- 172.21.15.60:0/2826034283 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe0810a970 0x7efe08111290 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:41.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:41.822+0000 7efe07200640 1 --2- 172.21.15.60:0/2826034283 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe08109fb0 0x7efe0810a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:41.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:41.822+0000 7efe07200640 1 --2- 172.21.15.60:0/2826034283 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe08108e60 0x7efe08109280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:41.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:41.822+0000 7efe07200640 1 -- 172.21.15.60:0/2826034283 >> 172.21.15.60:0/2826034283 conn(0x7efe08077e40 msgr2=0x7efe0807a260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T14:49:41.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:41.822+0000 7efe07200640 1 -- 172.21.15.60:0/2826034283 shutdown_connections 2024-05-03T14:49:41.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:41.822+0000 7efe07200640 1 -- 172.21.15.60:0/2826034283 wait complete. 2024-05-03T14:49:41.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:41.823+0000 7efe07200640 1 Processor -- start 2024-05-03T14:49:41.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:41.823+0000 7efe07200640 1 -- start start 2024-05-03T14:49:41.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:41.823+0000 7efe07200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe08108e60 0x7efe0819f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:41.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:41.823+0000 7efe07200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe08109fb0 0x7efe0819f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:41.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:41.823+0000 7efe07200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe0810a970 0x7efe081a6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:41.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:41.823+0000 7efe07200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7efe08114c40 con 0x7efe08108e60 2024-05-03T14:49:41.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:41.823+0000 7efe07200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7efe08114ac0 con 0x7efe0810a970 2024-05-03T14:49:41.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:41.823+0000 7efe07200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7efe08114dc0 con 0x7efe08109fb0 2024-05-03T14:49:41.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:41.824+0000 7efe05e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe08108e60 0x7efe0819f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:41.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:41.824+0000 7efe06800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe0810a970 0x7efe081a6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:41.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:41.824+0000 7efe05e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe08108e60 0x7efe0819f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34334/0 (socket says 172.21.15.60:34334) 2024-05-03T14:49:41.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:41.824+0000 7efe06800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe0810a970 0x7efe081a6750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.60:36436/0 (socket says 172.21.15.60:36436) 2024-05-03T14:49:41.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:41.824+0000 7efe05e00640 1 -- 172.21.15.60:0/3449599367 learned_addr learned my addr 172.21.15.60:0/3449599367 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T14:49:41.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:41.824+0000 7efe05400640 1 --2- 172.21.15.60:0/3449599367 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe08109fb0 0x7efe0819f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:41.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:41.824+0000 7efe06800640 1 -- 172.21.15.60:0/3449599367 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe08109fb0 msgr2=0x7efe0819f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:41.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:41.824+0000 7efe06800640 1 --2- 172.21.15.60:0/3449599367 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe08109fb0 0x7efe0819f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:41.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:41.824+0000 7efe06800640 1 -- 172.21.15.60:0/3449599367 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe08108e60 msgr2=0x7efe0819f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T14:49:41.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:41.824+0000 7efe06800640 1 --2- 172.21.15.60:0/3449599367 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe08108e60 0x7efe0819f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T14:49:41.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:41.824+0000 7efe06800640 1 -- 172.21.15.60:0/3449599367 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe081a6dc0 con 0x7efe0810a970 2024-05-03T14:49:41.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:41.824+0000 7efe05e00640 1 --2- 172.21.15.60:0/3449599367 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe08108e60 0x7efe0819f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T14:49:41.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:41.825+0000 7efe06800640 1 --2- 172.21.15.60:0/3449599367 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe0810a970 0x7efe081a6750 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7efe0002ade0 tx=0x7efe0002f850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:49:41.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:41.825+0000 7efdfea00640 1 -- 172.21.15.60:0/3449599367 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0002fc40 con 0x7efe0810a970 2024-05-03T14:49:41.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:41.825+0000 7efe07200640 1 -- 172.21.15.60:0/3449599367 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe081a7050 con 0x7efe0810a970 2024-05-03T14:49:41.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:41.825+0000 7efdfea00640 1 -- 172.21.15.60:0/3449599367 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7efe0002f310 con 0x7efe0810a970 2024-05-03T14:49:41.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:41.825+0000 7efdfea00640 1 -- 172.21.15.60:0/3449599367 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe000045b0 con 0x7efe0810a970 2024-05-03T14:49:41.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:41.825+0000 7efe07200640 1 -- 172.21.15.60:0/3449599367 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe081a75f0 con 0x7efe0810a970 2024-05-03T14:49:41.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:41.826+0000 7efe07200640 1 -- 172.21.15.60:0/3449599367 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe08110110 con 0x7efe0810a970 2024-05-03T14:49:41.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:41.828+0000 7efdfea00640 1 -- 172.21.15.60:0/3449599367 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7efe00003970 con 0x7efe0810a970 2024-05-03T14:49:41.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:41.830+0000 7efdfea00640 1 --2- 172.21.15.60:0/3449599367 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7efdec0b28f0 0x7efdec0b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T14:49:41.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:41.830+0000 7efdfea00640 1 -- 172.21.15.60:0/3449599367 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7efe00103510 con 0x7efe0810a970 2024-05-03T14:49:41.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:41.830+0000 7efe05e00640 1 --2- 172.21.15.60:0/3449599367 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7efdec0b28f0 0x7efdec0b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T14:49:41.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:41.831+0000 7efe05e00640 1 --2- 172.21.15.60:0/3449599367 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7efdec0b28f0 0x7efdec0b4db0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7efdf8011500 tx=0x7efdf8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T14:49:41.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:41.834+0000 7efdfea00640 1 -- 172.21.15.60:0/3449599367 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7efe000cae00 con 0x7efe0810a970 2024-05-03T14:49:41.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:49:41 smithi090 ceph-mon[33568]: pgmap v487: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 170 B/s wr, 12 op/s 2024-05-03T14:49:41.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:49:41 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/3684311849' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-03T14:49:41.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:49:41 smithi149 ceph-mon[33332]: pgmap v487: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 170 B/s wr, 12 op/s 2024-05-03T14:49:41.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:49:41 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/3684311849' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-03T14:49:42.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:49:42.076+0000 7efe07200640 1 -- 172.21.15.60:0/3449599367 --> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7efe08062e90 con 0x7efdec0b28f0 2024-05-03T14:49:42.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:49:41 smithi060 ceph-mon[28873]: pgmap v487: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 170 B/s wr, 12 op/s 2024-05-03T14:49:42.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:49:41 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/3684311849' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-03T14:49:42.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:49:42 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/3684311849' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-03T14:49:42.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:49:42 smithi090 ceph-mon[33568]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-03T14:49:42.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:49:42 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:49:42.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:49:42 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/3684311849' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-03T14:49:42.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:49:42 smithi149 ceph-mon[33332]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-03T14:49:42.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:49:42 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:49:43.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:49:42 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/3684311849' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-03T14:49:43.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:49:42 smithi060 ceph-mon[28873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-03T14:49:43.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:49:42 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:49:43.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:49:43 smithi149 ceph-mon[33332]: from='client.25196 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:49:43.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:49:43 smithi149 ceph-mon[33332]: pgmap v488: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-03T14:49:44.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:49:43 smithi060 ceph-mon[28873]: from='client.25196 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:49:44.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:49:43 smithi060 ceph-mon[28873]: pgmap v488: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-03T14:49:44.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:49:43 smithi090 ceph-mon[33568]: from='client.25196 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T14:49:44.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:49:43 smithi090 ceph-mon[33568]: pgmap v488: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-03T14:49:44.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:49:44 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:49:44.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:49:44 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:49:44.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:49:44 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:49:45.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:49:44 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:49:45.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:49:44 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:49:45.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:49:44 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:49:45.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:49:44 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:49:45.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:49:44 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:49:45.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:49:44 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:49:45.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:49:45 smithi149 ceph-mon[33332]: pgmap v489: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 9 op/s 2024-05-03T14:49:46.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:49:45 smithi060 ceph-mon[28873]: pgmap v489: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 9 op/s 2024-05-03T14:49:46.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:49:45 smithi090 ceph-mon[33568]: pgmap v489: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 9 op/s 2024-05-03T14:49:47.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:49:47 smithi149 ceph-mon[33332]: pgmap v490: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-03T14:49:48.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:49:47 smithi060 ceph-mon[28873]: pgmap v490: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-03T14:49:48.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:49:47 smithi090 ceph-mon[33568]: pgmap v490: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-03T14:49:49.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:49:49 smithi149 ceph-mon[33332]: pgmap v491: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-03T14:49:50.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:49:49 smithi060 ceph-mon[28873]: pgmap v491: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-03T14:49:50.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:49:49 smithi090 ceph-mon[33568]: pgmap v491: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-03T14:49:51.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:49:51 smithi149 ceph-mon[33332]: pgmap v492: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 12 op/s 2024-05-03T14:49:52.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:49:51 smithi060 ceph-mon[28873]: pgmap v492: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 12 op/s 2024-05-03T14:49:52.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:49:51 smithi090 ceph-mon[33568]: pgmap v492: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 12 op/s 2024-05-03T14:49:53.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:49:53 smithi149 ceph-mon[33332]: pgmap v493: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 8 op/s 2024-05-03T14:49:54.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:49:53 smithi060 ceph-mon[28873]: pgmap v493: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 8 op/s 2024-05-03T14:49:54.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:49:53 smithi090 ceph-mon[33568]: pgmap v493: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 8 op/s 2024-05-03T14:49:55.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:49:55 smithi149 ceph-mon[33332]: pgmap v494: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 511 B/s wr, 9 op/s 2024-05-03T14:49:56.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:49:55 smithi060 ceph-mon[28873]: pgmap v494: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 511 B/s wr, 9 op/s 2024-05-03T14:49:56.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:49:55 smithi090 ceph-mon[33568]: pgmap v494: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 511 B/s wr, 9 op/s 2024-05-03T14:49:57.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:49:57 smithi149 ceph-mon[33332]: pgmap v495: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 511 B/s wr, 11 op/s 2024-05-03T14:49:57.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:49:57 smithi090 ceph-mon[33568]: pgmap v495: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 511 B/s wr, 11 op/s 2024-05-03T14:49:58.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:49:57 smithi060 ceph-mon[28873]: pgmap v495: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 511 B/s wr, 11 op/s 2024-05-03T14:49:59.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:49:59 smithi149 ceph-mon[33332]: pgmap v496: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:50:00.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:49:59 smithi060 ceph-mon[28873]: pgmap v496: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:50:00.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:49:59 smithi090 ceph-mon[33568]: pgmap v496: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:50:01.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:50:00 smithi060 ceph-mon[28873]: overall HEALTH_OK 2024-05-03T14:50:01.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:50:00 smithi090 ceph-mon[33568]: overall HEALTH_OK 2024-05-03T14:50:01.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:50:00 smithi149 ceph-mon[33332]: overall HEALTH_OK 2024-05-03T14:50:02.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:50:01 smithi060 ceph-mon[28873]: pgmap v497: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:50:02.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:50:01 smithi090 ceph-mon[33568]: pgmap v497: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:50:02.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:50:01 smithi149 ceph-mon[33332]: pgmap v497: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:50:04.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:50:03 smithi060 ceph-mon[28873]: pgmap v498: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T14:50:04.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:50:03 smithi090 ceph-mon[33568]: pgmap v498: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T14:50:04.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:50:03 smithi149 ceph-mon[33332]: pgmap v498: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T14:50:06.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:50:05 smithi060 ceph-mon[28873]: pgmap v499: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:50:06.126 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:50:05 smithi090 ceph-mon[33568]: pgmap v499: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:50:06.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:50:05 smithi149 ceph-mon[33332]: pgmap v499: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:50:08.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:50:07 smithi060 ceph-mon[28873]: pgmap v500: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:50:08.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:50:07 smithi090 ceph-mon[33568]: pgmap v500: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:50:08.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:50:07 smithi149 ceph-mon[33332]: pgmap v500: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:50:10.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:50:09 smithi060 ceph-mon[28873]: pgmap v501: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:50:10.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:50:09 smithi090 ceph-mon[33568]: pgmap v501: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:50:10.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:50:09 smithi149 ceph-mon[33332]: pgmap v501: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:50:12.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:50:11 smithi060 ceph-mon[28873]: pgmap v502: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:50:12.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:50:11 smithi090 ceph-mon[33568]: pgmap v502: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:50:12.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:50:11 smithi149 ceph-mon[33332]: pgmap v502: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:50:14.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:50:13 smithi060 ceph-mon[28873]: pgmap v503: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T14:50:14.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:50:13 smithi090 ceph-mon[33568]: pgmap v503: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T14:50:14.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:50:13 smithi149 ceph-mon[33332]: pgmap v503: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T14:50:16.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:50:15 smithi060 ceph-mon[28873]: pgmap v504: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:50:16.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:50:15 smithi090 ceph-mon[33568]: pgmap v504: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:50:16.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:50:15 smithi149 ceph-mon[33332]: pgmap v504: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:50:17.975 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:50:17 smithi149 ceph-mon[33332]: pgmap v505: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:50:17.997 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:50:17 smithi060 ceph-mon[28873]: pgmap v505: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:50:18.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:50:17 smithi090 ceph-mon[33568]: pgmap v505: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:50:20.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:50:19 smithi060 ceph-mon[28873]: pgmap v506: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:50:20.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:50:19 smithi090 ceph-mon[33568]: pgmap v506: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:50:20.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:50:19 smithi149 ceph-mon[33332]: pgmap v506: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:50:22.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:50:21 smithi060 ceph-mon[28873]: pgmap v507: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:50:22.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:50:21 smithi090 ceph-mon[33568]: pgmap v507: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:50:22.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:50:21 smithi149 ceph-mon[33332]: pgmap v507: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:50:24.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:50:23 smithi060 ceph-mon[28873]: pgmap v508: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:50:24.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:50:23 smithi090 ceph-mon[33568]: pgmap v508: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:50:24.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:50:23 smithi149 ceph-mon[33332]: pgmap v508: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:50:26.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:50:25 smithi060 ceph-mon[28873]: pgmap v509: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:50:26.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:50:25 smithi090 ceph-mon[33568]: pgmap v509: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:50:26.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:50:25 smithi149 ceph-mon[33332]: pgmap v509: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:50:28.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:50:27 smithi060 ceph-mon[28873]: pgmap v510: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:50:28.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:50:27 smithi090 ceph-mon[33568]: pgmap v510: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:50:28.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:50:27 smithi149 ceph-mon[33332]: pgmap v510: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:50:30.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:50:29 smithi060 ceph-mon[28873]: pgmap v511: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:50:30.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:50:29 smithi090 ceph-mon[33568]: pgmap v511: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:50:30.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:50:29 smithi149 ceph-mon[33332]: pgmap v511: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:50:32.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:50:31 smithi060 ceph-mon[28873]: pgmap v512: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:50:32.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:50:31 smithi090 ceph-mon[33568]: pgmap v512: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:50:32.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:50:31 smithi149 ceph-mon[33332]: pgmap v512: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:50:34.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:50:33 smithi060 ceph-mon[28873]: pgmap v513: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:50:34.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:50:33 smithi090 ceph-mon[33568]: pgmap v513: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:50:34.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:50:33 smithi149 ceph-mon[33332]: pgmap v513: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:50:36.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:50:35 smithi060 ceph-mon[28873]: pgmap v514: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:50:36.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:50:35 smithi090 ceph-mon[33568]: pgmap v514: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:50:36.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:50:35 smithi149 ceph-mon[33332]: pgmap v514: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:50:38.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:50:37 smithi060 ceph-mon[28873]: pgmap v515: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:50:38.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:50:37 smithi090 ceph-mon[33568]: pgmap v515: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:50:38.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:50:37 smithi149 ceph-mon[33332]: pgmap v515: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:50:40.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:50:39 smithi060 ceph-mon[28873]: pgmap v516: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:50:40.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:50:39 smithi090 ceph-mon[33568]: pgmap v516: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:50:40.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:50:39 smithi149 ceph-mon[33332]: pgmap v516: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:50:42.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:50:41 smithi060 ceph-mon[28873]: pgmap v517: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:50:42.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:50:41 smithi090 ceph-mon[33568]: pgmap v517: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:50:42.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:50:41 smithi149 ceph-mon[33332]: pgmap v517: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:50:44.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:50:43 smithi060 ceph-mon[28873]: pgmap v518: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:50:44.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:50:43 smithi090 ceph-mon[33568]: pgmap v518: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:50:44.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:50:43 smithi149 ceph-mon[33332]: pgmap v518: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:50:45.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:50:44 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:50:45.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:50:44 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:50:45.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:50:44 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:50:46.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:50:45 smithi060 ceph-mon[28873]: pgmap v519: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:50:46.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:50:45 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:50:46.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:50:45 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:50:46.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:50:45 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:50:46.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:50:45 smithi090 ceph-mon[33568]: pgmap v519: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:50:46.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:50:45 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:50:46.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:50:45 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:50:46.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:50:45 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:50:46.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:50:45 smithi149 ceph-mon[33332]: pgmap v519: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:50:46.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:50:45 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:50:46.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:50:45 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:50:46.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:50:45 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:50:48.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:50:47 smithi060 ceph-mon[28873]: pgmap v520: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:50:48.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:50:47 smithi090 ceph-mon[33568]: pgmap v520: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:50:48.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:50:47 smithi149 ceph-mon[33332]: pgmap v520: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:50:50.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:50:49 smithi060 ceph-mon[28873]: pgmap v521: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:50:50.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:50:49 smithi090 ceph-mon[33568]: pgmap v521: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:50:50.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:50:49 smithi149 ceph-mon[33332]: pgmap v521: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:50:52.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:50:51 smithi060 ceph-mon[28873]: pgmap v522: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:50:52.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:50:51 smithi090 ceph-mon[33568]: pgmap v522: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:50:52.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:50:51 smithi149 ceph-mon[33332]: pgmap v522: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:50:54.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:50:53 smithi060 ceph-mon[28873]: pgmap v523: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:50:54.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:50:53 smithi090 ceph-mon[33568]: pgmap v523: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:50:54.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:50:53 smithi149 ceph-mon[33332]: pgmap v523: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:50:56.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:50:55 smithi060 ceph-mon[28873]: pgmap v524: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:50:56.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:50:55 smithi090 ceph-mon[33568]: pgmap v524: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:50:56.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:50:55 smithi149 ceph-mon[33332]: pgmap v524: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:50:57.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:50:57 smithi060 ceph-mon[28873]: pgmap v525: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:50:57.624 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:50:57 smithi090 ceph-mon[33568]: pgmap v525: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:50:57.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:50:57 smithi149 ceph-mon[33332]: pgmap v525: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:50:59.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:50:59 smithi060 ceph-mon[28873]: pgmap v526: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:50:59.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:50:59 smithi090 ceph-mon[33568]: pgmap v526: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:50:59.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:50:59 smithi149 ceph-mon[33332]: pgmap v526: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:51:01.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:51:01 smithi060 ceph-mon[28873]: pgmap v527: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:51:01.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:51:01 smithi090 ceph-mon[33568]: pgmap v527: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:51:01.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:51:01 smithi149 ceph-mon[33332]: pgmap v527: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:51:03.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:51:03 smithi060 ceph-mon[28873]: pgmap v528: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:51:03.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:51:03 smithi090 ceph-mon[33568]: pgmap v528: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:51:03.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:51:03 smithi149 ceph-mon[33332]: pgmap v528: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:51:05.836 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:51:05 smithi060 ceph-mon[28873]: pgmap v529: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:51:05.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:51:05 smithi090 ceph-mon[33568]: pgmap v529: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:51:05.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:51:05 smithi149 ceph-mon[33332]: pgmap v529: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:51:07.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:51:07 smithi060 ceph-mon[28873]: pgmap v530: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:51:07.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:51:07 smithi090 ceph-mon[33568]: pgmap v530: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:51:07.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:51:07 smithi149 ceph-mon[33332]: pgmap v530: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:51:09.836 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:51:09 smithi060 ceph-mon[28873]: pgmap v531: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:51:09.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:51:09 smithi090 ceph-mon[33568]: pgmap v531: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:51:09.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:51:09 smithi149 ceph-mon[33332]: pgmap v531: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:51:11.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:51:11 smithi090 ceph-mon[33568]: pgmap v532: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:51:11.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:51:11 smithi149 ceph-mon[33332]: pgmap v532: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:51:12.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:51:11 smithi060 ceph-mon[28873]: pgmap v532: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:51:13.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:51:13 smithi090 ceph-mon[33568]: pgmap v533: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:51:13.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:51:13 smithi149 ceph-mon[33332]: pgmap v533: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:51:14.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:51:13 smithi060 ceph-mon[28873]: pgmap v533: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:51:15.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:51:15 smithi090 ceph-mon[33568]: pgmap v534: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:51:15.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:51:15 smithi149 ceph-mon[33332]: pgmap v534: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:51:16.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:51:15 smithi060 ceph-mon[28873]: pgmap v534: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:51:17.836 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:51:17 smithi060 ceph-mon[28873]: pgmap v535: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:51:17.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:51:17 smithi090 ceph-mon[33568]: pgmap v535: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:51:17.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:51:17 smithi149 ceph-mon[33332]: pgmap v535: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:51:19.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:51:19 smithi090 ceph-mon[33568]: pgmap v536: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:51:19.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:51:19 smithi149 ceph-mon[33332]: pgmap v536: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:51:20.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:51:19 smithi060 ceph-mon[28873]: pgmap v536: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:51:21.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:51:21 smithi090 ceph-mon[33568]: pgmap v537: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:51:21.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:51:21 smithi149 ceph-mon[33332]: pgmap v537: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:51:22.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:51:21 smithi060 ceph-mon[28873]: pgmap v537: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:51:23.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:51:23 smithi090 ceph-mon[33568]: pgmap v538: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:51:23.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:51:23 smithi149 ceph-mon[33332]: pgmap v538: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:51:24.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:51:23 smithi060 ceph-mon[28873]: pgmap v538: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:51:25.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:51:25 smithi090 ceph-mon[33568]: pgmap v539: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:51:25.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:51:25 smithi149 ceph-mon[33332]: pgmap v539: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:51:26.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:51:25 smithi060 ceph-mon[28873]: pgmap v539: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:51:27.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:51:27 smithi149 ceph-mon[33332]: pgmap v540: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:51:28.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:51:27 smithi060 ceph-mon[28873]: pgmap v540: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:51:28.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:51:27 smithi090 ceph-mon[33568]: pgmap v540: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:51:29.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:51:29 smithi149 ceph-mon[33332]: pgmap v541: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:51:30.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:51:29 smithi060 ceph-mon[28873]: pgmap v541: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:51:30.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:51:29 smithi090 ceph-mon[33568]: pgmap v541: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:51:31.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:51:31 smithi149 ceph-mon[33332]: pgmap v542: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:51:32.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:51:31 smithi060 ceph-mon[28873]: pgmap v542: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:51:32.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:51:31 smithi090 ceph-mon[33568]: pgmap v542: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:51:33.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:51:33 smithi149 ceph-mon[33332]: pgmap v543: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:51:34.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:51:33 smithi060 ceph-mon[28873]: pgmap v543: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:51:34.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:51:33 smithi090 ceph-mon[33568]: pgmap v543: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:51:35.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:51:35 smithi149 ceph-mon[33332]: pgmap v544: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:51:36.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:51:35 smithi060 ceph-mon[28873]: pgmap v544: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:51:36.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:51:35 smithi090 ceph-mon[33568]: pgmap v544: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:51:37.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:51:37 smithi149 ceph-mon[33332]: pgmap v545: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:51:38.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:51:37 smithi060 ceph-mon[28873]: pgmap v545: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:51:38.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:51:37 smithi090 ceph-mon[33568]: pgmap v545: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:51:39.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:51:39 smithi149 ceph-mon[33332]: pgmap v546: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:51:40.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:51:39 smithi060 ceph-mon[28873]: pgmap v546: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:51:40.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:51:39 smithi090 ceph-mon[33568]: pgmap v546: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:51:41.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:51:41 smithi149 ceph-mon[33332]: pgmap v547: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:51:42.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:51:41 smithi060 ceph-mon[28873]: pgmap v547: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:51:42.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:51:41 smithi090 ceph-mon[33568]: pgmap v547: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:51:43.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:51:43 smithi149 ceph-mon[33332]: pgmap v548: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:51:44.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:51:43 smithi060 ceph-mon[28873]: pgmap v548: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:51:44.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:51:43 smithi090 ceph-mon[33568]: pgmap v548: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:51:46.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:51:45 smithi060 ceph-mon[28873]: pgmap v549: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:51:46.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:51:45 smithi090 ceph-mon[33568]: pgmap v549: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:51:46.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:51:45 smithi149 ceph-mon[33332]: pgmap v549: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:51:47.074 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:51:46 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:51:47.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:51:46 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:51:47.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:51:46 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:51:48.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:51:47 smithi060 ceph-mon[28873]: pgmap v550: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:51:48.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:51:47 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:51:48.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:51:47 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:51:48.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:51:47 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:51:48.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:51:47 smithi090 ceph-mon[33568]: pgmap v550: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:51:48.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:51:47 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:51:48.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:51:47 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:51:48.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:51:47 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:51:48.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:51:47 smithi149 ceph-mon[33332]: pgmap v550: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:51:48.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:51:47 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:51:48.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:51:47 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:51:48.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:51:47 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:51:50.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:51:49 smithi060 ceph-mon[28873]: pgmap v551: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:51:50.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:51:49 smithi090 ceph-mon[33568]: pgmap v551: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:51:50.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:51:49 smithi149 ceph-mon[33332]: pgmap v551: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:51:52.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:51:51 smithi060 ceph-mon[28873]: pgmap v552: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:51:52.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:51:51 smithi090 ceph-mon[33568]: pgmap v552: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:51:52.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:51:51 smithi149 ceph-mon[33332]: pgmap v552: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:51:54.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:51:53 smithi060 ceph-mon[28873]: pgmap v553: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:51:54.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:51:53 smithi090 ceph-mon[33568]: pgmap v553: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:51:54.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:51:53 smithi149 ceph-mon[33332]: pgmap v553: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:51:56.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:51:55 smithi060 ceph-mon[28873]: pgmap v554: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:51:56.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:51:55 smithi090 ceph-mon[33568]: pgmap v554: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:51:56.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:51:55 smithi149 ceph-mon[33332]: pgmap v554: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:51:58.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:51:57 smithi060 ceph-mon[28873]: pgmap v555: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:51:58.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:51:57 smithi090 ceph-mon[33568]: pgmap v555: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:51:58.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:51:57 smithi149 ceph-mon[33332]: pgmap v555: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:52:00.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:51:59 smithi060 ceph-mon[28873]: pgmap v556: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:52:00.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:51:59 smithi090 ceph-mon[33568]: pgmap v556: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:52:00.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:51:59 smithi149 ceph-mon[33332]: pgmap v556: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:52:02.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:52:01 smithi060 ceph-mon[28873]: pgmap v557: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:52:02.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:52:01 smithi090 ceph-mon[33568]: pgmap v557: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:52:02.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:52:01 smithi149 ceph-mon[33332]: pgmap v557: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:52:04.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:52:03 smithi060 ceph-mon[28873]: pgmap v558: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:52:04.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:52:03 smithi090 ceph-mon[33568]: pgmap v558: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:52:04.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:52:03 smithi149 ceph-mon[33332]: pgmap v558: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:52:06.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:52:05 smithi060 ceph-mon[28873]: pgmap v559: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:52:06.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:52:05 smithi090 ceph-mon[33568]: pgmap v559: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:52:06.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:52:05 smithi149 ceph-mon[33332]: pgmap v559: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:52:08.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:52:07 smithi060 ceph-mon[28873]: pgmap v560: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:52:08.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:52:07 smithi090 ceph-mon[33568]: pgmap v560: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:52:08.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:52:07 smithi149 ceph-mon[33332]: pgmap v560: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:52:10.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:52:09 smithi060 ceph-mon[28873]: pgmap v561: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:52:10.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:52:09 smithi090 ceph-mon[33568]: pgmap v561: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:52:10.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:52:09 smithi149 ceph-mon[33332]: pgmap v561: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:52:12.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:52:11 smithi060 ceph-mon[28873]: pgmap v562: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:52:12.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:52:11 smithi090 ceph-mon[33568]: pgmap v562: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:52:12.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:52:11 smithi149 ceph-mon[33332]: pgmap v562: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:52:14.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:52:13 smithi060 ceph-mon[28873]: pgmap v563: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:52:14.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:52:13 smithi090 ceph-mon[33568]: pgmap v563: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:52:14.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:52:13 smithi149 ceph-mon[33332]: pgmap v563: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:52:16.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:52:15 smithi060 ceph-mon[28873]: pgmap v564: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:52:16.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:52:15 smithi090 ceph-mon[33568]: pgmap v564: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:52:16.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:52:15 smithi149 ceph-mon[33332]: pgmap v564: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:52:18.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:52:17 smithi060 ceph-mon[28873]: pgmap v565: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:52:18.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:52:17 smithi090 ceph-mon[33568]: pgmap v565: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:52:18.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:52:17 smithi149 ceph-mon[33332]: pgmap v565: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:52:20.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:52:19 smithi060 ceph-mon[28873]: pgmap v566: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:52:20.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:52:19 smithi090 ceph-mon[33568]: pgmap v566: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:52:20.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:52:19 smithi149 ceph-mon[33332]: pgmap v566: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:52:22.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:52:21 smithi060 ceph-mon[28873]: pgmap v567: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:52:22.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:52:21 smithi090 ceph-mon[33568]: pgmap v567: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:52:22.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:52:21 smithi149 ceph-mon[33332]: pgmap v567: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:52:24.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:52:23 smithi060 ceph-mon[28873]: pgmap v568: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:52:24.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:52:23 smithi090 ceph-mon[33568]: pgmap v568: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:52:24.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:52:23 smithi149 ceph-mon[33332]: pgmap v568: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:52:26.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:52:25 smithi060 ceph-mon[28873]: pgmap v569: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:52:26.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:52:25 smithi090 ceph-mon[33568]: pgmap v569: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:52:26.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:52:25 smithi149 ceph-mon[33332]: pgmap v569: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:52:28.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:52:27 smithi060 ceph-mon[28873]: pgmap v570: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:52:28.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:52:27 smithi090 ceph-mon[33568]: pgmap v570: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:52:28.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:52:27 smithi149 ceph-mon[33332]: pgmap v570: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:52:30.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:52:29 smithi060 ceph-mon[28873]: pgmap v571: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:52:30.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:52:29 smithi090 ceph-mon[33568]: pgmap v571: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:52:30.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:52:29 smithi149 ceph-mon[33332]: pgmap v571: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:52:32.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:52:31 smithi060 ceph-mon[28873]: pgmap v572: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:52:32.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:52:31 smithi090 ceph-mon[33568]: pgmap v572: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:52:32.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:52:31 smithi149 ceph-mon[33332]: pgmap v572: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:52:34.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:52:33 smithi060 ceph-mon[28873]: pgmap v573: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:52:34.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:52:33 smithi090 ceph-mon[33568]: pgmap v573: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:52:34.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:52:33 smithi149 ceph-mon[33332]: pgmap v573: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:52:36.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:52:35 smithi060 ceph-mon[28873]: pgmap v574: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:52:36.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:52:35 smithi090 ceph-mon[33568]: pgmap v574: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:52:36.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:52:35 smithi149 ceph-mon[33332]: pgmap v574: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:52:38.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:52:37 smithi060 ceph-mon[28873]: pgmap v575: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:52:38.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:52:37 smithi090 ceph-mon[33568]: pgmap v575: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:52:38.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:52:37 smithi149 ceph-mon[33332]: pgmap v575: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:52:40.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:52:39 smithi060 ceph-mon[28873]: pgmap v576: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:52:40.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:52:39 smithi090 ceph-mon[33568]: pgmap v576: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:52:40.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:52:39 smithi149 ceph-mon[33332]: pgmap v576: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:52:41.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:52:41.856+0000 7efdfd600640 1 -- 172.21.15.60:0/3449599367 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7efde00034c0 con 0x7efe0810a970 2024-05-03T14:52:41.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:52:41.856+0000 7efdfea00640 1 -- 172.21.15.60:0/3449599367 <== mon.2 v2:172.21.15.90:3300/0 7 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7efde00034c0 con 0x7efe0810a970 2024-05-03T14:52:42.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:52:41 smithi060 ceph-mon[28873]: pgmap v577: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:52:42.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:52:41 smithi090 ceph-mon[33568]: pgmap v577: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:52:42.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:52:41 smithi149 ceph-mon[33332]: pgmap v577: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:52:44.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:52:43 smithi060 ceph-mon[28873]: pgmap v578: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:52:44.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:52:43 smithi090 ceph-mon[33568]: pgmap v578: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:52:44.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:52:43 smithi149 ceph-mon[33332]: pgmap v578: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:52:46.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:52:45 smithi090 ceph-mon[33568]: pgmap v579: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:52:46.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:52:45 smithi149 ceph-mon[33332]: pgmap v579: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:52:46.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:52:45 smithi060 ceph-mon[28873]: pgmap v579: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:52:48.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:52:47 smithi090 ceph-mon[33568]: pgmap v580: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:52:48.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:52:47 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:52:48.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:52:47 smithi149 ceph-mon[33332]: pgmap v580: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:52:48.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:52:47 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:52:48.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:52:47 smithi060 ceph-mon[28873]: pgmap v580: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:52:48.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:52:47 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:52:50.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:52:49 smithi090 ceph-mon[33568]: pgmap v581: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:52:50.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:52:49 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:52:50.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:52:49 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:52:50.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:52:49 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:52:50.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:52:49 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:52:50.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:52:49 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:52:50.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:52:49 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:52:50.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:52:49 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:52:50.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:52:49 smithi149 ceph-mon[33332]: pgmap v581: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:52:50.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:52:49 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:52:50.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:52:49 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:52:50.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:52:49 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:52:50.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:52:49 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:52:50.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:52:49 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:52:50.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:52:49 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:52:50.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:52:49 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:52:50.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:52:49 smithi060 ceph-mon[28873]: pgmap v581: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:52:50.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:52:49 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:52:50.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:52:49 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:52:50.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:52:49 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:52:50.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:52:49 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:52:50.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:52:49 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:52:50.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:52:49 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:52:50.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:52:49 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:52:51.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:52:50 smithi090 ceph-mon[33568]: Adjusting osd_memory_target on smithi060 to 177.5M 2024-05-03T14:52:51.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:52:50 smithi090 ceph-mon[33568]: Unable to set osd_memory_target on smithi060 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-05-03T14:52:51.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:52:50 smithi149 ceph-mon[33332]: Adjusting osd_memory_target on smithi060 to 177.5M 2024-05-03T14:52:51.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:52:50 smithi149 ceph-mon[33332]: Unable to set osd_memory_target on smithi060 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-05-03T14:52:51.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:52:50 smithi060 ceph-mon[28873]: Adjusting osd_memory_target on smithi060 to 177.5M 2024-05-03T14:52:51.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:52:50 smithi060 ceph-mon[28873]: Unable to set osd_memory_target on smithi060 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-05-03T14:52:52.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:52:51 smithi090 ceph-mon[33568]: pgmap v582: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:52:52.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:52:51 smithi149 ceph-mon[33332]: pgmap v582: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:52:52.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:52:51 smithi060 ceph-mon[28873]: pgmap v582: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:52:54.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:52:53 smithi090 ceph-mon[33568]: pgmap v583: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:52:54.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:52:53 smithi149 ceph-mon[33332]: pgmap v583: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:52:54.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:52:53 smithi060 ceph-mon[28873]: pgmap v583: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:52:56.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:52:55 smithi090 ceph-mon[33568]: pgmap v584: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:52:56.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:52:55 smithi149 ceph-mon[33332]: pgmap v584: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:52:56.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:52:55 smithi060 ceph-mon[28873]: pgmap v584: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:52:57.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:52:56 smithi149 ceph-mon[33332]: pgmap v585: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:52:57.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:52:56 smithi060 ceph-mon[28873]: pgmap v585: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:52:57.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:52:56 smithi090 ceph-mon[33568]: pgmap v585: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:52:59.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:52:59 smithi090 ceph-mon[33568]: pgmap v586: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:52:59.922 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:52:59 smithi149 ceph-mon[33332]: pgmap v586: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:53:00.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:52:59 smithi060 ceph-mon[28873]: pgmap v586: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:53:01.922 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:53:01 smithi149 ceph-mon[33332]: pgmap v587: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:53:02.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:53:01 smithi060 ceph-mon[28873]: pgmap v587: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:53:02.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:53:01 smithi090 ceph-mon[33568]: pgmap v587: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:53:03.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:53:03 smithi149 ceph-mon[33332]: pgmap v588: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:53:04.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:53:03 smithi060 ceph-mon[28873]: pgmap v588: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:53:04.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:53:03 smithi090 ceph-mon[33568]: pgmap v588: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:53:05.889 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:53:05 smithi060 ceph-mon[28873]: pgmap v589: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:53:05.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:53:05 smithi149 ceph-mon[33332]: pgmap v589: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:53:06.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:53:05 smithi090 ceph-mon[33568]: pgmap v589: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:53:07.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:53:07 smithi149 ceph-mon[33332]: pgmap v590: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:53:08.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:53:07 smithi060 ceph-mon[28873]: pgmap v590: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:53:08.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:53:07 smithi090 ceph-mon[33568]: pgmap v590: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:53:09.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:53:09 smithi149 ceph-mon[33332]: pgmap v591: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:53:10.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:53:09 smithi060 ceph-mon[28873]: pgmap v591: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:53:10.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:53:09 smithi090 ceph-mon[33568]: pgmap v591: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:53:11.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:53:11 smithi149 ceph-mon[33332]: pgmap v592: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:53:12.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:53:11 smithi060 ceph-mon[28873]: pgmap v592: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:53:12.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:53:11 smithi090 ceph-mon[33568]: pgmap v592: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:53:13.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:53:13 smithi149 ceph-mon[33332]: pgmap v593: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:53:14.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:53:13 smithi060 ceph-mon[28873]: pgmap v593: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:53:14.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:53:13 smithi090 ceph-mon[33568]: pgmap v593: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:53:15.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:53:15 smithi149 ceph-mon[33332]: pgmap v594: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:53:16.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:53:15 smithi060 ceph-mon[28873]: pgmap v594: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:53:16.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:53:15 smithi090 ceph-mon[33568]: pgmap v594: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:53:17.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:53:17 smithi060 ceph-mon[28873]: pgmap v595: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:53:17.624 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:53:17 smithi090 ceph-mon[33568]: pgmap v595: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:53:17.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:53:17 smithi149 ceph-mon[33332]: pgmap v595: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:53:19.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:53:19 smithi149 ceph-mon[33332]: pgmap v596: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:53:20.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:53:19 smithi060 ceph-mon[28873]: pgmap v596: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:53:20.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:53:19 smithi090 ceph-mon[33568]: pgmap v596: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:53:21.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:53:21 smithi149 ceph-mon[33332]: pgmap v597: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:53:22.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:53:21 smithi060 ceph-mon[28873]: pgmap v597: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:53:22.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:53:21 smithi090 ceph-mon[33568]: pgmap v597: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:53:23.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:53:23 smithi149 ceph-mon[33332]: pgmap v598: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:53:24.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:53:23 smithi060 ceph-mon[28873]: pgmap v598: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:53:24.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:53:23 smithi090 ceph-mon[33568]: pgmap v598: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:53:25.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:53:25 smithi149 ceph-mon[33332]: pgmap v599: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:53:26.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:53:25 smithi060 ceph-mon[28873]: pgmap v599: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:53:26.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:53:25 smithi090 ceph-mon[33568]: pgmap v599: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:53:27.922 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:53:27 smithi149 ceph-mon[33332]: pgmap v600: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:53:27.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:53:27 smithi090 ceph-mon[33568]: pgmap v600: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:53:28.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:53:27 smithi060 ceph-mon[28873]: pgmap v600: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:53:29.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:53:29 smithi149 ceph-mon[33332]: pgmap v601: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:53:30.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:53:29 smithi060 ceph-mon[28873]: pgmap v601: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:53:30.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:53:29 smithi090 ceph-mon[33568]: pgmap v601: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:53:31.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:53:31 smithi149 ceph-mon[33332]: pgmap v602: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:53:32.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:53:31 smithi060 ceph-mon[28873]: pgmap v602: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:53:32.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:53:31 smithi090 ceph-mon[33568]: pgmap v602: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:53:33.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:53:33 smithi149 ceph-mon[33332]: pgmap v603: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:53:34.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:53:33 smithi060 ceph-mon[28873]: pgmap v603: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:53:34.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:53:33 smithi090 ceph-mon[33568]: pgmap v603: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:53:35.923 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:53:35 smithi149 ceph-mon[33332]: pgmap v604: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:53:36.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:53:35 smithi060 ceph-mon[28873]: pgmap v604: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:53:36.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:53:35 smithi090 ceph-mon[33568]: pgmap v604: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:53:38.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:53:37 smithi060 ceph-mon[28873]: pgmap v605: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:53:38.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:53:37 smithi090 ceph-mon[33568]: pgmap v605: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:53:38.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:53:37 smithi149 ceph-mon[33332]: pgmap v605: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:53:40.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:53:39 smithi060 ceph-mon[28873]: pgmap v606: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:53:40.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:53:39 smithi090 ceph-mon[33568]: pgmap v606: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:53:40.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:53:39 smithi149 ceph-mon[33332]: pgmap v606: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:53:42.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:53:41 smithi060 ceph-mon[28873]: pgmap v607: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:53:42.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:53:41 smithi090 ceph-mon[33568]: pgmap v607: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:53:42.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:53:41 smithi149 ceph-mon[33332]: pgmap v607: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:53:44.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:53:43 smithi060 ceph-mon[28873]: pgmap v608: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:53:44.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:53:43 smithi090 ceph-mon[33568]: pgmap v608: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:53:44.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:53:43 smithi149 ceph-mon[33332]: pgmap v608: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:53:46.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:53:45 smithi060 ceph-mon[28873]: pgmap v609: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:53:46.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:53:45 smithi090 ceph-mon[33568]: pgmap v609: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:53:46.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:53:45 smithi149 ceph-mon[33332]: pgmap v609: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:53:47.975 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:53:47 smithi149 ceph-mon[33332]: pgmap v610: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:53:48.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:53:47 smithi060 ceph-mon[28873]: pgmap v610: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:53:48.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:53:47 smithi090 ceph-mon[33568]: pgmap v610: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:53:50.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:53:49 smithi060 ceph-mon[28873]: pgmap v611: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:53:50.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:53:49 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:53:50.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:53:49 smithi090 ceph-mon[33568]: pgmap v611: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:53:50.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:53:49 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:53:50.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:53:49 smithi149 ceph-mon[33332]: pgmap v611: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:53:50.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:53:49 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:53:51.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:53:50 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:53:51.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:53:50 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:53:51.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:53:50 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]': finished 2024-05-03T14:53:51.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:53:50 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:53:51.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:53:50 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:53:51.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:53:50 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:53:51.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:53:50 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:53:51.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:53:50 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:53:51.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:53:50 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]': finished 2024-05-03T14:53:51.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:53:50 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:53:51.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:53:50 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:53:51.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:53:50 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:53:51.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:53:50 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:53:51.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:53:50 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:53:51.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:53:50 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd='[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]': finished 2024-05-03T14:53:51.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:53:50 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:53:51.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:53:50 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:53:51.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:53:50 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-03T14:53:52.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:53:51 smithi060 ceph-mon[28873]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-05-03T14:53:52.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:53:51 smithi060 ceph-mon[28873]: Unable to set osd_memory_target on smithi090 to 184763801: error parsing value: Value '184763801' is below minimum 939524096 2024-05-03T14:53:52.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:53:51 smithi060 ceph-mon[28873]: pgmap v612: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:53:52.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:53:51 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:53:52.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:53:51 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:53:52.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:53:51 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:53:52.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:53:51 smithi090 ceph-mon[33568]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-05-03T14:53:52.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:53:51 smithi090 ceph-mon[33568]: Unable to set osd_memory_target on smithi090 to 184763801: error parsing value: Value '184763801' is below minimum 939524096 2024-05-03T14:53:52.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:53:51 smithi090 ceph-mon[33568]: pgmap v612: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:53:52.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:53:51 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:53:52.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:53:51 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:53:52.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:53:51 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:53:52.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:53:51 smithi149 ceph-mon[33332]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-05-03T14:53:52.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:53:51 smithi149 ceph-mon[33332]: Unable to set osd_memory_target on smithi090 to 184763801: error parsing value: Value '184763801' is below minimum 939524096 2024-05-03T14:53:52.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:53:51 smithi149 ceph-mon[33332]: pgmap v612: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:53:52.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:53:51 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:53:52.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:53:51 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:53:52.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:53:51 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:53:54.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:53:53 smithi060 ceph-mon[28873]: pgmap v613: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:53:54.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:53:53 smithi090 ceph-mon[33568]: pgmap v613: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:53:54.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:53:53 smithi149 ceph-mon[33332]: pgmap v613: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:53:56.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:53:55 smithi060 ceph-mon[28873]: pgmap v614: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:53:56.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:53:55 smithi090 ceph-mon[33568]: pgmap v614: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:53:56.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:53:55 smithi149 ceph-mon[33332]: pgmap v614: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:53:58.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:53:57 smithi060 ceph-mon[28873]: pgmap v615: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:53:58.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:53:57 smithi090 ceph-mon[33568]: pgmap v615: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:53:58.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:53:57 smithi149 ceph-mon[33332]: pgmap v615: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:54:00.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:53:59 smithi060 ceph-mon[28873]: pgmap v616: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:54:00.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:53:59 smithi090 ceph-mon[33568]: pgmap v616: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:54:00.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:53:59 smithi149 ceph-mon[33332]: pgmap v616: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:54:02.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:54:01 smithi060 ceph-mon[28873]: pgmap v617: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:54:02.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:54:01 smithi090 ceph-mon[33568]: pgmap v617: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:54:02.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:54:01 smithi149 ceph-mon[33332]: pgmap v617: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:54:04.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:54:03 smithi060 ceph-mon[28873]: pgmap v618: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:54:04.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:54:03 smithi090 ceph-mon[33568]: pgmap v618: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:54:04.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:54:03 smithi149 ceph-mon[33332]: pgmap v618: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:54:06.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:54:05 smithi060 ceph-mon[28873]: pgmap v619: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:54:06.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:54:05 smithi090 ceph-mon[33568]: pgmap v619: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:54:06.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:54:05 smithi149 ceph-mon[33332]: pgmap v619: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:54:08.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:54:07 smithi060 ceph-mon[28873]: pgmap v620: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:54:08.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:54:07 smithi090 ceph-mon[33568]: pgmap v620: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:54:08.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:54:07 smithi149 ceph-mon[33332]: pgmap v620: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:54:10.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:54:09 smithi060 ceph-mon[28873]: pgmap v621: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:54:10.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:54:09 smithi090 ceph-mon[33568]: pgmap v621: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:54:10.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:54:09 smithi149 ceph-mon[33332]: pgmap v621: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:54:12.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:54:11 smithi060 ceph-mon[28873]: pgmap v622: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:54:12.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:54:11 smithi090 ceph-mon[33568]: pgmap v622: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:54:12.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:54:11 smithi149 ceph-mon[33332]: pgmap v622: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:54:14.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:54:13 smithi060 ceph-mon[28873]: pgmap v623: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:54:14.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:54:13 smithi090 ceph-mon[33568]: pgmap v623: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:54:14.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:54:13 smithi149 ceph-mon[33332]: pgmap v623: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:54:16.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:54:15 smithi060 ceph-mon[28873]: pgmap v624: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:54:16.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:54:15 smithi090 ceph-mon[33568]: pgmap v624: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:54:16.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:54:15 smithi149 ceph-mon[33332]: pgmap v624: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:54:18.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:54:17 smithi060 ceph-mon[28873]: pgmap v625: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:54:18.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:54:17 smithi090 ceph-mon[33568]: pgmap v625: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:54:18.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:54:17 smithi149 ceph-mon[33332]: pgmap v625: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:54:20.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:54:19 smithi060 ceph-mon[28873]: pgmap v626: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:54:20.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:54:19 smithi090 ceph-mon[33568]: pgmap v626: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:54:20.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:54:19 smithi149 ceph-mon[33332]: pgmap v626: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:54:22.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:54:21 smithi060 ceph-mon[28873]: pgmap v627: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:54:22.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:54:21 smithi090 ceph-mon[33568]: pgmap v627: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:54:22.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:54:21 smithi149 ceph-mon[33332]: pgmap v627: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:54:24.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:54:23 smithi060 ceph-mon[28873]: pgmap v628: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:54:24.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:54:23 smithi090 ceph-mon[33568]: pgmap v628: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:54:24.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:54:23 smithi149 ceph-mon[33332]: pgmap v628: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:54:26.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:54:25 smithi060 ceph-mon[28873]: pgmap v629: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:54:26.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:54:25 smithi090 ceph-mon[33568]: pgmap v629: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:54:26.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:54:25 smithi149 ceph-mon[33332]: pgmap v629: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:54:28.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:54:27 smithi060 ceph-mon[28873]: pgmap v630: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:54:28.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:54:27 smithi090 ceph-mon[33568]: pgmap v630: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:54:28.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:54:27 smithi149 ceph-mon[33332]: pgmap v630: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:54:30.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:54:29 smithi060 ceph-mon[28873]: pgmap v631: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:54:30.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:54:29 smithi090 ceph-mon[33568]: pgmap v631: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:54:30.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:54:29 smithi149 ceph-mon[33332]: pgmap v631: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:54:32.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:54:31 smithi060 ceph-mon[28873]: pgmap v632: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:54:32.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:54:31 smithi090 ceph-mon[33568]: pgmap v632: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:54:32.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:54:31 smithi149 ceph-mon[33332]: pgmap v632: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:54:34.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:54:33 smithi060 ceph-mon[28873]: pgmap v633: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:54:34.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:54:33 smithi090 ceph-mon[33568]: pgmap v633: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:54:34.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:54:33 smithi149 ceph-mon[33332]: pgmap v633: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:54:36.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:54:35 smithi060 ceph-mon[28873]: pgmap v634: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:54:36.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:54:35 smithi090 ceph-mon[33568]: pgmap v634: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:54:36.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:54:35 smithi149 ceph-mon[33332]: pgmap v634: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:54:38.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:54:37 smithi060 ceph-mon[28873]: pgmap v635: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:54:38.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:54:37 smithi090 ceph-mon[33568]: pgmap v635: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:54:38.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:54:37 smithi149 ceph-mon[33332]: pgmap v635: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:54:40.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:54:39 smithi090 ceph-mon[33568]: pgmap v636: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:54:40.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:54:39 smithi149 ceph-mon[33332]: pgmap v636: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:54:40.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:54:39 smithi060 ceph-mon[28873]: pgmap v636: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:54:42.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:54:41 smithi090 ceph-mon[33568]: pgmap v637: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:54:42.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:54:41 smithi149 ceph-mon[33332]: pgmap v637: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:54:42.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:54:41 smithi060 ceph-mon[28873]: pgmap v637: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:54:44.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:54:43 smithi090 ceph-mon[33568]: pgmap v638: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:54:44.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:54:43 smithi149 ceph-mon[33332]: pgmap v638: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:54:44.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:54:43 smithi060 ceph-mon[28873]: pgmap v638: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:54:46.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:54:45 smithi090 ceph-mon[33568]: pgmap v639: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:54:46.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:54:45 smithi149 ceph-mon[33332]: pgmap v639: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:54:46.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:54:45 smithi060 ceph-mon[28873]: pgmap v639: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:54:48.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:54:47 smithi090 ceph-mon[33568]: pgmap v640: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:54:48.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:54:47 smithi149 ceph-mon[33332]: pgmap v640: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:54:48.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:54:47 smithi060 ceph-mon[28873]: pgmap v640: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:54:50.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:54:49 smithi090 ceph-mon[33568]: pgmap v641: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:54:50.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:54:49 smithi149 ceph-mon[33332]: pgmap v641: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:54:50.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:54:49 smithi060 ceph-mon[28873]: pgmap v641: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:54:52.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:54:51 smithi090 ceph-mon[33568]: pgmap v642: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:54:52.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:54:51 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:54:52.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:54:51 smithi149 ceph-mon[33332]: pgmap v642: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:54:52.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:54:51 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:54:52.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:54:51 smithi060 ceph-mon[28873]: pgmap v642: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:54:52.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:54:51 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:54:53.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:54:52 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:54:53.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:54:52 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:54:53.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:54:52 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:54:53.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:54:52 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:54:53.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:54:52 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:54:53.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:54:52 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:54:53.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:54:52 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:54:53.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:54:52 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:54:53.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:54:52 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:54:54.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:54:53 smithi149 ceph-mon[33332]: pgmap v643: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:54:54.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:54:53 smithi060 ceph-mon[28873]: pgmap v643: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:54:54.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:54:53 smithi090 ceph-mon[33568]: pgmap v643: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:54:56.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:54:55 smithi149 ceph-mon[33332]: pgmap v644: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:54:56.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:54:55 smithi060 ceph-mon[28873]: pgmap v644: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:54:56.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:54:55 smithi090 ceph-mon[33568]: pgmap v644: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:54:57.586 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:54:57 smithi060 ceph-mon[28873]: pgmap v645: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:54:57.624 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:54:57 smithi090 ceph-mon[33568]: pgmap v645: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:54:57.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:54:57 smithi149 ceph-mon[33332]: pgmap v645: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:55:00.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:54:59 smithi060 ceph-mon[28873]: pgmap v646: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:55:00.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:54:59 smithi090 ceph-mon[33568]: pgmap v646: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:55:00.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:54:59 smithi149 ceph-mon[33332]: pgmap v646: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:55:02.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:55:01 smithi060 ceph-mon[28873]: pgmap v647: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:55:02.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:55:01 smithi090 ceph-mon[33568]: pgmap v647: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:55:02.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:55:01 smithi149 ceph-mon[33332]: pgmap v647: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:55:04.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:55:03 smithi060 ceph-mon[28873]: pgmap v648: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:55:04.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:55:03 smithi090 ceph-mon[33568]: pgmap v648: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:55:04.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:55:03 smithi149 ceph-mon[33332]: pgmap v648: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:55:06.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:55:05 smithi060 ceph-mon[28873]: pgmap v649: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:55:06.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:55:05 smithi090 ceph-mon[33568]: pgmap v649: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:55:06.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:55:05 smithi149 ceph-mon[33332]: pgmap v649: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:55:08.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:55:07 smithi060 ceph-mon[28873]: pgmap v650: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:55:08.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:55:07 smithi090 ceph-mon[33568]: pgmap v650: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:55:08.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:55:07 smithi149 ceph-mon[33332]: pgmap v650: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:55:10.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:55:09 smithi060 ceph-mon[28873]: pgmap v651: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:55:10.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:55:09 smithi090 ceph-mon[33568]: pgmap v651: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:55:10.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:55:09 smithi149 ceph-mon[33332]: pgmap v651: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:55:12.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:55:11 smithi060 ceph-mon[28873]: pgmap v652: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:55:12.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:55:11 smithi090 ceph-mon[33568]: pgmap v652: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:55:12.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:55:11 smithi149 ceph-mon[33332]: pgmap v652: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:55:14.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:55:13 smithi060 ceph-mon[28873]: pgmap v653: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:55:14.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:55:13 smithi090 ceph-mon[33568]: pgmap v653: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:55:14.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:55:13 smithi149 ceph-mon[33332]: pgmap v653: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:55:16.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:55:15 smithi060 ceph-mon[28873]: pgmap v654: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:55:16.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:55:15 smithi090 ceph-mon[33568]: pgmap v654: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:55:16.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:55:15 smithi149 ceph-mon[33332]: pgmap v654: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:55:18.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:55:17 smithi060 ceph-mon[28873]: pgmap v655: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:55:18.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:55:17 smithi090 ceph-mon[33568]: pgmap v655: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:55:18.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:55:17 smithi149 ceph-mon[33332]: pgmap v655: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:55:20.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:55:19 smithi060 ceph-mon[28873]: pgmap v656: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:55:20.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:55:19 smithi090 ceph-mon[33568]: pgmap v656: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:55:20.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:55:19 smithi149 ceph-mon[33332]: pgmap v656: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:55:22.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:55:21 smithi060 ceph-mon[28873]: pgmap v657: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:55:22.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:55:21 smithi090 ceph-mon[33568]: pgmap v657: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:55:22.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:55:21 smithi149 ceph-mon[33332]: pgmap v657: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:55:24.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:55:23 smithi060 ceph-mon[28873]: pgmap v658: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:55:24.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:55:23 smithi090 ceph-mon[33568]: pgmap v658: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:55:24.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:55:23 smithi149 ceph-mon[33332]: pgmap v658: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:55:26.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:55:25 smithi060 ceph-mon[28873]: pgmap v659: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:55:26.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:55:25 smithi090 ceph-mon[33568]: pgmap v659: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:55:26.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:55:25 smithi149 ceph-mon[33332]: pgmap v659: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:55:28.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:55:27 smithi060 ceph-mon[28873]: pgmap v660: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:55:28.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:55:27 smithi090 ceph-mon[33568]: pgmap v660: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:55:28.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:55:27 smithi149 ceph-mon[33332]: pgmap v660: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:55:30.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:55:29 smithi060 ceph-mon[28873]: pgmap v661: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-05-03T14:55:30.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:55:29 smithi090 ceph-mon[33568]: pgmap v661: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-05-03T14:55:30.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:55:29 smithi149 ceph-mon[33332]: pgmap v661: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-05-03T14:55:32.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:55:31 smithi060 ceph-mon[28873]: pgmap v662: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-03T14:55:32.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:55:31 smithi090 ceph-mon[33568]: pgmap v662: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-03T14:55:32.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:55:31 smithi149 ceph-mon[33332]: pgmap v662: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-03T14:55:34.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:55:33 smithi060 ceph-mon[28873]: pgmap v663: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-05-03T14:55:34.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:55:33 smithi090 ceph-mon[33568]: pgmap v663: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-05-03T14:55:34.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:55:33 smithi149 ceph-mon[33332]: pgmap v663: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-05-03T14:55:36.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:55:35 smithi060 ceph-mon[28873]: pgmap v664: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-05-03T14:55:36.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:55:35 smithi090 ceph-mon[33568]: pgmap v664: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-05-03T14:55:36.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:55:35 smithi149 ceph-mon[33332]: pgmap v664: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-05-03T14:55:38.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:55:37 smithi060 ceph-mon[28873]: pgmap v665: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-03T14:55:38.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:55:37 smithi090 ceph-mon[33568]: pgmap v665: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-03T14:55:38.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:55:37 smithi149 ceph-mon[33332]: pgmap v665: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-03T14:55:40.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:55:39 smithi060 ceph-mon[28873]: pgmap v666: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-05-03T14:55:40.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:55:39 smithi090 ceph-mon[33568]: pgmap v666: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-05-03T14:55:40.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:55:39 smithi149 ceph-mon[33332]: pgmap v666: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-05-03T14:55:41.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:55:41.886+0000 7efdfd600640 1 -- 172.21.15.60:0/3449599367 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7efde0005850 con 0x7efe0810a970 2024-05-03T14:55:41.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:55:41.887+0000 7efdfea00640 1 -- 172.21.15.60:0/3449599367 <== mon.2 v2:172.21.15.90:3300/0 8 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7efde0005850 con 0x7efe0810a970 2024-05-03T14:55:42.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:55:41 smithi060 ceph-mon[28873]: pgmap v667: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:55:42.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:55:41 smithi090 ceph-mon[33568]: pgmap v667: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:55:42.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:55:41 smithi149 ceph-mon[33332]: pgmap v667: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:55:44.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:55:43 smithi060 ceph-mon[28873]: pgmap v668: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:55:44.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:55:43 smithi090 ceph-mon[33568]: pgmap v668: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:55:44.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:55:43 smithi149 ceph-mon[33332]: pgmap v668: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:55:46.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:55:45 smithi060 ceph-mon[28873]: pgmap v669: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:55:46.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:55:45 smithi090 ceph-mon[33568]: pgmap v669: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:55:46.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:55:45 smithi149 ceph-mon[33332]: pgmap v669: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:55:48.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:55:47 smithi060 ceph-mon[28873]: pgmap v670: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-05-03T14:55:48.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:55:47 smithi090 ceph-mon[33568]: pgmap v670: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-05-03T14:55:48.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:55:47 smithi149 ceph-mon[33332]: pgmap v670: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-05-03T14:55:50.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:55:49 smithi060 ceph-mon[28873]: pgmap v671: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-03T14:55:50.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:55:49 smithi090 ceph-mon[33568]: pgmap v671: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-03T14:55:50.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:55:49 smithi149 ceph-mon[33332]: pgmap v671: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-03T14:55:52.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:55:51 smithi060 ceph-mon[28873]: pgmap v672: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-05-03T14:55:52.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:55:51 smithi090 ceph-mon[33568]: pgmap v672: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-05-03T14:55:52.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:55:51 smithi149 ceph-mon[33332]: pgmap v672: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-05-03T14:55:53.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:55:52 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:55:53.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:55:52 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:55:53.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:55:52 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:55:54.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:55:53 smithi060 ceph-mon[28873]: pgmap v673: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-03T14:55:54.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:55:53 smithi090 ceph-mon[33568]: pgmap v673: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-03T14:55:54.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:55:53 smithi149 ceph-mon[33332]: pgmap v673: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-03T14:55:55.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:55:54 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:55:55.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:55:54 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:55:55.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:55:54 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:55:55.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:55:54 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:55:55.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:55:54 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:55:55.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:55:54 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:55:55.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:55:54 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:55:55.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:55:54 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:55:55.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:55:54 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:55:56.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:55:55 smithi060 ceph-mon[28873]: pgmap v674: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 9 op/s 2024-05-03T14:55:56.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:55:55 smithi090 ceph-mon[33568]: pgmap v674: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 9 op/s 2024-05-03T14:55:56.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:55:55 smithi149 ceph-mon[33332]: pgmap v674: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 9 op/s 2024-05-03T14:55:58.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:55:57 smithi090 ceph-mon[33568]: pgmap v675: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-05-03T14:55:58.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:55:57 smithi149 ceph-mon[33332]: pgmap v675: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-05-03T14:55:58.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:55:57 smithi060 ceph-mon[28873]: pgmap v675: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-05-03T14:56:00.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:55:59 smithi090 ceph-mon[33568]: pgmap v676: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:56:00.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:55:59 smithi149 ceph-mon[33332]: pgmap v676: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:56:00.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:55:59 smithi060 ceph-mon[28873]: pgmap v676: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:56:02.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:56:01 smithi090 ceph-mon[33568]: pgmap v677: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-03T14:56:02.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:56:01 smithi149 ceph-mon[33332]: pgmap v677: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-03T14:56:02.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:56:01 smithi060 ceph-mon[28873]: pgmap v677: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-03T14:56:04.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:56:03 smithi090 ceph-mon[33568]: pgmap v678: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-05-03T14:56:04.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:56:03 smithi149 ceph-mon[33332]: pgmap v678: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-05-03T14:56:04.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:56:03 smithi060 ceph-mon[28873]: pgmap v678: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-05-03T14:56:06.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:56:05 smithi090 ceph-mon[33568]: pgmap v679: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-05-03T14:56:06.139 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:56:05 smithi060 ceph-mon[28873]: pgmap v679: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-05-03T14:56:06.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:56:05 smithi149 ceph-mon[33332]: pgmap v679: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-05-03T14:56:08.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:56:07 smithi090 ceph-mon[33568]: pgmap v680: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-03T14:56:08.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:56:07 smithi149 ceph-mon[33332]: pgmap v680: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-03T14:56:08.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:56:07 smithi060 ceph-mon[28873]: pgmap v680: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-03T14:56:10.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:56:09 smithi090 ceph-mon[33568]: pgmap v681: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-05-03T14:56:10.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:56:09 smithi149 ceph-mon[33332]: pgmap v681: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-05-03T14:56:10.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:56:09 smithi060 ceph-mon[28873]: pgmap v681: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-05-03T14:56:12.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:56:11 smithi149 ceph-mon[33332]: pgmap v682: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-03T14:56:12.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:56:11 smithi060 ceph-mon[28873]: pgmap v682: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-03T14:56:12.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:56:11 smithi090 ceph-mon[33568]: pgmap v682: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-03T14:56:14.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:56:13 smithi149 ceph-mon[33332]: pgmap v683: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:56:14.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:56:13 smithi060 ceph-mon[28873]: pgmap v683: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:56:14.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:56:13 smithi090 ceph-mon[33568]: pgmap v683: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:56:16.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:56:15 smithi149 ceph-mon[33332]: pgmap v684: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:56:16.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:56:15 smithi060 ceph-mon[28873]: pgmap v684: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:56:16.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:56:15 smithi090 ceph-mon[33568]: pgmap v684: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:56:18.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:56:17 smithi149 ceph-mon[33332]: pgmap v685: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:56:18.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:56:17 smithi060 ceph-mon[28873]: pgmap v685: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:56:18.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:56:17 smithi090 ceph-mon[33568]: pgmap v685: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:56:20.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:56:19 smithi149 ceph-mon[33332]: pgmap v686: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:56:20.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:56:19 smithi060 ceph-mon[28873]: pgmap v686: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:56:20.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:56:19 smithi090 ceph-mon[33568]: pgmap v686: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:56:22.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:56:21 smithi149 ceph-mon[33332]: pgmap v687: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:56:22.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:56:21 smithi060 ceph-mon[28873]: pgmap v687: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:56:22.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:56:21 smithi090 ceph-mon[33568]: pgmap v687: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:56:24.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:56:23 smithi149 ceph-mon[33332]: pgmap v688: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:56:24.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:56:23 smithi060 ceph-mon[28873]: pgmap v688: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:56:24.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:56:23 smithi090 ceph-mon[33568]: pgmap v688: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:56:26.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:56:25 smithi149 ceph-mon[33332]: pgmap v689: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:56:26.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:56:25 smithi060 ceph-mon[28873]: pgmap v689: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:56:26.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:56:25 smithi090 ceph-mon[33568]: pgmap v689: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:56:27.275 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:56:26 smithi090 ceph-mon[33568]: pgmap v690: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:56:27.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:56:26 smithi060 ceph-mon[28873]: pgmap v690: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:56:27.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:56:26 smithi149 ceph-mon[33332]: pgmap v690: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:56:30.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:56:29 smithi060 ceph-mon[28873]: pgmap v691: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:56:30.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:56:29 smithi090 ceph-mon[33568]: pgmap v691: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:56:30.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:56:29 smithi149 ceph-mon[33332]: pgmap v691: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:56:32.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:56:31 smithi060 ceph-mon[28873]: pgmap v692: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:56:32.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:56:31 smithi090 ceph-mon[33568]: pgmap v692: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:56:32.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:56:31 smithi149 ceph-mon[33332]: pgmap v692: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:56:34.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:56:33 smithi060 ceph-mon[28873]: pgmap v693: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:56:34.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:56:33 smithi090 ceph-mon[33568]: pgmap v693: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:56:34.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:56:33 smithi149 ceph-mon[33332]: pgmap v693: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:56:36.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:56:35 smithi060 ceph-mon[28873]: pgmap v694: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:56:36.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:56:35 smithi090 ceph-mon[33568]: pgmap v694: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:56:36.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:56:35 smithi149 ceph-mon[33332]: pgmap v694: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:56:38.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:56:37 smithi060 ceph-mon[28873]: pgmap v695: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:56:38.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:56:37 smithi090 ceph-mon[33568]: pgmap v695: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:56:38.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:56:37 smithi149 ceph-mon[33332]: pgmap v695: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:56:40.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:56:39 smithi060 ceph-mon[28873]: pgmap v696: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:56:40.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:56:39 smithi090 ceph-mon[33568]: pgmap v696: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:56:40.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:56:39 smithi149 ceph-mon[33332]: pgmap v696: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:56:42.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:56:41 smithi060 ceph-mon[28873]: pgmap v697: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:56:42.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:56:41 smithi090 ceph-mon[33568]: pgmap v697: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:56:42.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:56:41 smithi149 ceph-mon[33332]: pgmap v697: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:56:44.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:56:43 smithi060 ceph-mon[28873]: pgmap v698: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:56:44.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:56:43 smithi090 ceph-mon[33568]: pgmap v698: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:56:44.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:56:43 smithi149 ceph-mon[33332]: pgmap v698: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:56:46.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:56:45 smithi060 ceph-mon[28873]: pgmap v699: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:56:46.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:56:45 smithi090 ceph-mon[33568]: pgmap v699: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:56:46.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:56:45 smithi149 ceph-mon[33332]: pgmap v699: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:56:48.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:56:47 smithi060 ceph-mon[28873]: pgmap v700: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:56:48.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:56:47 smithi090 ceph-mon[33568]: pgmap v700: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:56:48.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:56:47 smithi149 ceph-mon[33332]: pgmap v700: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:56:50.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:56:49 smithi060 ceph-mon[28873]: pgmap v701: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:56:50.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:56:49 smithi090 ceph-mon[33568]: pgmap v701: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:56:50.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:56:49 smithi149 ceph-mon[33332]: pgmap v701: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:56:52.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:56:51 smithi060 ceph-mon[28873]: pgmap v702: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:56:52.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:56:51 smithi090 ceph-mon[33568]: pgmap v702: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:56:52.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:56:51 smithi149 ceph-mon[33332]: pgmap v702: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:56:54.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:56:53 smithi060 ceph-mon[28873]: pgmap v703: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:56:54.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:56:53 smithi090 ceph-mon[33568]: pgmap v703: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:56:54.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:56:53 smithi149 ceph-mon[33332]: pgmap v703: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:56:55.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:56:54 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:56:55.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:56:54 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:56:55.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:56:54 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:56:56.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:56:55 smithi060 ceph-mon[28873]: pgmap v704: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:56:56.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:56:55 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:56:56.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:56:55 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:56:56.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:56:55 smithi090 ceph-mon[33568]: pgmap v704: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:56:56.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:56:55 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:56:56.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:56:55 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:56:56.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:56:55 smithi149 ceph-mon[33332]: pgmap v704: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:56:56.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:56:55 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:56:56.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:56:55 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:56:57.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:56:57 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:56:57.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:56:57 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:56:57.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:56:57 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:56:57.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:56:57 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:56:57.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:56:57 smithi149 ceph-mon[33332]: pgmap v705: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:56:57.586 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:56:57 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:56:57.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:56:57 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:56:57.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:56:57 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:56:57.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:56:57 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:56:57.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:56:57 smithi060 ceph-mon[28873]: pgmap v705: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:56:57.624 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:56:57 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:56:57.624 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:56:57 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:56:57.624 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:56:57 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:56:57.624 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:56:57 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:56:57.624 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:56:57 smithi090 ceph-mon[33568]: pgmap v705: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:56:59.624 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:56:59 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:56:59.624 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:56:59 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:56:59.624 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:56:59 smithi090 ceph-mon[33568]: pgmap v706: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:56:59.672 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:56:59 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:56:59.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:56:59 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:56:59.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:56:59 smithi149 ceph-mon[33332]: pgmap v706: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:56:59.836 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:56:59 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:56:59.836 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:56:59 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:56:59.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:56:59 smithi060 ceph-mon[28873]: pgmap v706: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:57:00.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:57:00 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:57:00.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:57:00 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:57:00.673 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:57:00 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:57:00.836 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:57:00 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:57:00.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:57:00 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:57:00.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:57:00 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:57:00.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:57:00 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:57:00.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:57:00 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:57:00.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:57:00 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:57:01.672 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:57:01 smithi149 ceph-mon[33332]: pgmap v707: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:57:01.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:57:01 smithi060 ceph-mon[28873]: pgmap v707: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:57:01.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:57:01 smithi090 ceph-mon[33568]: pgmap v707: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:57:04.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:57:03 smithi060 ceph-mon[28873]: pgmap v708: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:57:04.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:57:03 smithi090 ceph-mon[33568]: pgmap v708: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:57:04.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:57:03 smithi149 ceph-mon[33332]: pgmap v708: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:57:06.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:57:05 smithi060 ceph-mon[28873]: pgmap v709: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:57:06.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:57:05 smithi090 ceph-mon[33568]: pgmap v709: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:57:06.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:57:05 smithi149 ceph-mon[33332]: pgmap v709: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:57:08.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:57:07 smithi060 ceph-mon[28873]: pgmap v710: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:57:08.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:57:07 smithi090 ceph-mon[33568]: pgmap v710: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:57:08.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:57:07 smithi149 ceph-mon[33332]: pgmap v710: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:57:10.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:57:09 smithi060 ceph-mon[28873]: pgmap v711: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:57:10.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:57:09 smithi090 ceph-mon[33568]: pgmap v711: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:57:10.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:57:09 smithi149 ceph-mon[33332]: pgmap v711: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:57:12.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:57:11 smithi060 ceph-mon[28873]: pgmap v712: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:57:12.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:57:11 smithi090 ceph-mon[33568]: pgmap v712: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:57:12.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:57:11 smithi149 ceph-mon[33332]: pgmap v712: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:57:14.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:57:13 smithi060 ceph-mon[28873]: pgmap v713: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:57:14.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:57:13 smithi090 ceph-mon[33568]: pgmap v713: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:57:14.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:57:13 smithi149 ceph-mon[33332]: pgmap v713: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:57:16.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:57:15 smithi060 ceph-mon[28873]: pgmap v714: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:57:16.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:57:15 smithi090 ceph-mon[33568]: pgmap v714: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:57:16.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:57:15 smithi149 ceph-mon[33332]: pgmap v714: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:57:18.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:57:17 smithi060 ceph-mon[28873]: pgmap v715: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:57:18.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:57:17 smithi090 ceph-mon[33568]: pgmap v715: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:57:18.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:57:17 smithi149 ceph-mon[33332]: pgmap v715: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:57:20.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:57:19 smithi060 ceph-mon[28873]: pgmap v716: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:57:20.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:57:19 smithi090 ceph-mon[33568]: pgmap v716: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:57:20.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:57:19 smithi149 ceph-mon[33332]: pgmap v716: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:57:22.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:57:21 smithi060 ceph-mon[28873]: pgmap v717: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:57:22.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:57:21 smithi090 ceph-mon[33568]: pgmap v717: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:57:22.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:57:21 smithi149 ceph-mon[33332]: pgmap v717: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-03T14:57:24.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:57:23 smithi060 ceph-mon[28873]: pgmap v718: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:57:24.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:57:23 smithi090 ceph-mon[33568]: pgmap v718: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:57:24.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:57:23 smithi149 ceph-mon[33332]: pgmap v718: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:57:26.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:57:25 smithi060 ceph-mon[28873]: pgmap v719: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:57:26.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:57:25 smithi090 ceph-mon[33568]: pgmap v719: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:57:26.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:57:25 smithi149 ceph-mon[33332]: pgmap v719: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:57:28.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:57:27 smithi060 ceph-mon[28873]: pgmap v720: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:57:28.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:57:27 smithi090 ceph-mon[33568]: pgmap v720: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:57:28.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:57:27 smithi149 ceph-mon[33332]: pgmap v720: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:57:30.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:57:29 smithi060 ceph-mon[28873]: pgmap v721: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:57:30.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:57:29 smithi090 ceph-mon[33568]: pgmap v721: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:57:30.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:57:29 smithi149 ceph-mon[33332]: pgmap v721: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:57:32.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:57:31 smithi060 ceph-mon[28873]: pgmap v722: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:57:32.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:57:31 smithi090 ceph-mon[33568]: pgmap v722: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:57:32.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:57:31 smithi149 ceph-mon[33332]: pgmap v722: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:57:34.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:57:33 smithi060 ceph-mon[28873]: pgmap v723: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:57:34.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:57:33 smithi090 ceph-mon[33568]: pgmap v723: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:57:34.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:57:33 smithi149 ceph-mon[33332]: pgmap v723: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:57:36.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:57:35 smithi090 ceph-mon[33568]: pgmap v724: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:57:36.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:57:35 smithi149 ceph-mon[33332]: pgmap v724: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:57:36.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:57:35 smithi060 ceph-mon[28873]: pgmap v724: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:57:38.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:57:37 smithi090 ceph-mon[33568]: pgmap v725: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:57:38.139 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:57:37 smithi060 ceph-mon[28873]: pgmap v725: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:57:38.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:57:37 smithi149 ceph-mon[33332]: pgmap v725: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:57:40.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:57:39 smithi090 ceph-mon[33568]: pgmap v726: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:57:40.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:57:39 smithi149 ceph-mon[33332]: pgmap v726: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:57:40.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:57:39 smithi060 ceph-mon[28873]: pgmap v726: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:57:42.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:57:41 smithi090 ceph-mon[33568]: pgmap v727: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:57:42.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:57:41 smithi149 ceph-mon[33332]: pgmap v727: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:57:42.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:57:41 smithi060 ceph-mon[28873]: pgmap v727: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:57:44.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:57:43 smithi090 ceph-mon[33568]: pgmap v728: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:57:44.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:57:43 smithi149 ceph-mon[33332]: pgmap v728: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:57:44.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:57:43 smithi060 ceph-mon[28873]: pgmap v728: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:57:46.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:57:45 smithi090 ceph-mon[33568]: pgmap v729: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:57:46.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:57:45 smithi149 ceph-mon[33332]: pgmap v729: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:57:46.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:57:45 smithi060 ceph-mon[28873]: pgmap v729: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:57:48.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:57:47 smithi090 ceph-mon[33568]: pgmap v730: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:57:48.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:57:47 smithi149 ceph-mon[33332]: pgmap v730: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:57:48.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:57:47 smithi060 ceph-mon[28873]: pgmap v730: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:57:50.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:57:49 smithi090 ceph-mon[33568]: pgmap v731: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:57:50.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:57:49 smithi149 ceph-mon[33332]: pgmap v731: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:57:50.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:57:49 smithi060 ceph-mon[28873]: pgmap v731: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:57:52.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:57:51 smithi149 ceph-mon[33332]: pgmap v732: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:57:52.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:57:51 smithi060 ceph-mon[28873]: pgmap v732: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:57:52.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:57:51 smithi090 ceph-mon[33568]: pgmap v732: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:57:54.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:57:53 smithi149 ceph-mon[33332]: pgmap v733: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:57:54.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:57:53 smithi060 ceph-mon[28873]: pgmap v733: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:57:54.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:57:53 smithi090 ceph-mon[33568]: pgmap v733: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:57:56.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:57:55 smithi149 ceph-mon[33332]: pgmap v734: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:57:56.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:57:55 smithi060 ceph-mon[28873]: pgmap v734: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:57:56.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:57:55 smithi090 ceph-mon[33568]: pgmap v734: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:57:56.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:57:56.908+0000 7efdfd600640 1 -- 172.21.15.60:0/3449599367 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7efde0005b70 con 0x7efe0810a970 2024-05-03T14:57:56.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:57:56.909+0000 7efdfea00640 1 -- 172.21.15.60:0/3449599367 <== mon.2 v2:172.21.15.90:3300/0 9 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7efde0005b70 con 0x7efe0810a970 2024-05-03T14:57:58.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:57:57 smithi149 ceph-mon[33332]: pgmap v735: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:57:58.176 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:57:57 smithi090 ceph-mon[33568]: pgmap v735: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:57:58.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:57:57 smithi060 ceph-mon[28873]: pgmap v735: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:58:00.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:57:59 smithi149 ceph-mon[33332]: pgmap v736: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:58:00.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:57:59 smithi060 ceph-mon[28873]: pgmap v736: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:58:00.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:57:59 smithi090 ceph-mon[33568]: pgmap v736: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:58:01.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:58:00 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:58:01.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:58:00 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:58:01.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:58:00 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:58:01.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:58:00 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:58:01.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:58:00 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:58:01.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:58:00 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:58:01.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:58:00 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:58:01.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:58:00 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:58:01.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:58:00 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:58:01.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:58:00 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:58:01.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:58:00 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:58:01.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:58:00 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:58:01.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:58:00 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:58:01.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:58:00 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:58:01.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:58:00 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:58:02.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:58:01 smithi149 ceph-mon[33332]: pgmap v737: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:58:02.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:58:01 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:58:02.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:58:01 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:58:02.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:58:01 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:58:02.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:58:01 smithi060 ceph-mon[28873]: pgmap v737: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:58:02.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:58:01 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:58:02.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:58:01 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:58:02.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:58:01 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:58:02.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:58:01 smithi090 ceph-mon[33568]: pgmap v737: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:58:02.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:58:01 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:58:02.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:58:01 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:58:02.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:58:01 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:58:04.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:58:03 smithi149 ceph-mon[33332]: pgmap v738: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:58:04.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:58:03 smithi060 ceph-mon[28873]: pgmap v738: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:58:04.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:58:03 smithi090 ceph-mon[33568]: pgmap v738: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:58:06.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:58:05 smithi149 ceph-mon[33332]: pgmap v739: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:58:06.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:58:05 smithi060 ceph-mon[28873]: pgmap v739: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:58:06.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:58:05 smithi090 ceph-mon[33568]: pgmap v739: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:58:08.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:58:07 smithi149 ceph-mon[33332]: pgmap v740: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:58:08.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:58:07 smithi060 ceph-mon[28873]: pgmap v740: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:58:08.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:58:07 smithi090 ceph-mon[33568]: pgmap v740: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:58:10.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:58:09 smithi060 ceph-mon[28873]: pgmap v741: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:58:10.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:58:09 smithi090 ceph-mon[33568]: pgmap v741: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:58:10.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:58:09 smithi149 ceph-mon[33332]: pgmap v741: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:58:12.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:58:11 smithi060 ceph-mon[28873]: pgmap v742: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:58:12.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:58:11 smithi090 ceph-mon[33568]: pgmap v742: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:58:12.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:58:11 smithi149 ceph-mon[33332]: pgmap v742: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:58:14.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:58:13 smithi060 ceph-mon[28873]: pgmap v743: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:58:14.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:58:13 smithi090 ceph-mon[33568]: pgmap v743: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:58:14.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:58:13 smithi149 ceph-mon[33332]: pgmap v743: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:58:15.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:58:14 smithi060 ceph-mon[28873]: pgmap v744: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:58:15.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:58:14 smithi090 ceph-mon[33568]: pgmap v744: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:58:15.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:58:14 smithi149 ceph-mon[33332]: pgmap v744: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:58:18.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:58:17 smithi060 ceph-mon[28873]: pgmap v745: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:58:18.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:58:17 smithi090 ceph-mon[33568]: pgmap v745: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:58:18.126 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:58:17 smithi149 ceph-mon[33332]: pgmap v745: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:58:20.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:58:19 smithi060 ceph-mon[28873]: pgmap v746: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:58:20.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:58:19 smithi090 ceph-mon[33568]: pgmap v746: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:58:20.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:58:19 smithi149 ceph-mon[33332]: pgmap v746: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:58:22.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:58:21 smithi060 ceph-mon[28873]: pgmap v747: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:58:22.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:58:21 smithi090 ceph-mon[33568]: pgmap v747: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:58:22.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:58:21 smithi149 ceph-mon[33332]: pgmap v747: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:58:24.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:58:23 smithi060 ceph-mon[28873]: pgmap v748: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:58:24.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:58:23 smithi090 ceph-mon[33568]: pgmap v748: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:58:24.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:58:23 smithi149 ceph-mon[33332]: pgmap v748: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:58:26.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:58:25 smithi060 ceph-mon[28873]: pgmap v749: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:58:26.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:58:25 smithi090 ceph-mon[33568]: pgmap v749: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:58:26.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:58:25 smithi149 ceph-mon[33332]: pgmap v749: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:58:28.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:58:27 smithi060 ceph-mon[28873]: pgmap v750: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:58:28.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:58:27 smithi090 ceph-mon[33568]: pgmap v750: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:58:28.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:58:27 smithi149 ceph-mon[33332]: pgmap v750: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:58:30.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:58:29 smithi060 ceph-mon[28873]: pgmap v751: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:58:30.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:58:29 smithi090 ceph-mon[33568]: pgmap v751: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:58:30.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:58:29 smithi149 ceph-mon[33332]: pgmap v751: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:58:32.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:58:31 smithi060 ceph-mon[28873]: pgmap v752: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:58:32.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:58:31 smithi090 ceph-mon[33568]: pgmap v752: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:58:32.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:58:31 smithi149 ceph-mon[33332]: pgmap v752: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:58:34.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:58:33 smithi060 ceph-mon[28873]: pgmap v753: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:58:34.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:58:33 smithi090 ceph-mon[33568]: pgmap v753: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:58:34.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:58:33 smithi149 ceph-mon[33332]: pgmap v753: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:58:36.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:58:35 smithi090 ceph-mon[33568]: pgmap v754: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:58:36.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:58:35 smithi149 ceph-mon[33332]: pgmap v754: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:58:36.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:58:35 smithi060 ceph-mon[28873]: pgmap v754: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:58:38.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:58:37 smithi090 ceph-mon[33568]: pgmap v755: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:58:38.139 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:58:37 smithi060 ceph-mon[28873]: pgmap v755: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:58:38.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:58:37 smithi149 ceph-mon[33332]: pgmap v755: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:58:40.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:58:39 smithi090 ceph-mon[33568]: pgmap v756: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:58:40.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:58:39 smithi149 ceph-mon[33332]: pgmap v756: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:58:40.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:58:39 smithi060 ceph-mon[28873]: pgmap v756: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:58:41.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:58:41.915+0000 7efdfd600640 1 -- 172.21.15.60:0/3449599367 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7efde0005ed0 con 0x7efe0810a970 2024-05-03T14:58:41.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T14:58:41.916+0000 7efdfea00640 1 -- 172.21.15.60:0/3449599367 <== mon.2 v2:172.21.15.90:3300/0 10 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7efde0005ed0 con 0x7efe0810a970 2024-05-03T14:58:42.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:58:41 smithi090 ceph-mon[33568]: pgmap v757: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:58:42.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:58:41 smithi149 ceph-mon[33332]: pgmap v757: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:58:42.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:58:41 smithi060 ceph-mon[28873]: pgmap v757: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:58:44.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:58:43 smithi090 ceph-mon[33568]: pgmap v758: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:58:44.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:58:43 smithi149 ceph-mon[33332]: pgmap v758: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:58:44.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:58:43 smithi060 ceph-mon[28873]: pgmap v758: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:58:46.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:58:45 smithi149 ceph-mon[33332]: pgmap v759: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:58:46.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:58:45 smithi060 ceph-mon[28873]: pgmap v759: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:58:46.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:58:45 smithi090 ceph-mon[33568]: pgmap v759: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:58:48.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:58:47 smithi149 ceph-mon[33332]: pgmap v760: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:58:48.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:58:47 smithi060 ceph-mon[28873]: pgmap v760: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:58:48.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:58:47 smithi090 ceph-mon[33568]: pgmap v760: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:58:50.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:58:49 smithi149 ceph-mon[33332]: pgmap v761: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:58:50.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:58:49 smithi060 ceph-mon[28873]: pgmap v761: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:58:50.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:58:49 smithi090 ceph-mon[33568]: pgmap v761: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:58:52.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:58:51 smithi149 ceph-mon[33332]: pgmap v762: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:58:52.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:58:51 smithi060 ceph-mon[28873]: pgmap v762: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:58:52.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:58:51 smithi090 ceph-mon[33568]: pgmap v762: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:58:54.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:58:53 smithi149 ceph-mon[33332]: pgmap v763: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:58:54.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:58:53 smithi060 ceph-mon[28873]: pgmap v763: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:58:54.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:58:53 smithi090 ceph-mon[33568]: pgmap v763: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:58:56.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:58:55 smithi149 ceph-mon[33332]: pgmap v764: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:58:56.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:58:55 smithi060 ceph-mon[28873]: pgmap v764: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:58:56.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:58:55 smithi090 ceph-mon[33568]: pgmap v764: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:58:57.672 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:58:57 smithi149 ceph-mon[33332]: pgmap v765: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:58:57.836 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:58:57 smithi060 ceph-mon[28873]: pgmap v765: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:58:57.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:58:57 smithi090 ceph-mon[33568]: pgmap v765: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:59:00.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:58:59 smithi060 ceph-mon[28873]: pgmap v766: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:59:00.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:58:59 smithi090 ceph-mon[33568]: pgmap v766: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:59:00.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:58:59 smithi149 ceph-mon[33332]: pgmap v766: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:59:02.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:59:01 smithi060 ceph-mon[28873]: pgmap v767: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:59:02.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:59:01 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:59:02.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:59:01 smithi090 ceph-mon[33568]: pgmap v767: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:59:02.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:59:01 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:59:02.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:59:01 smithi149 ceph-mon[33332]: pgmap v767: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:59:02.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:59:01 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T14:59:04.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:59:03 smithi060 ceph-mon[28873]: pgmap v768: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:59:04.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:59:03 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:59:04.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:59:03 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:59:04.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:59:03 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:59:04.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:59:03 smithi090 ceph-mon[33568]: pgmap v768: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:59:04.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:59:03 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:59:04.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:59:03 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:59:04.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:59:03 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:59:04.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:59:03 smithi149 ceph-mon[33332]: pgmap v768: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:59:04.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:59:03 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T14:59:04.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:59:03 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T14:59:04.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:59:03 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T14:59:06.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:59:05 smithi060 ceph-mon[28873]: pgmap v769: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:59:06.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:59:05 smithi090 ceph-mon[33568]: pgmap v769: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:59:06.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:59:05 smithi149 ceph-mon[33332]: pgmap v769: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:59:08.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:59:07 smithi090 ceph-mon[33568]: pgmap v770: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:59:08.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:59:07 smithi149 ceph-mon[33332]: pgmap v770: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:59:08.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:59:07 smithi060 ceph-mon[28873]: pgmap v770: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:59:10.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:59:09 smithi090 ceph-mon[33568]: pgmap v771: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:59:10.139 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:59:09 smithi060 ceph-mon[28873]: pgmap v771: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:59:10.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:59:09 smithi149 ceph-mon[33332]: pgmap v771: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:59:12.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:59:11 smithi090 ceph-mon[33568]: pgmap v772: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:59:12.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:59:11 smithi149 ceph-mon[33332]: pgmap v772: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:59:12.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:59:11 smithi060 ceph-mon[28873]: pgmap v772: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:59:14.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:59:13 smithi090 ceph-mon[33568]: pgmap v773: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:59:14.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:59:13 smithi149 ceph-mon[33332]: pgmap v773: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:59:14.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:59:13 smithi060 ceph-mon[28873]: pgmap v773: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:59:16.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:59:15 smithi090 ceph-mon[33568]: pgmap v774: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:59:16.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:59:15 smithi149 ceph-mon[33332]: pgmap v774: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:59:16.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:59:15 smithi060 ceph-mon[28873]: pgmap v774: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:59:18.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:59:17 smithi090 ceph-mon[33568]: pgmap v775: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:59:18.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:59:17 smithi149 ceph-mon[33332]: pgmap v775: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:59:18.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:59:17 smithi060 ceph-mon[28873]: pgmap v775: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:59:20.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:59:19 smithi090 ceph-mon[33568]: pgmap v776: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:59:20.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:59:19 smithi149 ceph-mon[33332]: pgmap v776: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:59:20.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:59:19 smithi060 ceph-mon[28873]: pgmap v776: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:59:22.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:59:21 smithi090 ceph-mon[33568]: pgmap v777: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:59:22.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:59:21 smithi149 ceph-mon[33332]: pgmap v777: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:59:22.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:59:21 smithi060 ceph-mon[28873]: pgmap v777: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:59:24.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:59:23 smithi149 ceph-mon[33332]: pgmap v778: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:59:24.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:59:23 smithi060 ceph-mon[28873]: pgmap v778: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:59:24.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:59:23 smithi090 ceph-mon[33568]: pgmap v778: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:59:26.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:59:25 smithi149 ceph-mon[33332]: pgmap v779: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:59:26.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:59:25 smithi060 ceph-mon[28873]: pgmap v779: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:59:26.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:59:25 smithi090 ceph-mon[33568]: pgmap v779: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:59:28.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:59:27 smithi149 ceph-mon[33332]: pgmap v780: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:59:28.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:59:27 smithi060 ceph-mon[28873]: pgmap v780: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:59:28.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:59:27 smithi090 ceph-mon[33568]: pgmap v780: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:59:30.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:59:29 smithi149 ceph-mon[33332]: pgmap v781: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:59:30.176 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:59:29 smithi090 ceph-mon[33568]: pgmap v781: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:59:30.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:59:29 smithi060 ceph-mon[28873]: pgmap v781: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:59:32.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:59:31 smithi149 ceph-mon[33332]: pgmap v782: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:59:32.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:59:31 smithi060 ceph-mon[28873]: pgmap v782: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:59:32.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:59:31 smithi090 ceph-mon[33568]: pgmap v782: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:59:34.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:59:33 smithi149 ceph-mon[33332]: pgmap v783: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:59:34.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:59:33 smithi060 ceph-mon[28873]: pgmap v783: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:59:34.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:59:33 smithi090 ceph-mon[33568]: pgmap v783: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:59:36.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:59:35 smithi149 ceph-mon[33332]: pgmap v784: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:59:36.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:59:35 smithi060 ceph-mon[28873]: pgmap v784: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:59:36.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:59:35 smithi090 ceph-mon[33568]: pgmap v784: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:59:38.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:59:37 smithi149 ceph-mon[33332]: pgmap v785: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:59:38.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:59:37 smithi060 ceph-mon[28873]: pgmap v785: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:59:38.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:59:37 smithi090 ceph-mon[33568]: pgmap v785: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:59:40.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:59:39 smithi149 ceph-mon[33332]: pgmap v786: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:59:40.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:59:39 smithi060 ceph-mon[28873]: pgmap v786: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:59:40.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:59:39 smithi090 ceph-mon[33568]: pgmap v786: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:59:42.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:59:41 smithi060 ceph-mon[28873]: pgmap v787: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:59:42.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:59:41 smithi090 ceph-mon[33568]: pgmap v787: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:59:42.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:59:41 smithi149 ceph-mon[33332]: pgmap v787: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:59:44.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:59:43 smithi060 ceph-mon[28873]: pgmap v788: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:59:44.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:59:43 smithi090 ceph-mon[33568]: pgmap v788: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:59:44.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:59:43 smithi149 ceph-mon[33332]: pgmap v788: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:59:46.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:59:45 smithi060 ceph-mon[28873]: pgmap v789: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:59:46.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:59:45 smithi090 ceph-mon[33568]: pgmap v789: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:59:46.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:59:45 smithi149 ceph-mon[33332]: pgmap v789: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:59:48.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:59:47 smithi060 ceph-mon[28873]: pgmap v790: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:59:48.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:59:47 smithi090 ceph-mon[33568]: pgmap v790: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:59:48.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:59:47 smithi149 ceph-mon[33332]: pgmap v790: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:59:50.225 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:59:49 smithi149 ceph-mon[33332]: pgmap v791: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:59:50.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:59:49 smithi060 ceph-mon[28873]: pgmap v791: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:59:50.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:59:49 smithi090 ceph-mon[33568]: pgmap v791: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T14:59:52.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:59:51 smithi060 ceph-mon[28873]: pgmap v792: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:59:52.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:59:51 smithi090 ceph-mon[33568]: pgmap v792: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:59:52.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:59:51 smithi149 ceph-mon[33332]: pgmap v792: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T14:59:54.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:59:53 smithi060 ceph-mon[28873]: pgmap v793: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:59:54.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:59:53 smithi090 ceph-mon[33568]: pgmap v793: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:59:54.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:59:53 smithi149 ceph-mon[33332]: pgmap v793: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T14:59:55.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:59:55 smithi060 ceph-mon[28873]: pgmap v794: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:59:55.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:59:55 smithi090 ceph-mon[33568]: pgmap v794: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:59:55.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:59:55 smithi149 ceph-mon[33332]: pgmap v794: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T14:59:58.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:59:57 smithi090 ceph-mon[33568]: pgmap v795: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:59:58.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:59:57 smithi149 ceph-mon[33332]: pgmap v795: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T14:59:58.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:59:57 smithi060 ceph-mon[28873]: pgmap v795: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:00:00.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 14:59:59 smithi090 ceph-mon[33568]: pgmap v796: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:00:00.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 14:59:59 smithi149 ceph-mon[33332]: pgmap v796: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:00:00.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 14:59:59 smithi060 ceph-mon[28873]: pgmap v796: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:00:01.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:00:00 smithi090 ceph-mon[33568]: overall HEALTH_OK 2024-05-03T15:00:01.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:00:00 smithi149 ceph-mon[33332]: overall HEALTH_OK 2024-05-03T15:00:01.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:00:00 smithi060 ceph-mon[28873]: overall HEALTH_OK 2024-05-03T15:00:02.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:00:01 smithi090 ceph-mon[33568]: pgmap v797: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:00:02.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:00:01 smithi149 ceph-mon[33332]: pgmap v797: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:00:02.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:00:01 smithi060 ceph-mon[28873]: pgmap v797: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:00:04.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:00:03 smithi090 ceph-mon[33568]: pgmap v798: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:00:04.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:00:03 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T15:00:04.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:00:03 smithi149 ceph-mon[33332]: pgmap v798: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:00:04.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:00:03 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T15:00:04.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:00:03 smithi060 ceph-mon[28873]: pgmap v798: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:00:04.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:00:03 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T15:00:05.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:00:04 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T15:00:05.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:00:04 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T15:00:05.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:00:04 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T15:00:05.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:00:04 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T15:00:05.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:00:04 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T15:00:05.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:00:04 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T15:00:05.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:00:04 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T15:00:05.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:00:04 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T15:00:05.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:00:04 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T15:00:06.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:00:05 smithi149 ceph-mon[33332]: pgmap v799: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:00:06.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:00:05 smithi060 ceph-mon[28873]: pgmap v799: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:00:06.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:00:05 smithi090 ceph-mon[33568]: pgmap v799: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:00:08.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:00:07 smithi149 ceph-mon[33332]: pgmap v800: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:00:08.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:00:07 smithi060 ceph-mon[28873]: pgmap v800: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:00:08.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:00:07 smithi090 ceph-mon[33568]: pgmap v800: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:00:10.140 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:00:09 smithi060 ceph-mon[28873]: pgmap v801: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:00:10.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:00:09 smithi149 ceph-mon[33332]: pgmap v801: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:00:10.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:00:09 smithi090 ceph-mon[33568]: pgmap v801: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:00:12.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:00:11 smithi149 ceph-mon[33332]: pgmap v802: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:00:12.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:00:11 smithi060 ceph-mon[28873]: pgmap v802: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:00:12.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:00:11 smithi090 ceph-mon[33568]: pgmap v802: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:00:14.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:00:13 smithi149 ceph-mon[33332]: pgmap v803: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:00:14.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:00:13 smithi060 ceph-mon[28873]: pgmap v803: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:00:14.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:00:13 smithi090 ceph-mon[33568]: pgmap v803: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:00:16.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:00:15 smithi149 ceph-mon[33332]: pgmap v804: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:00:16.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:00:15 smithi060 ceph-mon[28873]: pgmap v804: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:00:16.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:00:15 smithi090 ceph-mon[33568]: pgmap v804: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:00:18.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:00:17 smithi149 ceph-mon[33332]: pgmap v805: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:00:18.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:00:17 smithi060 ceph-mon[28873]: pgmap v805: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:00:18.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:00:17 smithi090 ceph-mon[33568]: pgmap v805: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:00:20.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:00:19 smithi149 ceph-mon[33332]: pgmap v806: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:00:20.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:00:19 smithi060 ceph-mon[28873]: pgmap v806: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:00:20.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:00:19 smithi090 ceph-mon[33568]: pgmap v806: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:00:22.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:00:21 smithi149 ceph-mon[33332]: pgmap v807: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:00:22.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:00:21 smithi060 ceph-mon[28873]: pgmap v807: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:00:22.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:00:21 smithi090 ceph-mon[33568]: pgmap v807: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:00:24.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:00:23 smithi060 ceph-mon[28873]: pgmap v808: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:00:24.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:00:23 smithi090 ceph-mon[33568]: pgmap v808: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:00:24.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:00:23 smithi149 ceph-mon[33332]: pgmap v808: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:00:26.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:00:25 smithi060 ceph-mon[28873]: pgmap v809: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:00:26.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:00:25 smithi090 ceph-mon[33568]: pgmap v809: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:00:26.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:00:25 smithi149 ceph-mon[33332]: pgmap v809: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:00:28.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:00:27 smithi060 ceph-mon[28873]: pgmap v810: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:00:28.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:00:27 smithi090 ceph-mon[33568]: pgmap v810: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:00:28.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:00:27 smithi149 ceph-mon[33332]: pgmap v810: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:00:30.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:00:29 smithi060 ceph-mon[28873]: pgmap v811: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:00:30.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:00:29 smithi090 ceph-mon[33568]: pgmap v811: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:00:30.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:00:29 smithi149 ceph-mon[33332]: pgmap v811: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:00:32.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:00:31 smithi060 ceph-mon[28873]: pgmap v812: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:00:32.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:00:31 smithi090 ceph-mon[33568]: pgmap v812: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:00:32.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:00:31 smithi149 ceph-mon[33332]: pgmap v812: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:00:34.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:00:33 smithi060 ceph-mon[28873]: pgmap v813: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:00:34.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:00:33 smithi090 ceph-mon[33568]: pgmap v813: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:00:34.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:00:33 smithi149 ceph-mon[33332]: pgmap v813: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:00:36.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:00:35 smithi060 ceph-mon[28873]: pgmap v814: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:00:36.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:00:35 smithi090 ceph-mon[33568]: pgmap v814: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:00:36.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:00:35 smithi149 ceph-mon[33332]: pgmap v814: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:00:38.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:00:37 smithi060 ceph-mon[28873]: pgmap v815: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:00:38.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:00:37 smithi090 ceph-mon[33568]: pgmap v815: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:00:38.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:00:37 smithi149 ceph-mon[33332]: pgmap v815: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:00:40.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:00:39 smithi060 ceph-mon[28873]: pgmap v816: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:00:40.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:00:39 smithi090 ceph-mon[33568]: pgmap v816: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:00:40.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:00:39 smithi149 ceph-mon[33332]: pgmap v816: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:00:41.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:00:40 smithi060 ceph-mon[28873]: pgmap v817: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:00:41.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:00:40 smithi090 ceph-mon[33568]: pgmap v817: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:00:41.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:00:40 smithi149 ceph-mon[33332]: pgmap v817: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:00:44.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:00:43 smithi149 ceph-mon[33332]: pgmap v818: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:00:44.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:00:43 smithi060 ceph-mon[28873]: pgmap v818: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:00:44.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:00:43 smithi090 ceph-mon[33568]: pgmap v818: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:00:46.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:00:45 smithi149 ceph-mon[33332]: pgmap v819: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:00:46.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:00:45 smithi060 ceph-mon[28873]: pgmap v819: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:00:46.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:00:45 smithi090 ceph-mon[33568]: pgmap v819: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:00:48.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:00:47 smithi149 ceph-mon[33332]: pgmap v820: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:00:48.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:00:47 smithi060 ceph-mon[28873]: pgmap v820: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:00:48.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:00:47 smithi090 ceph-mon[33568]: pgmap v820: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:00:50.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:00:49 smithi149 ceph-mon[33332]: pgmap v821: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:00:50.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:00:49 smithi060 ceph-mon[28873]: pgmap v821: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:00:50.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:00:49 smithi090 ceph-mon[33568]: pgmap v821: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:00:52.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:00:51 smithi149 ceph-mon[33332]: pgmap v822: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:00:52.262 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:00:51 smithi060 ceph-mon[28873]: pgmap v822: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:00:52.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:00:51 smithi090 ceph-mon[33568]: pgmap v822: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:00:54.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:00:53 smithi149 ceph-mon[33332]: pgmap v823: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:00:54.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:00:53 smithi060 ceph-mon[28873]: pgmap v823: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:00:54.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:00:53 smithi090 ceph-mon[33568]: pgmap v823: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:00:56.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:00:55 smithi149 ceph-mon[33332]: pgmap v824: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:00:56.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:00:55 smithi060 ceph-mon[28873]: pgmap v824: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:00:56.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:00:55 smithi090 ceph-mon[33568]: pgmap v824: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:00:58.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:00:57 smithi149 ceph-mon[33332]: pgmap v825: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:00:58.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:00:57 smithi060 ceph-mon[28873]: pgmap v825: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:00:58.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:00:57 smithi090 ceph-mon[33568]: pgmap v825: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:01:00.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:00:59 smithi149 ceph-mon[33332]: pgmap v826: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:01:00.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:00:59 smithi060 ceph-mon[28873]: pgmap v826: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:01:00.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:00:59 smithi090 ceph-mon[33568]: pgmap v826: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:01:02.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:01:01 smithi149 ceph-mon[33332]: pgmap v827: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:01:02.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:01:01 smithi060 ceph-mon[28873]: pgmap v827: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:01:02.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:01:01 smithi090 ceph-mon[33568]: pgmap v827: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:01:04.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:01:03 smithi060 ceph-mon[28873]: pgmap v828: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:01:04.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:01:03 smithi090 ceph-mon[33568]: pgmap v828: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:01:04.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:01:03 smithi149 ceph-mon[33332]: pgmap v828: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:01:06.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:01:05 smithi060 ceph-mon[28873]: pgmap v829: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:01:06.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:01:05 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T15:01:06.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:01:05 smithi090 ceph-mon[33568]: pgmap v829: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:01:06.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:01:05 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T15:01:06.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:01:05 smithi149 ceph-mon[33332]: pgmap v829: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:01:06.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:01:05 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T15:01:07.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:01:06 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T15:01:07.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:01:06 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T15:01:07.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:01:06 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T15:01:07.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:01:06 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T15:01:07.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:01:06 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T15:01:07.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:01:06 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T15:01:07.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:01:06 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T15:01:07.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:01:06 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T15:01:07.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:01:06 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T15:01:08.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:01:07 smithi060 ceph-mon[28873]: pgmap v830: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:01:08.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:01:07 smithi090 ceph-mon[33568]: pgmap v830: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:01:08.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:01:07 smithi149 ceph-mon[33332]: pgmap v830: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:01:10.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:01:09 smithi060 ceph-mon[28873]: pgmap v831: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:01:10.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:01:09 smithi090 ceph-mon[33568]: pgmap v831: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:01:10.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:01:09 smithi149 ceph-mon[33332]: pgmap v831: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:01:12.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:01:11 smithi060 ceph-mon[28873]: pgmap v832: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:01:12.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:01:11 smithi090 ceph-mon[33568]: pgmap v832: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:01:12.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:01:11 smithi149 ceph-mon[33332]: pgmap v832: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:01:14.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:01:13 smithi060 ceph-mon[28873]: pgmap v833: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:01:14.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:01:13 smithi090 ceph-mon[33568]: pgmap v833: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:01:14.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:01:13 smithi149 ceph-mon[33332]: pgmap v833: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:01:16.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:01:15 smithi060 ceph-mon[28873]: pgmap v834: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:01:16.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:01:15 smithi090 ceph-mon[33568]: pgmap v834: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:01:16.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:01:15 smithi149 ceph-mon[33332]: pgmap v834: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:01:18.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:01:17 smithi060 ceph-mon[28873]: pgmap v835: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:01:18.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:01:17 smithi090 ceph-mon[33568]: pgmap v835: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:01:18.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:01:17 smithi149 ceph-mon[33332]: pgmap v835: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:01:20.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:01:19 smithi060 ceph-mon[28873]: pgmap v836: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:01:20.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:01:19 smithi090 ceph-mon[33568]: pgmap v836: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:01:20.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:01:19 smithi149 ceph-mon[33332]: pgmap v836: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:01:22.225 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:01:21 smithi149 ceph-mon[33332]: pgmap v837: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:01:22.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:01:21 smithi060 ceph-mon[28873]: pgmap v837: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:01:22.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:01:21 smithi090 ceph-mon[33568]: pgmap v837: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:01:23.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:01:23 smithi060 ceph-mon[28873]: pgmap v838: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:01:23.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:01:23 smithi090 ceph-mon[33568]: pgmap v838: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:01:23.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:01:23 smithi149 ceph-mon[33332]: pgmap v838: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:01:26.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:01:25 smithi149 ceph-mon[33332]: pgmap v839: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:01:26.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:01:25 smithi060 ceph-mon[28873]: pgmap v839: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:01:26.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:01:25 smithi090 ceph-mon[33568]: pgmap v839: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:01:28.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:01:27 smithi149 ceph-mon[33332]: pgmap v840: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:01:28.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:01:27 smithi060 ceph-mon[28873]: pgmap v840: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:01:28.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:01:27 smithi090 ceph-mon[33568]: pgmap v840: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:01:30.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:01:29 smithi149 ceph-mon[33332]: pgmap v841: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:01:30.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:01:29 smithi060 ceph-mon[28873]: pgmap v841: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:01:30.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:01:29 smithi090 ceph-mon[33568]: pgmap v841: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:01:32.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:01:31 smithi149 ceph-mon[33332]: pgmap v842: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:01:32.176 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:01:31 smithi090 ceph-mon[33568]: pgmap v842: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:01:32.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:01:31 smithi060 ceph-mon[28873]: pgmap v842: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:01:34.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:01:33 smithi149 ceph-mon[33332]: pgmap v843: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:01:34.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:01:33 smithi060 ceph-mon[28873]: pgmap v843: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:01:34.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:01:33 smithi090 ceph-mon[33568]: pgmap v843: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:01:36.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:01:35 smithi060 ceph-mon[28873]: pgmap v844: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:01:36.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:01:35 smithi090 ceph-mon[33568]: pgmap v844: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:01:36.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:01:35 smithi149 ceph-mon[33332]: pgmap v844: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:01:37.836 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:01:37 smithi060 ceph-mon[28873]: pgmap v845: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:01:37.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:01:37 smithi090 ceph-mon[33568]: pgmap v845: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:01:37.922 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:01:37 smithi149 ceph-mon[33332]: pgmap v845: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:01:40.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:01:39 smithi149 ceph-mon[33332]: pgmap v846: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:01:40.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:01:39 smithi060 ceph-mon[28873]: pgmap v846: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:01:40.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:01:39 smithi090 ceph-mon[33568]: pgmap v846: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:01:41.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:01:41.946+0000 7efdfd600640 1 -- 172.21.15.60:0/3449599367 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7efde0006130 con 0x7efe0810a970 2024-05-03T15:01:41.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:01:41.947+0000 7efdfea00640 1 -- 172.21.15.60:0/3449599367 <== mon.2 v2:172.21.15.90:3300/0 11 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7efde0006130 con 0x7efe0810a970 2024-05-03T15:01:42.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:01:41 smithi149 ceph-mon[33332]: pgmap v847: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:01:42.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:01:41 smithi060 ceph-mon[28873]: pgmap v847: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:01:42.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:01:41 smithi090 ceph-mon[33568]: pgmap v847: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:01:44.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:01:43 smithi149 ceph-mon[33332]: pgmap v848: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:01:44.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:01:43 smithi060 ceph-mon[28873]: pgmap v848: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:01:44.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:01:43 smithi090 ceph-mon[33568]: pgmap v848: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:01:46.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:01:45 smithi149 ceph-mon[33332]: pgmap v849: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:01:46.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:01:45 smithi060 ceph-mon[28873]: pgmap v849: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:01:46.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:01:45 smithi090 ceph-mon[33568]: pgmap v849: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:01:48.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:01:47 smithi060 ceph-mon[28873]: pgmap v850: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:01:48.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:01:47 smithi090 ceph-mon[33568]: pgmap v850: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:01:48.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:01:47 smithi149 ceph-mon[33332]: pgmap v850: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:01:50.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:01:49 smithi060 ceph-mon[28873]: pgmap v851: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:01:50.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:01:49 smithi090 ceph-mon[33568]: pgmap v851: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:01:50.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:01:49 smithi149 ceph-mon[33332]: pgmap v851: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:01:52.225 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:01:51 smithi149 ceph-mon[33332]: pgmap v852: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:01:52.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:01:51 smithi060 ceph-mon[28873]: pgmap v852: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:01:52.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:01:51 smithi090 ceph-mon[33568]: pgmap v852: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:01:54.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:01:53 smithi060 ceph-mon[28873]: pgmap v853: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:01:54.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:01:53 smithi090 ceph-mon[33568]: pgmap v853: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:01:54.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:01:53 smithi149 ceph-mon[33332]: pgmap v853: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:01:56.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:01:55 smithi060 ceph-mon[28873]: pgmap v854: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:01:56.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:01:55 smithi090 ceph-mon[33568]: pgmap v854: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:01:56.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:01:55 smithi149 ceph-mon[33332]: pgmap v854: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:01:58.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:01:57 smithi060 ceph-mon[28873]: pgmap v855: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:01:58.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:01:57 smithi090 ceph-mon[33568]: pgmap v855: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:01:58.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:01:57 smithi149 ceph-mon[33332]: pgmap v855: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:02:00.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:01:59 smithi060 ceph-mon[28873]: pgmap v856: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:02:00.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:01:59 smithi090 ceph-mon[33568]: pgmap v856: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:02:00.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:01:59 smithi149 ceph-mon[33332]: pgmap v856: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:02:02.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:02:01 smithi060 ceph-mon[28873]: pgmap v857: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:02:02.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:02:01 smithi090 ceph-mon[33568]: pgmap v857: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:02:02.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:02:01 smithi149 ceph-mon[33332]: pgmap v857: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:02:04.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:02:03 smithi060 ceph-mon[28873]: pgmap v858: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:02:04.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:02:03 smithi090 ceph-mon[33568]: pgmap v858: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:02:04.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:02:03 smithi149 ceph-mon[33332]: pgmap v858: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:02:06.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:02:05 smithi060 ceph-mon[28873]: pgmap v859: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:02:06.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:02:05 smithi090 ceph-mon[33568]: pgmap v859: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:02:06.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:02:05 smithi149 ceph-mon[33332]: pgmap v859: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:02:07.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:02:06 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T15:02:07.373 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:02:06 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T15:02:07.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:02:06 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T15:02:08.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:02:07 smithi060 ceph-mon[28873]: pgmap v860: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:02:08.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:02:07 smithi090 ceph-mon[33568]: pgmap v860: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:02:08.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:02:07 smithi149 ceph-mon[33332]: pgmap v860: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:02:09.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:02:08 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T15:02:09.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:02:08 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T15:02:09.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:02:08 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T15:02:09.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:02:08 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T15:02:09.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:02:08 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T15:02:09.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:02:08 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T15:02:09.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:02:08 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T15:02:09.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:02:08 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T15:02:09.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:02:08 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T15:02:10.237 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:02:09 smithi060 ceph-mon[28873]: pgmap v861: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-03T15:02:10.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:02:09 smithi090 ceph-mon[33568]: pgmap v861: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-03T15:02:10.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:02:09 smithi149 ceph-mon[33332]: pgmap v861: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-03T15:02:11.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:02:11 smithi060 ceph-mon[28873]: pgmap v862: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-03T15:02:11.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:02:11 smithi090 ceph-mon[33568]: pgmap v862: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-03T15:02:11.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:02:11 smithi149 ceph-mon[33332]: pgmap v862: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-03T15:02:14.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:02:13 smithi060 ceph-mon[28873]: pgmap v863: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-05-03T15:02:14.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:02:13 smithi090 ceph-mon[33568]: pgmap v863: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-05-03T15:02:14.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:02:13 smithi149 ceph-mon[33332]: pgmap v863: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-05-03T15:02:16.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:02:15 smithi060 ceph-mon[28873]: pgmap v864: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-05-03T15:02:16.373 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:02:15 smithi090 ceph-mon[33568]: pgmap v864: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-05-03T15:02:16.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:02:15 smithi149 ceph-mon[33332]: pgmap v864: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-05-03T15:02:18.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:02:17 smithi060 ceph-mon[28873]: pgmap v865: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-03T15:02:18.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:02:17 smithi090 ceph-mon[33568]: pgmap v865: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-03T15:02:18.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:02:17 smithi149 ceph-mon[33332]: pgmap v865: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-03T15:02:20.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:02:19 smithi060 ceph-mon[28873]: pgmap v866: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-03T15:02:20.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:02:19 smithi090 ceph-mon[33568]: pgmap v866: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-03T15:02:20.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:02:19 smithi149 ceph-mon[33332]: pgmap v866: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-03T15:02:22.225 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:02:21 smithi149 ceph-mon[33332]: pgmap v867: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:02:22.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:02:21 smithi060 ceph-mon[28873]: pgmap v867: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:02:22.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:02:21 smithi090 ceph-mon[33568]: pgmap v867: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:02:24.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:02:23 smithi060 ceph-mon[28873]: pgmap v868: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:02:24.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:02:23 smithi090 ceph-mon[33568]: pgmap v868: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:02:24.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:02:23 smithi149 ceph-mon[33332]: pgmap v868: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:02:26.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:02:25 smithi060 ceph-mon[28873]: pgmap v869: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:02:26.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:02:25 smithi090 ceph-mon[33568]: pgmap v869: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:02:26.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:02:25 smithi149 ceph-mon[33332]: pgmap v869: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:02:28.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:02:27 smithi060 ceph-mon[28873]: pgmap v870: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:02:28.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:02:27 smithi090 ceph-mon[33568]: pgmap v870: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:02:28.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:02:27 smithi149 ceph-mon[33332]: pgmap v870: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:02:30.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:02:29 smithi060 ceph-mon[28873]: pgmap v871: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:02:30.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:02:29 smithi090 ceph-mon[33568]: pgmap v871: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:02:30.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:02:29 smithi149 ceph-mon[33332]: pgmap v871: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:02:32.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:02:31 smithi060 ceph-mon[28873]: pgmap v872: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:02:32.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:02:31 smithi090 ceph-mon[33568]: pgmap v872: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:02:32.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:02:31 smithi149 ceph-mon[33332]: pgmap v872: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:02:34.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:02:33 smithi060 ceph-mon[28873]: pgmap v873: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:02:34.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:02:33 smithi090 ceph-mon[33568]: pgmap v873: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:02:34.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:02:33 smithi149 ceph-mon[33332]: pgmap v873: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:02:36.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:02:35 smithi060 ceph-mon[28873]: pgmap v874: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:02:36.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:02:35 smithi090 ceph-mon[33568]: pgmap v874: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:02:36.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:02:35 smithi149 ceph-mon[33332]: pgmap v874: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:02:38.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:02:37 smithi060 ceph-mon[28873]: pgmap v875: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:02:38.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:02:37 smithi090 ceph-mon[33568]: pgmap v875: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:02:38.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:02:37 smithi149 ceph-mon[33332]: pgmap v875: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:02:39.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:02:39 smithi060 ceph-mon[28873]: pgmap v876: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:02:39.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:02:39 smithi090 ceph-mon[33568]: pgmap v876: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:02:39.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:02:39 smithi149 ceph-mon[33332]: pgmap v876: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:02:42.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:02:41 smithi060 ceph-mon[28873]: pgmap v877: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:02:42.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:02:41 smithi090 ceph-mon[33568]: pgmap v877: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:02:42.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:02:41 smithi149 ceph-mon[33332]: pgmap v877: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:02:44.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:02:43 smithi060 ceph-mon[28873]: pgmap v878: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:02:44.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:02:43 smithi090 ceph-mon[33568]: pgmap v878: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:02:44.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:02:43 smithi149 ceph-mon[33332]: pgmap v878: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:02:46.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:02:45 smithi060 ceph-mon[28873]: pgmap v879: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:02:46.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:02:45 smithi090 ceph-mon[33568]: pgmap v879: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:02:46.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:02:45 smithi149 ceph-mon[33332]: pgmap v879: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:02:48.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:02:47 smithi060 ceph-mon[28873]: pgmap v880: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:02:48.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:02:47 smithi090 ceph-mon[33568]: pgmap v880: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:02:48.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:02:47 smithi149 ceph-mon[33332]: pgmap v880: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:02:50.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:02:49 smithi060 ceph-mon[28873]: pgmap v881: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:02:50.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:02:49 smithi090 ceph-mon[33568]: pgmap v881: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:02:50.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:02:49 smithi149 ceph-mon[33332]: pgmap v881: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:02:52.225 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:02:51 smithi149 ceph-mon[33332]: pgmap v882: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:02:52.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:02:51 smithi060 ceph-mon[28873]: pgmap v882: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:02:52.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:02:51 smithi090 ceph-mon[33568]: pgmap v882: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:02:54.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:02:53 smithi060 ceph-mon[28873]: pgmap v883: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:02:54.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:02:53 smithi090 ceph-mon[33568]: pgmap v883: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:02:54.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:02:53 smithi149 ceph-mon[33332]: pgmap v883: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:02:56.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:02:55 smithi060 ceph-mon[28873]: pgmap v884: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:02:56.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:02:55 smithi090 ceph-mon[33568]: pgmap v884: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:02:56.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:02:55 smithi149 ceph-mon[33332]: pgmap v884: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:02:57.836 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:02:57 smithi060 ceph-mon[28873]: pgmap v885: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:02:57.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:02:57 smithi090 ceph-mon[33568]: pgmap v885: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:02:57.922 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:02:57 smithi149 ceph-mon[33332]: pgmap v885: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:03:00.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:02:59 smithi060 ceph-mon[28873]: pgmap v886: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:03:00.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:02:59 smithi090 ceph-mon[33568]: pgmap v886: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:03:00.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:02:59 smithi149 ceph-mon[33332]: pgmap v886: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:03:02.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:03:01 smithi060 ceph-mon[28873]: pgmap v887: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:03:02.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:03:01 smithi090 ceph-mon[33568]: pgmap v887: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:03:02.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:03:01 smithi149 ceph-mon[33332]: pgmap v887: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:03:04.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:03:03 smithi060 ceph-mon[28873]: pgmap v888: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:03:04.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:03:03 smithi090 ceph-mon[33568]: pgmap v888: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:03:04.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:03:03 smithi149 ceph-mon[33332]: pgmap v888: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:03:06.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:03:05 smithi060 ceph-mon[28873]: pgmap v889: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:03:06.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:03:05 smithi090 ceph-mon[33568]: pgmap v889: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:03:06.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:03:05 smithi149 ceph-mon[33332]: pgmap v889: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:03:08.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:03:07 smithi060 ceph-mon[28873]: pgmap v890: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:03:08.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:03:07 smithi090 ceph-mon[33568]: pgmap v890: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:03:08.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:03:07 smithi149 ceph-mon[33332]: pgmap v890: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:03:09.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:03:08 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T15:03:09.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:03:08 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T15:03:09.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:03:08 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T15:03:10.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:03:09 smithi060 ceph-mon[28873]: pgmap v891: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:03:10.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:03:09 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T15:03:10.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:03:09 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T15:03:10.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:03:09 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-03T15:03:10.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:03:09 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-03T15:03:10.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:03:09 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T15:03:10.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:03:09 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T15:03:10.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:03:09 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T15:03:10.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:03:09 smithi090 ceph-mon[33568]: pgmap v891: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:03:10.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:03:09 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T15:03:10.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:03:09 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T15:03:10.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:03:09 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-03T15:03:10.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:03:09 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-03T15:03:10.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:03:09 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T15:03:10.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:03:09 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T15:03:10.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:03:09 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T15:03:10.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:03:09 smithi149 ceph-mon[33332]: pgmap v891: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:03:10.425 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:03:09 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T15:03:10.425 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:03:09 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T15:03:10.425 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:03:09 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-03T15:03:10.425 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:03:09 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-03T15:03:10.425 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:03:09 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T15:03:10.425 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:03:09 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T15:03:10.426 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:03:09 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T15:03:11.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:03:10 smithi060 ceph-mon[28873]: Adjusting osd_memory_target on smithi060 to 177.5M 2024-05-03T15:03:11.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:03:10 smithi060 ceph-mon[28873]: Unable to set osd_memory_target on smithi060 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-05-03T15:03:11.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:03:10 smithi090 ceph-mon[33568]: Adjusting osd_memory_target on smithi060 to 177.5M 2024-05-03T15:03:11.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:03:10 smithi090 ceph-mon[33568]: Unable to set osd_memory_target on smithi060 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-05-03T15:03:11.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:03:10 smithi149 ceph-mon[33332]: Adjusting osd_memory_target on smithi060 to 177.5M 2024-05-03T15:03:11.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:03:10 smithi149 ceph-mon[33332]: Unable to set osd_memory_target on smithi060 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-05-03T15:03:12.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:03:11 smithi060 ceph-mon[28873]: pgmap v892: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:03:12.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:03:11 smithi090 ceph-mon[33568]: pgmap v892: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:03:12.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:03:11 smithi149 ceph-mon[33332]: pgmap v892: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:03:14.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:03:14 smithi060 ceph-mon[28873]: pgmap v893: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:03:14.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:03:14 smithi090 ceph-mon[33568]: pgmap v893: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:03:14.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:03:14 smithi149 ceph-mon[33332]: pgmap v893: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:03:15.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:03:15 smithi060 ceph-mon[28873]: pgmap v894: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:03:15.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:03:15 smithi090 ceph-mon[33568]: pgmap v894: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:03:15.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:03:15 smithi149 ceph-mon[33332]: pgmap v894: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:03:17.836 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:03:17 smithi060 ceph-mon[28873]: pgmap v895: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:03:17.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:03:17 smithi090 ceph-mon[33568]: pgmap v895: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:03:17.922 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:03:17 smithi149 ceph-mon[33332]: pgmap v895: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:03:20.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:03:19 smithi060 ceph-mon[28873]: pgmap v896: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:03:20.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:03:19 smithi090 ceph-mon[33568]: pgmap v896: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:03:20.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:03:19 smithi149 ceph-mon[33332]: pgmap v896: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:03:22.225 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:03:21 smithi149 ceph-mon[33332]: pgmap v897: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:03:22.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:03:21 smithi060 ceph-mon[28873]: pgmap v897: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:03:22.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:03:21 smithi090 ceph-mon[33568]: pgmap v897: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:03:24.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:03:23 smithi060 ceph-mon[28873]: pgmap v898: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:03:24.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:03:23 smithi090 ceph-mon[33568]: pgmap v898: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:03:24.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:03:23 smithi149 ceph-mon[33332]: pgmap v898: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:03:26.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:03:25 smithi060 ceph-mon[28873]: pgmap v899: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:03:26.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:03:25 smithi090 ceph-mon[33568]: pgmap v899: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:03:26.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:03:25 smithi149 ceph-mon[33332]: pgmap v899: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:03:28.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:03:27 smithi060 ceph-mon[28873]: pgmap v900: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:03:28.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:03:27 smithi090 ceph-mon[33568]: pgmap v900: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:03:28.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:03:27 smithi149 ceph-mon[33332]: pgmap v900: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:03:30.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:03:29 smithi060 ceph-mon[28873]: pgmap v901: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:03:30.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:03:29 smithi090 ceph-mon[33568]: pgmap v901: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:03:30.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:03:29 smithi149 ceph-mon[33332]: pgmap v901: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:03:32.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:03:31 smithi060 ceph-mon[28873]: pgmap v902: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:03:32.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:03:31 smithi090 ceph-mon[33568]: pgmap v902: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:03:32.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:03:31 smithi149 ceph-mon[33332]: pgmap v902: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:03:34.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:03:33 smithi060 ceph-mon[28873]: pgmap v903: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:03:34.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:03:33 smithi090 ceph-mon[33568]: pgmap v903: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:03:34.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:03:33 smithi149 ceph-mon[33332]: pgmap v903: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:03:36.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:03:35 smithi060 ceph-mon[28873]: pgmap v904: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:03:36.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:03:35 smithi090 ceph-mon[33568]: pgmap v904: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:03:36.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:03:35 smithi149 ceph-mon[33332]: pgmap v904: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:03:38.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:03:38 smithi060 ceph-mon[28873]: pgmap v905: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-03T15:03:38.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:03:38 smithi090 ceph-mon[33568]: pgmap v905: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-03T15:03:38.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:03:38 smithi149 ceph-mon[33332]: pgmap v905: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-03T15:03:39.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:03:39 smithi060 ceph-mon[28873]: pgmap v906: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-03T15:03:39.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:03:39 smithi090 ceph-mon[33568]: pgmap v906: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-03T15:03:39.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:03:39 smithi149 ceph-mon[33332]: pgmap v906: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-03T15:03:42.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:03:41 smithi060 ceph-mon[28873]: pgmap v907: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-03T15:03:42.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:03:41 smithi090 ceph-mon[33568]: pgmap v907: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-03T15:03:42.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:03:41 smithi149 ceph-mon[33332]: pgmap v907: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-03T15:03:44.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:03:43 smithi060 ceph-mon[28873]: pgmap v908: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-05-03T15:03:44.373 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:03:43 smithi090 ceph-mon[33568]: pgmap v908: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-05-03T15:03:44.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:03:43 smithi149 ceph-mon[33332]: pgmap v908: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-05-03T15:03:46.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:03:45 smithi060 ceph-mon[28873]: pgmap v909: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-05-03T15:03:46.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:03:45 smithi090 ceph-mon[33568]: pgmap v909: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-05-03T15:03:46.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:03:45 smithi149 ceph-mon[33332]: pgmap v909: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-05-03T15:03:48.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:03:47 smithi060 ceph-mon[28873]: pgmap v910: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-03T15:03:48.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:03:47 smithi090 ceph-mon[33568]: pgmap v910: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-03T15:03:48.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:03:47 smithi149 ceph-mon[33332]: pgmap v910: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-03T15:03:50.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:03:49 smithi060 ceph-mon[28873]: pgmap v911: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:03:50.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:03:49 smithi090 ceph-mon[33568]: pgmap v911: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:03:50.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:03:49 smithi149 ceph-mon[33332]: pgmap v911: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:03:52.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:03:51 smithi060 ceph-mon[28873]: pgmap v912: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:03:52.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:03:51 smithi090 ceph-mon[33568]: pgmap v912: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:03:52.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:03:51 smithi149 ceph-mon[33332]: pgmap v912: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:03:54.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:03:54 smithi060 ceph-mon[28873]: pgmap v913: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:03:54.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:03:54 smithi090 ceph-mon[33568]: pgmap v913: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:03:54.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:03:54 smithi149 ceph-mon[33332]: pgmap v913: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:03:56.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:03:56 smithi060 ceph-mon[28873]: pgmap v914: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:03:56.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:03:56 smithi090 ceph-mon[33568]: pgmap v914: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:03:56.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:03:56 smithi149 ceph-mon[33332]: pgmap v914: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:03:57.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:03:57 smithi060 ceph-mon[28873]: pgmap v915: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:03:57.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:03:57 smithi090 ceph-mon[33568]: pgmap v915: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:03:57.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:03:57 smithi149 ceph-mon[33332]: pgmap v915: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:04:00.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:03:59 smithi060 ceph-mon[28873]: pgmap v916: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:04:00.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:03:59 smithi090 ceph-mon[33568]: pgmap v916: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:04:00.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:03:59 smithi149 ceph-mon[33332]: pgmap v916: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:04:02.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:04:01 smithi060 ceph-mon[28873]: pgmap v917: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:04:02.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:04:01 smithi090 ceph-mon[33568]: pgmap v917: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:04:02.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:04:01 smithi149 ceph-mon[33332]: pgmap v917: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:04:04.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:04:03 smithi060 ceph-mon[28873]: pgmap v918: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:04:04.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:04:03 smithi090 ceph-mon[33568]: pgmap v918: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:04:04.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:04:03 smithi149 ceph-mon[33332]: pgmap v918: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:04:06.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:04:05 smithi060 ceph-mon[28873]: pgmap v919: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:04:06.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:04:05 smithi090 ceph-mon[33568]: pgmap v919: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:04:06.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:04:05 smithi149 ceph-mon[33332]: pgmap v919: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:04:08.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:04:08 smithi060 ceph-mon[28873]: pgmap v920: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:04:08.373 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:04:08 smithi090 ceph-mon[33568]: pgmap v920: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:04:08.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:04:08 smithi149 ceph-mon[33332]: pgmap v920: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:04:10.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:04:10 smithi060 ceph-mon[28873]: pgmap v921: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:04:10.373 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:04:10 smithi090 ceph-mon[33568]: pgmap v921: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:04:10.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:04:10 smithi149 ceph-mon[33332]: pgmap v921: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:04:11.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:04:11 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T15:04:11.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:04:11 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T15:04:11.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:04:11 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T15:04:12.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:04:12 smithi060 ceph-mon[28873]: pgmap v922: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:04:12.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:04:12 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-05-03T15:04:12.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:04:12 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T15:04:12.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:04:12 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T15:04:12.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:04:12 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-03T15:04:12.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:04:12 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-03T15:04:12.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:04:12 smithi060 ceph-mon[28873]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-05-03T15:04:12.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:04:12 smithi060 ceph-mon[28873]: Unable to set osd_memory_target on smithi090 to 184763801: error parsing value: Value '184763801' is below minimum 939524096 2024-05-03T15:04:12.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:04:12 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T15:04:12.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:04:12 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T15:04:12.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:04:12 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T15:04:12.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:04:12 smithi090 ceph-mon[33568]: pgmap v922: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:04:12.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:04:12 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-05-03T15:04:12.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:04:12 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T15:04:12.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:04:12 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T15:04:12.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:04:12 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-03T15:04:12.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:04:12 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-03T15:04:12.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:04:12 smithi090 ceph-mon[33568]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-05-03T15:04:12.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:04:12 smithi090 ceph-mon[33568]: Unable to set osd_memory_target on smithi090 to 184763801: error parsing value: Value '184763801' is below minimum 939524096 2024-05-03T15:04:12.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:04:12 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T15:04:12.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:04:12 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T15:04:12.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:04:12 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T15:04:12.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:04:12 smithi149 ceph-mon[33332]: pgmap v922: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:04:12.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:04:12 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-05-03T15:04:12.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:04:12 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T15:04:12.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:04:12 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T15:04:12.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:04:12 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-03T15:04:12.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:04:12 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-03T15:04:12.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:04:12 smithi149 ceph-mon[33332]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-05-03T15:04:12.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:04:12 smithi149 ceph-mon[33332]: Unable to set osd_memory_target on smithi090 to 184763801: error parsing value: Value '184763801' is below minimum 939524096 2024-05-03T15:04:12.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:04:12 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T15:04:12.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:04:12 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T15:04:12.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:04:12 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T15:04:13.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:04:13 smithi060 ceph-mon[28873]: pgmap v923: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:04:13.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:04:13 smithi090 ceph-mon[33568]: pgmap v923: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:04:13.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:04:13 smithi149 ceph-mon[33332]: pgmap v923: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:04:16.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:04:15 smithi060 ceph-mon[28873]: pgmap v924: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:04:16.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:04:15 smithi090 ceph-mon[33568]: pgmap v924: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:04:16.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:04:15 smithi149 ceph-mon[33332]: pgmap v924: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:04:18.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:04:17 smithi060 ceph-mon[28873]: pgmap v925: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:04:18.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:04:17 smithi090 ceph-mon[33568]: pgmap v925: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:04:18.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:04:17 smithi149 ceph-mon[33332]: pgmap v925: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:04:20.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:04:20 smithi060 ceph-mon[28873]: pgmap v926: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:04:20.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:04:20 smithi090 ceph-mon[33568]: pgmap v926: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:04:20.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:04:20 smithi149 ceph-mon[33332]: pgmap v926: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:04:22.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:04:22 smithi060 ceph-mon[28873]: pgmap v927: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:04:22.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:04:22 smithi090 ceph-mon[33568]: pgmap v927: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:04:22.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:04:22 smithi149 ceph-mon[33332]: pgmap v927: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:04:24.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:04:24 smithi060 ceph-mon[28873]: pgmap v928: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:04:24.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:04:24 smithi090 ceph-mon[33568]: pgmap v928: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:04:24.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:04:24 smithi149 ceph-mon[33332]: pgmap v928: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:04:26.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:04:26 smithi060 ceph-mon[28873]: pgmap v929: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:04:26.373 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:04:26 smithi090 ceph-mon[33568]: pgmap v929: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:04:26.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:04:26 smithi149 ceph-mon[33332]: pgmap v929: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:04:27.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:04:27 smithi060 ceph-mon[28873]: pgmap v930: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:04:27.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:04:27 smithi090 ceph-mon[33568]: pgmap v930: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:04:27.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:04:27 smithi149 ceph-mon[33332]: pgmap v930: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:04:30.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:04:29 smithi060 ceph-mon[28873]: pgmap v931: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:04:30.373 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:04:29 smithi090 ceph-mon[33568]: pgmap v931: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:04:30.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:04:29 smithi149 ceph-mon[33332]: pgmap v931: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:04:32.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:04:32 smithi060 ceph-mon[28873]: pgmap v932: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:04:32.373 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:04:32 smithi090 ceph-mon[33568]: pgmap v932: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:04:32.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:04:32 smithi149 ceph-mon[33332]: pgmap v932: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:04:34.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:04:34 smithi060 ceph-mon[28873]: pgmap v933: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:04:34.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:04:34 smithi090 ceph-mon[33568]: pgmap v933: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:04:34.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:04:34 smithi149 ceph-mon[33332]: pgmap v933: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:04:36.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:04:36 smithi060 ceph-mon[28873]: pgmap v934: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:04:36.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:04:36 smithi090 ceph-mon[33568]: pgmap v934: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:04:36.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:04:36 smithi149 ceph-mon[33332]: pgmap v934: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:04:38.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:04:38 smithi060 ceph-mon[28873]: pgmap v935: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:04:38.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:04:38 smithi090 ceph-mon[33568]: pgmap v935: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:04:38.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:04:38 smithi149 ceph-mon[33332]: pgmap v935: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:04:39.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:04:39 smithi060 ceph-mon[28873]: pgmap v936: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:04:39.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:04:39 smithi090 ceph-mon[33568]: pgmap v936: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:04:39.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:04:39 smithi149 ceph-mon[33332]: pgmap v936: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T15:04:41.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:41.860+0000 7efe05e00640 1 -- 172.21.15.60:0/3449599367 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7efdec0b28f0 msgr2=0x7efdec0b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).tick idle (900028473) for more than 900000000 us, fault. 2024-05-03T15:04:41.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:41.860+0000 7efe05e00640 1 --2- 172.21.15.60:0/3449599367 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7efdec0b28f0 0x7efdec0b4db0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7efdf8011500 tx=0x7efdf8005f90 comp rx=0 tx=0).stop 2024-05-03T15:04:41.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:41.860+0000 7efdfea00640 1 -- 172.21.15.60:0/3449599367 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7efdec0b28f0 msgr2=0x7efdec0b4db0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-03T15:04:41.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:41.860+0000 7efdfea00640 1 --2- 172.21.15.60:0/3449599367 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7efdec0b28f0 0x7efdec0b4db0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:04:41.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:41.860+0000 7efdfea00640 1 --2- 172.21.15.60:0/3449599367 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7efdec0bdf20 0x7efdec0c0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T15:04:41.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:41.861+0000 7efdfea00640 1 -- 172.21.15.60:0/3449599367 --> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7efdec0c08f0 con 0x7efdec0bdf20 2024-05-03T15:04:41.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:41.861+0000 7efe05e00640 1 --2- 172.21.15.60:0/3449599367 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7efdec0bdf20 0x7efdec0c0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T15:04:41.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:41.863+0000 7efe05e00640 1 --2- 172.21.15.60:0/3449599367 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7efdec0bdf20 0x7efdec0c0310 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7efdf8009fd0 tx=0x7efdf8018000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T15:04:41.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:41.976+0000 7efdfd600640 1 -- 172.21.15.60:0/3449599367 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7efde00063a0 con 0x7efe0810a970 2024-05-03T15:04:41.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:41.977+0000 7efdfea00640 1 -- 172.21.15.60:0/3449599367 <== mon.2 v2:172.21.15.90:3300/0 12 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7efde00063a0 con 0x7efe0810a970 2024-05-03T15:04:42.078 INFO:teuthology.orchestra.run.smithi060.stderr:Error ETIMEDOUT: Connection timed out. Please check the client's permission and connection. 2024-05-03T15:04:42.102 DEBUG:teuthology.orchestra.run:got remote process result: 110 2024-05-03T15:04:42.102 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/task/exec.py", line 66, in task remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi060 with status 110: "sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10'" 2024-05-03T15:04:42.348 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=d7925e8176b54e8c95f9433176f35efe Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/task/exec.py", line 66, in task remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi060 with status 110: "sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10'" 2024-05-03T15:04:42.350 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-05-03T15:04:42.359 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-03T15:04:42.360 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-05-03T15:04:42.360 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-03T15:04:42.364 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:04:42 smithi060 ceph-mon[28873]: pgmap v937: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:04:42.373 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:04:42 smithi090 ceph-mon[33568]: pgmap v937: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:04:42.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:04:42 smithi149 ceph-mon[33332]: pgmap v937: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T15:04:42.426 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-05-03T15:04:42.426 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-03T15:04:42.481 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph osd blocklist ls 2024-05-03T15:04:42.948 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T15:04:43.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:04:43 smithi060 ceph-mon[28873]: from='client.25196 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T15:04:43.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:04:43 smithi090 ceph-mon[33568]: from='client.25196 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T15:04:43.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:04:43 smithi149 ceph-mon[33332]: from='client.25196 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T15:04:44.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:04:44 smithi060 ceph-mon[28873]: pgmap v938: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:04:44.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:04:44 smithi090 ceph-mon[33568]: pgmap v938: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:04:44.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:04:44 smithi149 ceph-mon[33332]: pgmap v938: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-03T15:04:45.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:45.505+0000 7fdbc5200700 1 -- 172.21.15.60:0/360755234 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdbc006bb40 msgr2=0x7fdbc0110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T15:04:45.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:45.505+0000 7fdbc5200700 1 --2- 172.21.15.60:0/360755234 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdbc006bb40 0x7fdbc0110330 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fdbb800b0a0 tx=0x7fdbb800b3b0 comp rx=0 tx=0).stop 2024-05-03T15:04:45.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:45.505+0000 7fdbc5200700 1 -- 172.21.15.60:0/360755234 shutdown_connections 2024-05-03T15:04:45.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:45.505+0000 7fdbc5200700 1 --2- 172.21.15.60:0/360755234 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdbc006bb40 0x7fdbc0110330 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:04:45.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:45.505+0000 7fdbc5200700 1 --2- 172.21.15.60:0/360755234 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdbc0109510 0x7fdbc01098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:04:45.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:45.505+0000 7fdbc5200700 1 --2- 172.21.15.60:0/360755234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdbc006b180 0x7fdbc006b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:04:45.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:45.505+0000 7fdbc5200700 1 -- 172.21.15.60:0/360755234 >> 172.21.15.60:0/360755234 conn(0x7fdbc0079350 msgr2=0x7fdbc0079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T15:04:45.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:45.505+0000 7fdbc5200700 1 -- 172.21.15.60:0/360755234 shutdown_connections 2024-05-03T15:04:45.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:45.505+0000 7fdbc5200700 1 -- 172.21.15.60:0/360755234 wait complete. 2024-05-03T15:04:45.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:45.507+0000 7fdbc5200700 1 Processor -- start 2024-05-03T15:04:45.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:45.507+0000 7fdbc5200700 1 -- start start 2024-05-03T15:04:45.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:45.507+0000 7fdbc5200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdbc006b180 0x7fdbc0102470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T15:04:45.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:45.507+0000 7fdbc5200700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdbc006bb40 0x7fdbc01029b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T15:04:45.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:45.507+0000 7fdbc5200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdbc0109510 0x7fdbc0102ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T15:04:45.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:45.507+0000 7fdbc5200700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc01138b0 con 0x7fdbc006bb40 2024-05-03T15:04:45.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:45.507+0000 7fdbc5200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc0113730 con 0x7fdbc0109510 2024-05-03T15:04:45.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:45.507+0000 7fdbc5200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc0113a30 con 0x7fdbc006b180 2024-05-03T15:04:45.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:45.508+0000 7fdbbde00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdbc006b180 0x7fdbc0102470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T15:04:45.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:45.508+0000 7fdbbd400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdbc006bb40 0x7fdbc01029b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T15:04:45.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:45.508+0000 7fdbbe800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdbc0109510 0x7fdbc0102ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T15:04:45.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:45.508+0000 7fdbbe800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdbc0109510 0x7fdbc0102ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.60:54838/0 (socket says 172.21.15.60:54838) 2024-05-03T15:04:45.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:45.508+0000 7fdbbe800700 1 -- 172.21.15.60:0/2021682643 learned_addr learned my addr 172.21.15.60:0/2021682643 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T15:04:45.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:45.508+0000 7fdbbd400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdbc006bb40 0x7fdbc01029b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56884/0 (socket says 172.21.15.60:56884) 2024-05-03T15:04:45.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:45.508+0000 7fdbbde00700 1 -- 172.21.15.60:0/2021682643 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdbc0109510 msgr2=0x7fdbc0102ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T15:04:45.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:45.508+0000 7fdbbde00700 1 --2- 172.21.15.60:0/2021682643 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdbc0109510 0x7fdbc0102ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:04:45.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:45.508+0000 7fdbbde00700 1 -- 172.21.15.60:0/2021682643 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdbc006bb40 msgr2=0x7fdbc01029b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T15:04:45.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:45.508+0000 7fdbbde00700 1 --2- 172.21.15.60:0/2021682643 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdbc006bb40 0x7fdbc01029b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:04:45.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:45.508+0000 7fdbbde00700 1 -- 172.21.15.60:0/2021682643 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbc01a81b0 con 0x7fdbc006b180 2024-05-03T15:04:45.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:45.509+0000 7fdbbde00700 1 --2- 172.21.15.60:0/2021682643 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdbc006b180 0x7fdbc0102470 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdbb000e9f0 tx=0x7fdbb000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T15:04:45.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:45.509+0000 7fdbb6a00700 1 -- 172.21.15.60:0/2021682643 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb000c9c0 con 0x7fdbc006b180 2024-05-03T15:04:45.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:45.509+0000 7fdbc5200700 1 -- 172.21.15.60:0/2021682643 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbc01a84a0 con 0x7fdbc006b180 2024-05-03T15:04:45.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:45.509+0000 7fdbb6a00700 1 -- 172.21.15.60:0/2021682643 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7fdbb000cb60 con 0x7fdbc006b180 2024-05-03T15:04:45.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:45.509+0000 7fdbb6a00700 1 -- 172.21.15.60:0/2021682643 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb0018660 con 0x7fdbc006b180 2024-05-03T15:04:45.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:45.509+0000 7fdbc5200700 1 -- 172.21.15.60:0/2021682643 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbc01a8a30 con 0x7fdbc006b180 2024-05-03T15:04:45.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:45.511+0000 7fdbc5200700 1 -- 172.21.15.60:0/2021682643 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbc006a190 con 0x7fdbc006b180 2024-05-03T15:04:45.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:45.513+0000 7fdbb6a00700 1 -- 172.21.15.60:0/2021682643 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fdbb000cd10 con 0x7fdbc006b180 2024-05-03T15:04:45.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:45.514+0000 7fdbb6a00700 1 --2- 172.21.15.60:0/2021682643 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fdba40b2990 0x7fdba40b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T15:04:45.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:45.515+0000 7fdbb6a00700 1 -- 172.21.15.60:0/2021682643 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7fdbb00e27c0 con 0x7fdbc006b180 2024-05-03T15:04:45.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:45.515+0000 7fdbbd400700 1 --2- 172.21.15.60:0/2021682643 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fdba40b2990 0x7fdba40b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T15:04:45.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:45.515+0000 7fdbbd400700 1 --2- 172.21.15.60:0/2021682643 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fdba40b2990 0x7fdba40b4e50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdbc00798e0 tx=0x7fdbac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T15:04:45.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:45.519+0000 7fdbb6a00700 1 -- 172.21.15.60:0/2021682643 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fdbb00a9fa0 con 0x7fdbc006b180 2024-05-03T15:04:45.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:45.775+0000 7fdbc5200700 1 -- 172.21.15.60:0/2021682643 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fdbc01039a0 con 0x7fdbc006b180 2024-05-03T15:04:45.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:45.775+0000 7fdbb6a00700 1 -- 172.21.15.60:0/2021682643 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) ==== 81+0+586 (secure 0 0 0) 0x7fdbb00a98a0 con 0x7fdbc006b180 2024-05-03T15:04:45.777 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6801/424391417 2024-05-04T14:36:04.154333+0000 2024-05-03T15:04:45.777 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6800/424391417 2024-05-04T14:36:04.154333+0000 2024-05-03T15:04:45.777 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/1426532561 2024-05-04T14:36:04.154333+0000 2024-05-03T15:04:45.777 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/3230079926 2024-05-04T14:35:28.066961+0000 2024-05-03T15:04:45.777 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/2334895610 2024-05-04T14:35:28.066961+0000 2024-05-03T15:04:45.777 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6800/888447185 2024-05-04T14:35:28.066961+0000 2024-05-03T15:04:45.778 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/3996899937 2024-05-04T14:35:28.066961+0000 2024-05-03T15:04:45.778 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/2779644018 2024-05-04T14:36:04.154333+0000 2024-05-03T15:04:45.778 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/71440838 2024-05-04T14:36:04.154333+0000 2024-05-03T15:04:45.778 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6801/888447185 2024-05-04T14:35:28.066961+0000 2024-05-03T15:04:45.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:45.779+0000 7fdbc5200700 1 -- 172.21.15.60:0/2021682643 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fdba40b2990 msgr2=0x7fdba40b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T15:04:45.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:45.779+0000 7fdbc5200700 1 --2- 172.21.15.60:0/2021682643 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fdba40b2990 0x7fdba40b4e50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdbc00798e0 tx=0x7fdbac009260 comp rx=0 tx=0).stop 2024-05-03T15:04:45.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:45.779+0000 7fdbc5200700 1 -- 172.21.15.60:0/2021682643 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdbc006b180 msgr2=0x7fdbc0102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T15:04:45.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:45.779+0000 7fdbc5200700 1 --2- 172.21.15.60:0/2021682643 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdbc006b180 0x7fdbc0102470 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdbb000e9f0 tx=0x7fdbb000ed00 comp rx=0 tx=0).stop 2024-05-03T15:04:45.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:45.779+0000 7fdbc5200700 1 -- 172.21.15.60:0/2021682643 shutdown_connections 2024-05-03T15:04:45.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:45.779+0000 7fdbc5200700 1 --2- 172.21.15.60:0/2021682643 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7fdba40b2990 0x7fdba40b4e50 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:04:45.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:45.779+0000 7fdbc5200700 1 --2- 172.21.15.60:0/2021682643 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdbc0109510 0x7fdbc0102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:04:45.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:45.779+0000 7fdbc5200700 1 --2- 172.21.15.60:0/2021682643 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdbc006b180 0x7fdbc0102470 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:04:45.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:45.779+0000 7fdbc5200700 1 --2- 172.21.15.60:0/2021682643 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdbc006bb40 0x7fdbc01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:04:45.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:45.779+0000 7fdbc5200700 1 -- 172.21.15.60:0/2021682643 >> 172.21.15.60:0/2021682643 conn(0x7fdbc0079350 msgr2=0x7fdbc0100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T15:04:45.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:45.780+0000 7fdbc5200700 1 -- 172.21.15.60:0/2021682643 shutdown_connections 2024-05-03T15:04:45.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:45.780+0000 7fdbc5200700 1 -- 172.21.15.60:0/2021682643 wait complete. 2024-05-03T15:04:45.782 INFO:teuthology.orchestra.run.smithi060.stderr:listed 10 entries 2024-05-03T15:04:46.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:04:46 smithi060 ceph-mon[28873]: pgmap v939: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:04:46.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:04:46 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/2021682643' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-03T15:04:46.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:04:46 smithi090 ceph-mon[33568]: pgmap v939: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:04:46.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:04:46 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/2021682643' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-03T15:04:46.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:04:46 smithi149 ceph-mon[33332]: pgmap v939: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-03T15:04:46.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:04:46 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/2021682643' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-03T15:04:46.570 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-03T15:04:46.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T15:04:46.571 DEBUG:teuthology.orchestra.run.smithi060:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-03T15:04:46.674 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2024-05-03T15:04:46.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T15:04:46.674 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-03T15:04:46.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T15:04:46.717 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-05-03T15:04:46.805 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-03T15:04:46.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T15:04:46.806 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-03T15:04:46.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T15:04:46.930 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link delete ceph-brx 2024-05-03T15:04:47.035 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-03T15:04:47.044 INFO:tasks.cephadm:Teardown begin 2024-05-03T15:04:47.045 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-03T15:04:47.045 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T15:04:47.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:04:47 smithi060 ceph-mon[28873]: pgmap v940: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:04:47.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:04:47 smithi090 ceph-mon[33568]: pgmap v940: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:04:47.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:04:47 smithi149 ceph-mon[33332]: pgmap v940: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-03T15:04:47.472 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T15:04:49.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:49.623+0000 7f913ca00700 1 -- 172.21.15.60:0/2906723934 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9138101cd0 msgr2=0x7f9138102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T15:04:49.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:49.623+0000 7f913ca00700 1 --2- 172.21.15.60:0/2906723934 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9138101cd0 0x7f9138102150 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f91240099c0 tx=0x7f9124009cd0 comp rx=0 tx=0).stop 2024-05-03T15:04:49.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:49.623+0000 7f913ca00700 1 -- 172.21.15.60:0/2906723934 shutdown_connections 2024-05-03T15:04:49.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:49.623+0000 7f913ca00700 1 --2- 172.21.15.60:0/2906723934 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9138102690 0x7f91381125a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:04:49.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:49.623+0000 7f913ca00700 1 --2- 172.21.15.60:0/2906723934 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f91381041b0 0x7f9138104590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:04:49.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:49.623+0000 7f913ca00700 1 --2- 172.21.15.60:0/2906723934 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9138101cd0 0x7f9138102150 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:04:49.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:49.623+0000 7f913ca00700 1 -- 172.21.15.60:0/2906723934 >> 172.21.15.60:0/2906723934 conn(0x7f9138077650 msgr2=0x7f9138077a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T15:04:49.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:49.623+0000 7f913ca00700 1 -- 172.21.15.60:0/2906723934 shutdown_connections 2024-05-03T15:04:49.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:49.623+0000 7f913ca00700 1 -- 172.21.15.60:0/2906723934 wait complete. 2024-05-03T15:04:49.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:49.625+0000 7f913ca00700 1 Processor -- start 2024-05-03T15:04:49.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:49.625+0000 7f913ca00700 1 -- start start 2024-05-03T15:04:49.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:49.625+0000 7f913ca00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9138101cd0 0x7f913819bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T15:04:49.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:49.625+0000 7f913ca00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9138102690 0x7f913819c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T15:04:49.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:49.625+0000 7f913ca00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91381041b0 0x7f91381a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T15:04:49.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:49.626+0000 7f913ca00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f9138115c30 con 0x7f91381041b0 2024-05-03T15:04:49.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:49.626+0000 7f913ca00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9138115ab0 con 0x7f9138101cd0 2024-05-03T15:04:49.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:49.626+0000 7f913ca00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9138115db0 con 0x7f9138102690 2024-05-03T15:04:49.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:49.626+0000 7f9137400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9138101cd0 0x7f913819bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T15:04:49.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:49.626+0000 7f9137400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9138101cd0 0x7f913819bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.60:54862/0 (socket says 172.21.15.60:54862) 2024-05-03T15:04:49.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:49.626+0000 7f9137400700 1 -- 172.21.15.60:0/1090152543 learned_addr learned my addr 172.21.15.60:0/1090152543 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T15:04:49.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:49.626+0000 7f9136a00700 1 --2- 172.21.15.60:0/1090152543 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9138102690 0x7f913819c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T15:04:49.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:49.626+0000 7f9137e00700 1 --2- 172.21.15.60:0/1090152543 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91381041b0 0x7f91381a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T15:04:49.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:49.626+0000 7f9137400700 1 -- 172.21.15.60:0/1090152543 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9138102690 msgr2=0x7f913819c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T15:04:49.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:49.626+0000 7f9137400700 1 --2- 172.21.15.60:0/1090152543 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9138102690 0x7f913819c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:04:49.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:49.626+0000 7f9137400700 1 -- 172.21.15.60:0/1090152543 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91381041b0 msgr2=0x7f91381a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T15:04:49.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:49.626+0000 7f9137400700 1 --2- 172.21.15.60:0/1090152543 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91381041b0 0x7f91381a0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:04:49.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:49.626+0000 7f9137400700 1 -- 172.21.15.60:0/1090152543 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91381a0fd0 con 0x7f9138101cd0 2024-05-03T15:04:49.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:49.626+0000 7f9136a00700 1 --2- 172.21.15.60:0/1090152543 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9138102690 0x7f913819c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T15:04:49.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:49.627+0000 7f9137400700 1 --2- 172.21.15.60:0/1090152543 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9138101cd0 0x7f913819bfc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f912800e9a0 tx=0x7f912800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T15:04:49.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:49.627+0000 7f912fe00700 1 -- 172.21.15.60:0/1090152543 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f912800ca10 con 0x7f9138101cd0 2024-05-03T15:04:49.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:49.627+0000 7f913ca00700 1 -- 172.21.15.60:0/1090152543 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91381a12c0 con 0x7f9138101cd0 2024-05-03T15:04:49.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:49.627+0000 7f912fe00700 1 -- 172.21.15.60:0/1090152543 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f912800cbb0 con 0x7f9138101cd0 2024-05-03T15:04:49.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:49.627+0000 7f912fe00700 1 -- 172.21.15.60:0/1090152543 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91280187a0 con 0x7f9138101cd0 2024-05-03T15:04:49.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:49.627+0000 7f913ca00700 1 -- 172.21.15.60:0/1090152543 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91381a8b40 con 0x7f9138101cd0 2024-05-03T15:04:49.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:49.628+0000 7f913ca00700 1 -- 172.21.15.60:0/1090152543 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f913806a190 con 0x7f9138101cd0 2024-05-03T15:04:49.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:49.631+0000 7f912fe00700 1 -- 172.21.15.60:0/1090152543 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f9128010430 con 0x7f9138101cd0 2024-05-03T15:04:49.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:49.632+0000 7f912fe00700 1 --2- 172.21.15.60:0/1090152543 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f911c0b2990 0x7f911c0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T15:04:49.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:49.633+0000 7f912fe00700 1 -- 172.21.15.60:0/1090152543 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f91280e2a50 con 0x7f9138101cd0 2024-05-03T15:04:49.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:49.633+0000 7f9136a00700 1 --2- 172.21.15.60:0/1090152543 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f911c0b2990 0x7f911c0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T15:04:49.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:49.634+0000 7f9136a00700 1 --2- 172.21.15.60:0/1090152543 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f911c0b2990 0x7f911c0b4e50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9138077be0 tx=0x7f9124005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T15:04:49.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:49.635+0000 7f912fe00700 1 -- 172.21.15.60:0/1090152543 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f91280aa230 con 0x7f9138101cd0 2024-05-03T15:04:49.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:49.923+0000 7f913ca00700 1 -- 172.21.15.60:0/1090152543 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f913802e510 con 0x7f9138101cd0 2024-05-03T15:04:49.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:49.924+0000 7f912fe00700 1 -- 172.21.15.60:0/1090152543 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11929 (secure 0 0 0) 0x7f91280a9b30 con 0x7f9138101cd0 2024-05-03T15:04:49.926 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T15:04:49.926 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1849151522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1849151522},{"type":"v1","addr":"172.21.15.60:6835","nonce":1849151522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/523710346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":523710346},{"type":"v1","addr":"172.21.15.60:6837","nonce":523710346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T14:44:34.226052+0000","modified":"2024-05-03T14:49:41.106977+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24511,"mds_1":24509,"mds_2":24487,"mds_3":24475,"mds_4":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14397":{"gid":14397,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.60:6839/3846205424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3846205424},{"type":"v1","addr":"172.21.15.60:6839","nonce":3846205424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6833/3033424631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":3033424631},{"type":"v1","addr":"172.21.15.90:6833","nonce":3033424631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6833/3454021291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3454021291},{"type":"v1","addr":"172.21.15.149:6833","nonce":3454021291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6835/1057049585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1057049585},{"type":"v1","addr":"172.21.15.90:6835","nonce":1057049585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/882239924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":882239924},{"type":"v1","addr":"172.21.15.149:6835","nonce":882239924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":31,"addr":"172.21.15.90:6837/2243702844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":2243702844},{"type":"v1","addr":"172.21.15.90:6837","nonce":2243702844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":32,"addr":"172.21.15.149:6837/2611278690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2611278690},{"type":"v1","addr":"172.21.15.149:6837","nonce":2611278690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.149:6839/2527622212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2527622212},{"type":"v1","addr":"172.21.15.149:6839","nonce":2527622212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.90:6839/3512381172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3512381172},{"type":"v1","addr":"172.21.15.90:6839","nonce":3512381172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.60:6841/924163182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":924163182},{"type":"v1","addr":"172.21.15.60:6841","nonce":924163182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24511,"qdb_cluster":[24511,24509,24487,24470,24475]},"id":1}]} 2024-05-03T15:04:49.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:49.927+0000 7f913ca00700 1 -- 172.21.15.60:0/1090152543 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f911c0b2990 msgr2=0x7f911c0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T15:04:49.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:49.927+0000 7f913ca00700 1 --2- 172.21.15.60:0/1090152543 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f911c0b2990 0x7f911c0b4e50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9138077be0 tx=0x7f9124005fb0 comp rx=0 tx=0).stop 2024-05-03T15:04:49.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:49.927+0000 7f913ca00700 1 -- 172.21.15.60:0/1090152543 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9138101cd0 msgr2=0x7f913819bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T15:04:49.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:49.927+0000 7f913ca00700 1 --2- 172.21.15.60:0/1090152543 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9138101cd0 0x7f913819bfc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f912800e9a0 tx=0x7f912800ecb0 comp rx=0 tx=0).stop 2024-05-03T15:04:49.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:49.928+0000 7f913ca00700 1 -- 172.21.15.60:0/1090152543 shutdown_connections 2024-05-03T15:04:49.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:49.928+0000 7f913ca00700 1 --2- 172.21.15.60:0/1090152543 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f911c0b2990 0x7f911c0b4e50 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:04:49.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:49.928+0000 7f913ca00700 1 --2- 172.21.15.60:0/1090152543 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91381041b0 0x7f91381a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:04:49.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:49.928+0000 7f913ca00700 1 --2- 172.21.15.60:0/1090152543 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9138101cd0 0x7f913819bfc0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:04:49.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:49.928+0000 7f913ca00700 1 --2- 172.21.15.60:0/1090152543 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9138102690 0x7f913819c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:04:49.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:49.928+0000 7f913ca00700 1 -- 172.21.15.60:0/1090152543 >> 172.21.15.60:0/1090152543 conn(0x7f9138077650 msgr2=0x7f91380ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T15:04:49.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:49.928+0000 7f913ca00700 1 -- 172.21.15.60:0/1090152543 shutdown_connections 2024-05-03T15:04:49.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:49.928+0000 7f913ca00700 1 -- 172.21.15.60:0/1090152543 wait complete. 2024-05-03T15:04:49.930 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 30 2024-05-03T15:04:50.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:04:50 smithi060 ceph-mon[28873]: pgmap v941: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-03T15:04:50.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:04:50 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/1090152543' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T15:04:50.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:04:50 smithi090 ceph-mon[33568]: pgmap v941: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-03T15:04:50.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:04:50 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/1090152543' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T15:04:50.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:04:50 smithi149 ceph-mon[33332]: pgmap v941: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-03T15:04:50.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:04:50 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/1090152543' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T15:04:50.767 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T15:04:51.229 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T15:04:52.317 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:04:52 smithi060 ceph-mon[28873]: pgmap v942: 129 pgs: 129 active+clean; 649 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 767 B/s wr, 12 op/s 2024-05-03T15:04:52.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:04:52 smithi090 ceph-mon[33568]: pgmap v942: 129 pgs: 129 active+clean; 649 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 767 B/s wr, 12 op/s 2024-05-03T15:04:52.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:04:52 smithi149 ceph-mon[33332]: pgmap v942: 129 pgs: 129 active+clean; 649 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 767 B/s wr, 12 op/s 2024-05-03T15:04:53.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:53.500+0000 7f67cac00700 1 -- 172.21.15.60:0/3317344231 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67c4107be0 msgr2=0x7f67c4107fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T15:04:53.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:53.500+0000 7f67cac00700 1 --2- 172.21.15.60:0/3317344231 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67c4107be0 0x7f67c4107fc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f67b40098f0 tx=0x7f67b4009c00 comp rx=0 tx=0).stop 2024-05-03T15:04:53.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:53.500+0000 7f67cac00700 1 -- 172.21.15.60:0/3317344231 shutdown_connections 2024-05-03T15:04:53.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:53.500+0000 7f67cac00700 1 --2- 172.21.15.60:0/3317344231 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67c4102540 0x7f67c410e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:04:53.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:53.500+0000 7f67cac00700 1 --2- 172.21.15.60:0/3317344231 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67c4107be0 0x7f67c4107fc0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:04:53.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:53.500+0000 7f67cac00700 1 --2- 172.21.15.60:0/3317344231 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f67c4101b80 0x7f67c4102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:04:53.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:53.500+0000 7f67cac00700 1 -- 172.21.15.60:0/3317344231 >> 172.21.15.60:0/3317344231 conn(0x7f67c4078a10 msgr2=0x7f67c40ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T15:04:53.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:53.500+0000 7f67cac00700 1 -- 172.21.15.60:0/3317344231 shutdown_connections 2024-05-03T15:04:53.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:53.500+0000 7f67cac00700 1 -- 172.21.15.60:0/3317344231 wait complete. 2024-05-03T15:04:53.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:53.502+0000 7f67cac00700 1 Processor -- start 2024-05-03T15:04:53.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:53.503+0000 7f67cac00700 1 -- start start 2024-05-03T15:04:53.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:53.503+0000 7f67cac00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67c4101b80 0x7f67c419e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T15:04:53.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:53.503+0000 7f67cac00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f67c4102540 0x7f67c419edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T15:04:53.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:53.503+0000 7f67cac00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67c4107be0 0x7f67c4198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T15:04:53.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:53.503+0000 7f67cac00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f67c4112820 con 0x7f67c4101b80 2024-05-03T15:04:53.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:53.503+0000 7f67cac00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f67c41126a0 con 0x7f67c4102540 2024-05-03T15:04:53.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:53.503+0000 7f67cac00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f67c41129a0 con 0x7f67c4107be0 2024-05-03T15:04:53.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:53.504+0000 7f67c3400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f67c4102540 0x7f67c419edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T15:04:53.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:53.504+0000 7f67c3e00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67c4101b80 0x7f67c419e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T15:04:53.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:53.504+0000 7f67c3400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f67c4102540 0x7f67c419edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.60:54650/0 (socket says 172.21.15.60:54650) 2024-05-03T15:04:53.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:53.504+0000 7f67c3400700 1 -- 172.21.15.60:0/493435408 learned_addr learned my addr 172.21.15.60:0/493435408 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T15:04:53.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:53.504+0000 7f67c8c00700 1 --2- 172.21.15.60:0/493435408 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67c4107be0 0x7f67c4198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T15:04:53.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:53.504+0000 7f67c3400700 1 -- 172.21.15.60:0/493435408 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67c4107be0 msgr2=0x7f67c4198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T15:04:53.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:53.504+0000 7f67c3400700 1 --2- 172.21.15.60:0/493435408 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67c4107be0 0x7f67c4198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:04:53.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:53.504+0000 7f67c3400700 1 -- 172.21.15.60:0/493435408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67c4101b80 msgr2=0x7f67c419e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T15:04:53.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:53.504+0000 7f67c3400700 1 --2- 172.21.15.60:0/493435408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67c4101b80 0x7f67c419e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:04:53.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:53.504+0000 7f67c3400700 1 -- 172.21.15.60:0/493435408 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67c41991a0 con 0x7f67c4102540 2024-05-03T15:04:53.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:53.504+0000 7f67c3e00700 1 --2- 172.21.15.60:0/493435408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67c4101b80 0x7f67c419e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T15:04:53.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:53.505+0000 7f67c3400700 1 --2- 172.21.15.60:0/493435408 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f67c4102540 0x7f67c419edc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f67b000e9a0 tx=0x7f67b000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T15:04:53.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:53.505+0000 7f67c0c00700 1 -- 172.21.15.60:0/493435408 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b001b070 con 0x7f67c4102540 2024-05-03T15:04:53.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:53.505+0000 7f67cac00700 1 -- 172.21.15.60:0/493435408 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67c4199490 con 0x7f67c4102540 2024-05-03T15:04:53.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:53.505+0000 7f67c0c00700 1 -- 172.21.15.60:0/493435408 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f67b000dd20 con 0x7f67c4102540 2024-05-03T15:04:53.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:53.505+0000 7f67c0c00700 1 -- 172.21.15.60:0/493435408 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b00153b0 con 0x7f67c4102540 2024-05-03T15:04:53.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:53.505+0000 7f67cac00700 1 -- 172.21.15.60:0/493435408 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67c410f370 con 0x7f67c4102540 2024-05-03T15:04:53.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:53.507+0000 7f67cac00700 1 -- 172.21.15.60:0/493435408 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67c406a190 con 0x7f67c4102540 2024-05-03T15:04:53.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:53.510+0000 7f67c0c00700 1 -- 172.21.15.60:0/493435408 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f67b001e070 con 0x7f67c4102540 2024-05-03T15:04:53.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:53.511+0000 7f67c0c00700 1 --2- 172.21.15.60:0/493435408 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f67a80b2910 0x7f67a80b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T15:04:53.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:53.511+0000 7f67c0c00700 1 -- 172.21.15.60:0/493435408 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f67b00e1cf0 con 0x7f67c4102540 2024-05-03T15:04:53.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:53.512+0000 7f67c3e00700 1 --2- 172.21.15.60:0/493435408 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f67a80b2910 0x7f67a80b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T15:04:53.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:53.512+0000 7f67c3e00700 1 --2- 172.21.15.60:0/493435408 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f67a80b2910 0x7f67a80b4dd0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f67b40098c0 tx=0x7f67b401d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T15:04:53.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:53.516+0000 7f67c0c00700 1 -- 172.21.15.60:0/493435408 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f67b00a94d0 con 0x7f67c4102540 2024-05-03T15:04:53.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:53.809+0000 7f67cac00700 1 -- 172.21.15.60:0/493435408 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f67c410fdd0 con 0x7f67c4102540 2024-05-03T15:04:53.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:53.810+0000 7f67c0c00700 1 -- 172.21.15.60:0/493435408 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11929 (secure 0 0 0) 0x7f67b00a8dd0 con 0x7f67c4102540 2024-05-03T15:04:53.811 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T15:04:53.811 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1849151522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1849151522},{"type":"v1","addr":"172.21.15.60:6835","nonce":1849151522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/523710346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":523710346},{"type":"v1","addr":"172.21.15.60:6837","nonce":523710346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T14:44:34.226052+0000","modified":"2024-05-03T14:49:41.106977+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24511,"mds_1":24509,"mds_2":24487,"mds_3":24475,"mds_4":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14397":{"gid":14397,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.60:6839/3846205424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3846205424},{"type":"v1","addr":"172.21.15.60:6839","nonce":3846205424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6833/3033424631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":3033424631},{"type":"v1","addr":"172.21.15.90:6833","nonce":3033424631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6833/3454021291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3454021291},{"type":"v1","addr":"172.21.15.149:6833","nonce":3454021291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6835/1057049585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1057049585},{"type":"v1","addr":"172.21.15.90:6835","nonce":1057049585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/882239924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":882239924},{"type":"v1","addr":"172.21.15.149:6835","nonce":882239924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":31,"addr":"172.21.15.90:6837/2243702844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":2243702844},{"type":"v1","addr":"172.21.15.90:6837","nonce":2243702844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":32,"addr":"172.21.15.149:6837/2611278690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2611278690},{"type":"v1","addr":"172.21.15.149:6837","nonce":2611278690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.149:6839/2527622212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2527622212},{"type":"v1","addr":"172.21.15.149:6839","nonce":2527622212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.90:6839/3512381172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3512381172},{"type":"v1","addr":"172.21.15.90:6839","nonce":3512381172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.60:6841/924163182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":924163182},{"type":"v1","addr":"172.21.15.60:6841","nonce":924163182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24511,"qdb_cluster":[24511,24509,24487,24470,24475]},"id":1}]} 2024-05-03T15:04:53.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:53.813+0000 7f67cac00700 1 -- 172.21.15.60:0/493435408 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f67a80b2910 msgr2=0x7f67a80b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T15:04:53.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:53.813+0000 7f67cac00700 1 --2- 172.21.15.60:0/493435408 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f67a80b2910 0x7f67a80b4dd0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f67b40098c0 tx=0x7f67b401d040 comp rx=0 tx=0).stop 2024-05-03T15:04:53.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:53.813+0000 7f67cac00700 1 -- 172.21.15.60:0/493435408 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f67c4102540 msgr2=0x7f67c419edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T15:04:53.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:53.813+0000 7f67cac00700 1 --2- 172.21.15.60:0/493435408 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f67c4102540 0x7f67c419edc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f67b000e9a0 tx=0x7f67b000ecb0 comp rx=0 tx=0).stop 2024-05-03T15:04:53.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:53.813+0000 7f67cac00700 1 -- 172.21.15.60:0/493435408 shutdown_connections 2024-05-03T15:04:53.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:53.813+0000 7f67cac00700 1 --2- 172.21.15.60:0/493435408 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f67a80b2910 0x7f67a80b4dd0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:04:53.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:53.813+0000 7f67cac00700 1 --2- 172.21.15.60:0/493435408 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67c4107be0 0x7f67c4198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:04:53.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:53.813+0000 7f67cac00700 1 --2- 172.21.15.60:0/493435408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67c4101b80 0x7f67c419e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:04:53.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:53.813+0000 7f67cac00700 1 --2- 172.21.15.60:0/493435408 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f67c4102540 0x7f67c419edc0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:04:53.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:53.813+0000 7f67cac00700 1 -- 172.21.15.60:0/493435408 >> 172.21.15.60:0/493435408 conn(0x7f67c4078a10 msgr2=0x7f67c40ffff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T15:04:53.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:53.814+0000 7f67cac00700 1 -- 172.21.15.60:0/493435408 shutdown_connections 2024-05-03T15:04:53.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:53.814+0000 7f67cac00700 1 -- 172.21.15.60:0/493435408 wait complete. 2024-05-03T15:04:53.817 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 30 2024-05-03T15:04:54.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:04:54 smithi060 ceph-mon[28873]: pgmap v943: 129 pgs: 129 active+clean; 649 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 9 op/s 2024-05-03T15:04:54.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:04:54 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/493435408' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T15:04:54.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:04:54 smithi090 ceph-mon[33568]: pgmap v943: 129 pgs: 129 active+clean; 649 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 9 op/s 2024-05-03T15:04:54.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:04:54 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/493435408' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T15:04:54.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:04:54 smithi149 ceph-mon[33332]: pgmap v943: 129 pgs: 129 active+clean; 649 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 9 op/s 2024-05-03T15:04:54.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:04:54 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/493435408' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T15:04:54.647 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-03T15:04:55.114 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T15:04:56.219 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:04:56 smithi060 ceph-mon[28873]: pgmap v944: 129 pgs: 129 active+clean; 649 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 767 B/s wr, 10 op/s 2024-05-03T15:04:56.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:04:56 smithi090 ceph-mon[33568]: pgmap v944: 129 pgs: 129 active+clean; 649 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 767 B/s wr, 10 op/s 2024-05-03T15:04:56.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:04:56 smithi149 ceph-mon[33332]: pgmap v944: 129 pgs: 129 active+clean; 649 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 767 B/s wr, 10 op/s 2024-05-03T15:04:57.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:57.093+0000 7f035f600700 1 -- 172.21.15.60:0/596680851 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f035810b6e0 msgr2=0x7f035810bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T15:04:57.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:57.093+0000 7f035f600700 1 --2- 172.21.15.60:0/596680851 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f035810b6e0 0x7f035810bac0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f034c0098f0 tx=0x7f034c009c00 comp rx=0 tx=0).stop 2024-05-03T15:04:57.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:57.093+0000 7f035f600700 1 -- 172.21.15.60:0/596680851 shutdown_connections 2024-05-03T15:04:57.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:57.093+0000 7f035f600700 1 --2- 172.21.15.60:0/596680851 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03581060a0 0x7f03581125b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:04:57.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:57.093+0000 7f035f600700 1 --2- 172.21.15.60:0/596680851 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f035810b6e0 0x7f035810bac0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:04:57.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:57.093+0000 7f035f600700 1 --2- 172.21.15.60:0/596680851 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03581056e0 0x7f0358105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:04:57.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:57.093+0000 7f035f600700 1 -- 172.21.15.60:0/596680851 >> 172.21.15.60:0/596680851 conn(0x7f03581011d0 msgr2=0x7f03581035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T15:04:57.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:57.094+0000 7f035f600700 1 -- 172.21.15.60:0/596680851 shutdown_connections 2024-05-03T15:04:57.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:57.094+0000 7f035f600700 1 -- 172.21.15.60:0/596680851 wait complete. 2024-05-03T15:04:57.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:57.095+0000 7f035f600700 1 Processor -- start 2024-05-03T15:04:57.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:57.095+0000 7f035f600700 1 -- start start 2024-05-03T15:04:57.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:57.095+0000 7f035f600700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03581056e0 0x7f035819bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T15:04:57.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:57.095+0000 7f035f600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03581060a0 0x7f035819c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T15:04:57.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:57.095+0000 7f035f600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f035810b6e0 0x7f03581a0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T15:04:57.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:57.095+0000 7f035f600700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f0358115b50 con 0x7f03581056e0 2024-05-03T15:04:57.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:57.095+0000 7f035f600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f03581159d0 con 0x7f035810b6e0 2024-05-03T15:04:57.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:57.095+0000 7f035f600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0358115cd0 con 0x7f03581060a0 2024-05-03T15:04:57.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:57.096+0000 7f035cc00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03581056e0 0x7f035819bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T15:04:57.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:57.096+0000 7f035cc00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03581056e0 0x7f035819bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57904/0 (socket says 172.21.15.60:57904) 2024-05-03T15:04:57.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:57.096+0000 7f035cc00700 1 -- 172.21.15.60:0/239247714 learned_addr learned my addr 172.21.15.60:0/239247714 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T15:04:57.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:57.096+0000 7f0357e00700 1 --2- 172.21.15.60:0/239247714 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03581060a0 0x7f035819c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T15:04:57.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:57.096+0000 7f035d600700 1 --2- 172.21.15.60:0/239247714 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f035810b6e0 0x7f03581a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T15:04:57.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:57.096+0000 7f035cc00700 1 -- 172.21.15.60:0/239247714 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03581060a0 msgr2=0x7f035819c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T15:04:57.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:57.096+0000 7f035cc00700 1 --2- 172.21.15.60:0/239247714 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03581060a0 0x7f035819c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:04:57.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:57.096+0000 7f035cc00700 1 -- 172.21.15.60:0/239247714 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f035810b6e0 msgr2=0x7f03581a0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T15:04:57.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:57.096+0000 7f035cc00700 1 --2- 172.21.15.60:0/239247714 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f035810b6e0 0x7f03581a0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:04:57.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:57.096+0000 7f035cc00700 1 -- 172.21.15.60:0/239247714 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03581a0f50 con 0x7f03581056e0 2024-05-03T15:04:57.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:57.096+0000 7f0357e00700 1 --2- 172.21.15.60:0/239247714 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03581060a0 0x7f035819c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T15:04:57.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:57.097+0000 7f035cc00700 1 --2- 172.21.15.60:0/239247714 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03581056e0 0x7f035819bf40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f034c009230 tx=0x7f034c010f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T15:04:57.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:57.097+0000 7f0355600700 1 -- 172.21.15.60:0/239247714 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034c027070 con 0x7f03581056e0 2024-05-03T15:04:57.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:57.097+0000 7f0355600700 1 -- 172.21.15.60:0/239247714 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f034c00bc40 con 0x7f03581056e0 2024-05-03T15:04:57.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:57.097+0000 7f035f600700 1 -- 172.21.15.60:0/239247714 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03581a11e0 con 0x7f03581056e0 2024-05-03T15:04:57.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:57.097+0000 7f0355600700 1 -- 172.21.15.60:0/239247714 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034c004a50 con 0x7f03581056e0 2024-05-03T15:04:57.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:57.097+0000 7f035f600700 1 -- 172.21.15.60:0/239247714 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03581a8ac0 con 0x7f03581056e0 2024-05-03T15:04:57.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:57.099+0000 7f035f600700 1 -- 172.21.15.60:0/239247714 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f035810dc70 con 0x7f03581056e0 2024-05-03T15:04:57.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:57.101+0000 7f0355600700 1 -- 172.21.15.60:0/239247714 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f034c02a070 con 0x7f03581056e0 2024-05-03T15:04:57.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:57.102+0000 7f0355600700 1 --2- 172.21.15.60:0/239247714 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f03400b2990 0x7f03400b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T15:04:57.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:57.102+0000 7f0355600700 1 -- 172.21.15.60:0/239247714 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f034c0e3760 con 0x7f03581056e0 2024-05-03T15:04:57.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:57.103+0000 7f0357e00700 1 --2- 172.21.15.60:0/239247714 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f03400b2990 0x7f03400b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T15:04:57.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:57.103+0000 7f0357e00700 1 --2- 172.21.15.60:0/239247714 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f03400b2990 0x7f03400b4e50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0358103760 tx=0x7f0344009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T15:04:57.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:57.105+0000 7f0355600700 1 -- 172.21.15.60:0/239247714 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f034c031070 con 0x7f03581056e0 2024-05-03T15:04:57.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:04:57 smithi060 ceph-mon[28873]: pgmap v945: 129 pgs: 129 active+clean; 649 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 767 B/s wr, 11 op/s 2024-05-03T15:04:57.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:57.341+0000 7f035f600700 1 -- 172.21.15.60:0/239247714 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f035819d2a0 con 0x7f03581056e0 2024-05-03T15:04:57.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:57.344+0000 7f0355600700 1 -- 172.21.15.60:0/239247714 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24674 (secure 0 0 0) 0x7f034c013030 con 0x7f03581056e0 2024-05-03T15:04:57.346 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-05-03T15:04:57.347 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":89,"fsid":"285d64b2-095a-11ef-bc97-c7b262605968","created":"2024-05-03T14:34:53.560460+0000","modified":"2024-05-03T14:46:05.582395+0000","last_up_change":"2024-05-03T14:43:14.877043+0000","last_in_change":"2024-05-03T14:42:55.085959+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-03T14:38:54.333714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-03T14:44:25.608599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-03T14:44:30.154009+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-03T14:44:42.549502+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f0797730-c4c1-47ea-9dce-9270d5ebb4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2952930162},{"type":"v1","addr":"172.21.15.60:6803","nonce":2952930162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2952930162},{"type":"v1","addr":"172.21.15.60:6805","nonce":2952930162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":2952930162},{"type":"v1","addr":"172.21.15.60:6809","nonce":2952930162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2952930162},{"type":"v1","addr":"172.21.15.60:6807","nonce":2952930162}]},"public_addr":"172.21.15.60:6803/2952930162","cluster_addr":"172.21.15.60:6805/2952930162","heartbeat_back_addr":"172.21.15.60:6809/2952930162","heartbeat_front_addr":"172.21.15.60:6807/2952930162","state":["exists","up"]},{"osd":1,"uuid":"96d0ad08-cda4-4824-bad4-e64973508c0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":720063242},{"type":"v1","addr":"172.21.15.90:6801","nonce":720063242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":720063242},{"type":"v1","addr":"172.21.15.90:6803","nonce":720063242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":720063242},{"type":"v1","addr":"172.21.15.90:6807","nonce":720063242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":720063242},{"type":"v1","addr":"172.21.15.90:6805","nonce":720063242}]},"public_addr":"172.21.15.90:6801/720063242","cluster_addr":"172.21.15.90:6803/720063242","heartbeat_back_addr":"172.21.15.90:6807/720063242","heartbeat_front_addr":"172.21.15.90:6805/720063242","state":["exists","up"]},{"osd":2,"uuid":"ce95fb7c-1611-4dd1-aee7-97c420d6a466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1359662077},{"type":"v1","addr":"172.21.15.149:6801","nonce":1359662077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1359662077},{"type":"v1","addr":"172.21.15.149:6803","nonce":1359662077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1359662077},{"type":"v1","addr":"172.21.15.149:6807","nonce":1359662077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1359662077},{"type":"v1","addr":"172.21.15.149:6805","nonce":1359662077}]},"public_addr":"172.21.15.149:6801/1359662077","cluster_addr":"172.21.15.149:6803/1359662077","heartbeat_back_addr":"172.21.15.149:6807/1359662077","heartbeat_front_addr":"172.21.15.149:6805/1359662077","state":["exists","up"]},{"osd":3,"uuid":"ef31136a-9b36-4b87-8389-f53d311bc5cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3226133334},{"type":"v1","addr":"172.21.15.60:6811","nonce":3226133334}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3226133334},{"type":"v1","addr":"172.21.15.60:6813","nonce":3226133334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3226133334},{"type":"v1","addr":"172.21.15.60:6817","nonce":3226133334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3226133334},{"type":"v1","addr":"172.21.15.60:6815","nonce":3226133334}]},"public_addr":"172.21.15.60:6811/3226133334","cluster_addr":"172.21.15.60:6813/3226133334","heartbeat_back_addr":"172.21.15.60:6817/3226133334","heartbeat_front_addr":"172.21.15.60:6815/3226133334","state":["exists","up"]},{"osd":4,"uuid":"73db7408-a2ff-47e9-b13d-d31688a93610","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2201391883},{"type":"v1","addr":"172.21.15.90:6809","nonce":2201391883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":2201391883},{"type":"v1","addr":"172.21.15.90:6811","nonce":2201391883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":2201391883},{"type":"v1","addr":"172.21.15.90:6815","nonce":2201391883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":2201391883},{"type":"v1","addr":"172.21.15.90:6813","nonce":2201391883}]},"public_addr":"172.21.15.90:6809/2201391883","cluster_addr":"172.21.15.90:6811/2201391883","heartbeat_back_addr":"172.21.15.90:6815/2201391883","heartbeat_front_addr":"172.21.15.90:6813/2201391883","state":["exists","up"]},{"osd":5,"uuid":"aefb78a2-522a-4b35-92b8-e5c04fa47b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2139732836},{"type":"v1","addr":"172.21.15.149:6809","nonce":2139732836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":2139732836},{"type":"v1","addr":"172.21.15.149:6811","nonce":2139732836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":2139732836},{"type":"v1","addr":"172.21.15.149:6815","nonce":2139732836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":2139732836},{"type":"v1","addr":"172.21.15.149:6813","nonce":2139732836}]},"public_addr":"172.21.15.149:6809/2139732836","cluster_addr":"172.21.15.149:6811/2139732836","heartbeat_back_addr":"172.21.15.149:6815/2139732836","heartbeat_front_addr":"172.21.15.149:6813/2139732836","state":["exists","up"]},{"osd":6,"uuid":"7c887819-6182-4636-8cfd-d845c30bd13c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3738521002},{"type":"v1","addr":"172.21.15.60:6819","nonce":3738521002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3738521002},{"type":"v1","addr":"172.21.15.60:6821","nonce":3738521002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3738521002},{"type":"v1","addr":"172.21.15.60:6825","nonce":3738521002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3738521002},{"type":"v1","addr":"172.21.15.60:6823","nonce":3738521002}]},"public_addr":"172.21.15.60:6819/3738521002","cluster_addr":"172.21.15.60:6821/3738521002","heartbeat_back_addr":"172.21.15.60:6825/3738521002","heartbeat_front_addr":"172.21.15.60:6823/3738521002","state":["exists","up"]},{"osd":7,"uuid":"ebce6acf-8873-4b35-bd5f-80de2163a307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":1686298661},{"type":"v1","addr":"172.21.15.90:6817","nonce":1686298661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":1686298661},{"type":"v1","addr":"172.21.15.90:6819","nonce":1686298661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":1686298661},{"type":"v1","addr":"172.21.15.90:6823","nonce":1686298661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1686298661},{"type":"v1","addr":"172.21.15.90:6821","nonce":1686298661}]},"public_addr":"172.21.15.90:6817/1686298661","cluster_addr":"172.21.15.90:6819/1686298661","heartbeat_back_addr":"172.21.15.90:6823/1686298661","heartbeat_front_addr":"172.21.15.90:6821/1686298661","state":["exists","up"]},{"osd":8,"uuid":"4ae04b98-e9b2-4956-83d9-17e5ac3c963f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":2608348248},{"type":"v1","addr":"172.21.15.149:6817","nonce":2608348248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2608348248},{"type":"v1","addr":"172.21.15.149:6819","nonce":2608348248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2608348248},{"type":"v1","addr":"172.21.15.149:6823","nonce":2608348248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2608348248},{"type":"v1","addr":"172.21.15.149:6821","nonce":2608348248}]},"public_addr":"172.21.15.149:6817/2608348248","cluster_addr":"172.21.15.149:6819/2608348248","heartbeat_back_addr":"172.21.15.149:6823/2608348248","heartbeat_front_addr":"172.21.15.149:6821/2608348248","state":["exists","up"]},{"osd":9,"uuid":"d0c0a70d-aa76-47e4-91e8-c343c8c41197","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":2156130999},{"type":"v1","addr":"172.21.15.60:6827","nonce":2156130999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":2156130999},{"type":"v1","addr":"172.21.15.60:6829","nonce":2156130999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":2156130999},{"type":"v1","addr":"172.21.15.60:6833","nonce":2156130999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":2156130999},{"type":"v1","addr":"172.21.15.60:6831","nonce":2156130999}]},"public_addr":"172.21.15.60:6827/2156130999","cluster_addr":"172.21.15.60:6829/2156130999","heartbeat_back_addr":"172.21.15.60:6833/2156130999","heartbeat_front_addr":"172.21.15.60:6831/2156130999","state":["exists","up"]},{"osd":10,"uuid":"cdc3d7b8-33c7-4cc4-be4e-6887d15a3d8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":2492593590},{"type":"v1","addr":"172.21.15.90:6825","nonce":2492593590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":2492593590},{"type":"v1","addr":"172.21.15.90:6827","nonce":2492593590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":2492593590},{"type":"v1","addr":"172.21.15.90:6831","nonce":2492593590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":2492593590},{"type":"v1","addr":"172.21.15.90:6829","nonce":2492593590}]},"public_addr":"172.21.15.90:6825/2492593590","cluster_addr":"172.21.15.90:6827/2492593590","heartbeat_back_addr":"172.21.15.90:6831/2492593590","heartbeat_front_addr":"172.21.15.90:6829/2492593590","state":["exists","up"]},{"osd":11,"uuid":"8d17ec1a-bdd5-41f1-9b59-c2b6a527bfbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":1063245126},{"type":"v1","addr":"172.21.15.149:6825","nonce":1063245126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":1063245126},{"type":"v1","addr":"172.21.15.149:6827","nonce":1063245126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":1063245126},{"type":"v1","addr":"172.21.15.149:6831","nonce":1063245126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":1063245126},{"type":"v1","addr":"172.21.15.149:6829","nonce":1063245126}]},"public_addr":"172.21.15.149:6825/1063245126","cluster_addr":"172.21.15.149:6827/1063245126","heartbeat_back_addr":"172.21.15.149:6831/1063245126","heartbeat_front_addr":"172.21.15.149:6829/1063245126","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:38:02.056563+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:38:27.140677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:38:51.287070+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:39:25.484410+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:39:51.483133+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:40:17.200423+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:40:51.979804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:41:18.742542+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:41:44.922541+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:42:19.778451+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:42:47.438440+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T14:43:13.429419+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6801/424391417":"2024-05-04T14:36:04.154333+0000","172.21.15.60:6800/424391417":"2024-05-04T14:36:04.154333+0000","172.21.15.60:0/1426532561":"2024-05-04T14:36:04.154333+0000","172.21.15.60:0/3230079926":"2024-05-04T14:35:28.066961+0000","172.21.15.60:0/2334895610":"2024-05-04T14:35:28.066961+0000","172.21.15.60:6800/888447185":"2024-05-04T14:35:28.066961+0000","172.21.15.60:0/3996899937":"2024-05-04T14:35:28.066961+0000","172.21.15.60:0/2779644018":"2024-05-04T14:36:04.154333+0000","172.21.15.60:0/71440838":"2024-05-04T14:36:04.154333+0000","172.21.15.60:6801/888447185":"2024-05-04T14:35:28.066961+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-03T15:04:57.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:57.347+0000 7f035f600700 1 -- 172.21.15.60:0/239247714 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f03400b2990 msgr2=0x7f03400b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T15:04:57.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:57.347+0000 7f035f600700 1 --2- 172.21.15.60:0/239247714 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f03400b2990 0x7f03400b4e50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0358103760 tx=0x7f0344009310 comp rx=0 tx=0).stop 2024-05-03T15:04:57.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:57.347+0000 7f035f600700 1 -- 172.21.15.60:0/239247714 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03581056e0 msgr2=0x7f035819bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T15:04:57.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:57.347+0000 7f035f600700 1 --2- 172.21.15.60:0/239247714 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03581056e0 0x7f035819bf40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f034c009230 tx=0x7f034c010f80 comp rx=0 tx=0).stop 2024-05-03T15:04:57.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:57.347+0000 7f035f600700 1 -- 172.21.15.60:0/239247714 shutdown_connections 2024-05-03T15:04:57.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:57.347+0000 7f035f600700 1 --2- 172.21.15.60:0/239247714 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f03400b2990 0x7f03400b4e50 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:04:57.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:57.347+0000 7f035f600700 1 --2- 172.21.15.60:0/239247714 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f035810b6e0 0x7f03581a0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:04:57.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:57.347+0000 7f035f600700 1 --2- 172.21.15.60:0/239247714 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03581056e0 0x7f035819bf40 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:04:57.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:57.347+0000 7f035f600700 1 --2- 172.21.15.60:0/239247714 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03581060a0 0x7f035819c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:04:57.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:57.347+0000 7f035f600700 1 -- 172.21.15.60:0/239247714 >> 172.21.15.60:0/239247714 conn(0x7f03581011d0 msgr2=0x7f0358101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T15:04:57.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:57.348+0000 7f035f600700 1 -- 172.21.15.60:0/239247714 shutdown_connections 2024-05-03T15:04:57.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:04:57.348+0000 7f035f600700 1 -- 172.21.15.60:0/239247714 wait complete. 2024-05-03T15:04:57.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:04:57 smithi090 ceph-mon[33568]: pgmap v945: 129 pgs: 129 active+clean; 649 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 767 B/s wr, 11 op/s 2024-05-03T15:04:57.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:04:57 smithi149 ceph-mon[33332]: pgmap v945: 129 pgs: 129 active+clean; 649 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 767 B/s wr, 11 op/s 2024-05-03T15:04:58.158 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph fs fail cephfs 2024-05-03T15:04:58.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:04:58 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/239247714' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T15:04:58.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:04:58 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/239247714' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T15:04:58.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:04:58 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/239247714' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T15:04:58.633 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T15:04:59.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:04:59 smithi090 ceph-mon[33568]: pgmap v946: 129 pgs: 129 active+clean; 649 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 10 op/s 2024-05-03T15:04:59.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:04:59 smithi149 ceph-mon[33332]: pgmap v946: 129 pgs: 129 active+clean; 649 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 10 op/s 2024-05-03T15:04:59.539 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:04:59 smithi060 ceph-mon[28873]: pgmap v946: 129 pgs: 129 active+clean; 649 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 10 op/s 2024-05-03T15:05:00.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:00.751+0000 7f097c600700 1 -- 172.21.15.60:0/3714537910 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0974109ad0 msgr2=0x7f0974109f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T15:05:00.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:00.751+0000 7f097c600700 1 --2- 172.21.15.60:0/3714537910 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0974109ad0 0x7f0974109f50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f09600098f0 tx=0x7f0960009c00 comp rx=0 tx=0).stop 2024-05-03T15:05:00.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:00.751+0000 7f097c600700 1 -- 172.21.15.60:0/3714537910 shutdown_connections 2024-05-03T15:05:00.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:00.751+0000 7f097c600700 1 --2- 172.21.15.60:0/3714537910 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f097410a490 0x7f09741169a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:05:00.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:00.751+0000 7f097c600700 1 --2- 172.21.15.60:0/3714537910 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f097410fb60 0x7f097410ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:05:00.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:00.751+0000 7f097c600700 1 --2- 172.21.15.60:0/3714537910 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0974109ad0 0x7f0974109f50 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:05:00.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:00.751+0000 7f097c600700 1 -- 172.21.15.60:0/3714537910 >> 172.21.15.60:0/3714537910 conn(0x7f0974076c70 msgr2=0x7f09741082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T15:05:00.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:00.752+0000 7f097c600700 1 -- 172.21.15.60:0/3714537910 shutdown_connections 2024-05-03T15:05:00.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:00.752+0000 7f097c600700 1 -- 172.21.15.60:0/3714537910 wait complete. 2024-05-03T15:05:00.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:00.753+0000 7f097c600700 1 Processor -- start 2024-05-03T15:05:00.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:00.753+0000 7f097c600700 1 -- start start 2024-05-03T15:05:00.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:00.754+0000 7f097c600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0974109ad0 0x7f09741a0350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T15:05:00.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:00.754+0000 7f097c600700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f097410a490 0x7f09741a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T15:05:00.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:00.754+0000 7f097c600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f097410fb60 0x7f09741a4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T15:05:00.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:00.754+0000 7f097c600700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f0974119f40 con 0x7f097410a490 2024-05-03T15:05:00.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:00.754+0000 7f097c600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0974119dc0 con 0x7f097410fb60 2024-05-03T15:05:00.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:00.754+0000 7f097c600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f097411a0c0 con 0x7f0974109ad0 2024-05-03T15:05:00.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:00.754+0000 7f0979c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0974109ad0 0x7f09741a0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T15:05:00.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:00.754+0000 7f0979c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0974109ad0 0x7f09741a0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.60:32970/0 (socket says 172.21.15.60:32970) 2024-05-03T15:05:00.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:00.754+0000 7f0979c00700 1 -- 172.21.15.60:0/1578748625 learned_addr learned my addr 172.21.15.60:0/1578748625 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T15:05:00.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:00.754+0000 7f097a600700 1 --2- 172.21.15.60:0/1578748625 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f097410fb60 0x7f09741a4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T15:05:00.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:00.755+0000 7f0979200700 1 --2- 172.21.15.60:0/1578748625 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f097410a490 0x7f09741a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T15:05:00.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:00.755+0000 7f0979c00700 1 -- 172.21.15.60:0/1578748625 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f097410fb60 msgr2=0x7f09741a4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T15:05:00.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:00.755+0000 7f0979c00700 1 --2- 172.21.15.60:0/1578748625 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f097410fb60 0x7f09741a4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:05:00.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:00.755+0000 7f0979c00700 1 -- 172.21.15.60:0/1578748625 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f097410a490 msgr2=0x7f09741a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T15:05:00.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:00.755+0000 7f0979c00700 1 --2- 172.21.15.60:0/1578748625 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f097410a490 0x7f09741a0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:05:00.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:00.755+0000 7f0979c00700 1 -- 172.21.15.60:0/1578748625 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09741a5310 con 0x7f0974109ad0 2024-05-03T15:05:00.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:00.755+0000 7f0979c00700 1 --2- 172.21.15.60:0/1578748625 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0974109ad0 0x7f09741a0350 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f096800e9a0 tx=0x7f096800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T15:05:00.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:00.756+0000 7f0972000700 1 -- 172.21.15.60:0/1578748625 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096800ca10 con 0x7f0974109ad0 2024-05-03T15:05:00.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:00.756+0000 7f097c600700 1 -- 172.21.15.60:0/1578748625 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09741a5600 con 0x7f0974109ad0 2024-05-03T15:05:00.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:00.756+0000 7f0972000700 1 -- 172.21.15.60:0/1578748625 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f096800cbb0 con 0x7f0974109ad0 2024-05-03T15:05:00.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:00.756+0000 7f0972000700 1 -- 172.21.15.60:0/1578748625 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09680187a0 con 0x7f0974109ad0 2024-05-03T15:05:00.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:00.756+0000 7f097c600700 1 -- 172.21.15.60:0/1578748625 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09741ace80 con 0x7f0974109ad0 2024-05-03T15:05:00.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:00.757+0000 7f097c600700 1 -- 172.21.15.60:0/1578748625 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f097406a190 con 0x7f0974109ad0 2024-05-03T15:05:00.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:00.760+0000 7f0972000700 1 -- 172.21.15.60:0/1578748625 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f0968010430 con 0x7f0974109ad0 2024-05-03T15:05:00.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:00.761+0000 7f0972000700 1 --2- 172.21.15.60:0/1578748625 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f095c0b2990 0x7f095c0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T15:05:00.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:00.761+0000 7f0972000700 1 -- 172.21.15.60:0/1578748625 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f09680e2a40 con 0x7f0974109ad0 2024-05-03T15:05:00.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:00.761+0000 7f0979200700 1 --2- 172.21.15.60:0/1578748625 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f095c0b2990 0x7f095c0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T15:05:00.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:00.762+0000 7f0979200700 1 --2- 172.21.15.60:0/1578748625 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f095c0b2990 0x7f095c0b4e50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f09600098c0 tx=0x7f0960019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T15:05:00.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:00.763+0000 7f0972000700 1 -- 172.21.15.60:0/1578748625 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f09680aa220 con 0x7f0974109ad0 2024-05-03T15:05:01.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:01.057+0000 7f097c600700 1 -- 172.21.15.60:0/1578748625 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f097402e510 con 0x7f0974109ad0 2024-05-03T15:05:01.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:01.071+0000 7f0972000700 1 -- 172.21.15.60:0/1578748625 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v31) ==== 161+0+0 (secure 0 0 0) 0x7f09680b22a0 con 0x7f0974109ad0 2024-05-03T15:05:01.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:01.075+0000 7f097c600700 1 -- 172.21.15.60:0/1578748625 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f095c0b2990 msgr2=0x7f095c0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T15:05:01.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:01.075+0000 7f097c600700 1 --2- 172.21.15.60:0/1578748625 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f095c0b2990 0x7f095c0b4e50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f09600098c0 tx=0x7f0960019040 comp rx=0 tx=0).stop 2024-05-03T15:05:01.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:01.075+0000 7f097c600700 1 -- 172.21.15.60:0/1578748625 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0974109ad0 msgr2=0x7f09741a0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T15:05:01.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:01.075+0000 7f097c600700 1 --2- 172.21.15.60:0/1578748625 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0974109ad0 0x7f09741a0350 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f096800e9a0 tx=0x7f096800ecb0 comp rx=0 tx=0).stop 2024-05-03T15:05:01.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:01.075+0000 7f097c600700 1 -- 172.21.15.60:0/1578748625 shutdown_connections 2024-05-03T15:05:01.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:01.075+0000 7f097c600700 1 --2- 172.21.15.60:0/1578748625 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f095c0b2990 0x7f095c0b4e50 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:05:01.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:01.075+0000 7f097c600700 1 --2- 172.21.15.60:0/1578748625 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f097410fb60 0x7f09741a4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:05:01.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:01.075+0000 7f097c600700 1 --2- 172.21.15.60:0/1578748625 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0974109ad0 0x7f09741a0350 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:05:01.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:01.075+0000 7f097c600700 1 --2- 172.21.15.60:0/1578748625 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f097410a490 0x7f09741a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:05:01.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:01.075+0000 7f097c600700 1 -- 172.21.15.60:0/1578748625 >> 172.21.15.60:0/1578748625 conn(0x7f0974076c70 msgr2=0x7f0974077230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T15:05:01.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:01.076+0000 7f097c600700 1 -- 172.21.15.60:0/1578748625 shutdown_connections 2024-05-03T15:05:01.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:01.076+0000 7f097c600700 1 -- 172.21.15.60:0/1578748625 wait complete. 2024-05-03T15:05:01.079 INFO:teuthology.orchestra.run.smithi060.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-03T15:05:01.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:01 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mon-a[28869]: 2024-05-03T15:05:01.059+0000 7fc7c3a00700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-03T15:05:01.338 INFO:journalctl@ceph.mds.g.smithi060.stdout:May 03 15:05:01 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-g[67852]: ignoring --setuser ceph since I am not root 2024-05-03T15:05:01.338 INFO:journalctl@ceph.mds.g.smithi060.stdout:May 03 15:05:01 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-g[67852]: ignoring --setgroup ceph since I am not root 2024-05-03T15:05:01.338 INFO:journalctl@ceph.mds.g.smithi060.stdout:May 03 15:05:01 smithi060 ceph-mds[67856]: -- 172.21.15.60:0/3565281757 <== mon.2 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5616acfada40 con 0x5616acf80800 2024-05-03T15:05:01.338 INFO:journalctl@ceph.mds.g.smithi060.stdout:May 03 15:05:01 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-g[67852]: starting mds.g at 2024-05-03T15:05:01.339 INFO:journalctl@ceph.mds.a.smithi060.stdout:May 03 15:05:01 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-a[68311]: ignoring --setuser ceph since I am not root 2024-05-03T15:05:01.339 INFO:journalctl@ceph.mds.a.smithi060.stdout:May 03 15:05:01 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-a[68311]: ignoring --setgroup ceph since I am not root 2024-05-03T15:05:01.339 INFO:journalctl@ceph.mds.a.smithi060.stdout:May 03 15:05:01 smithi060 ceph-mds[68315]: -- 172.21.15.60:0/3012866176 <== mon.0 v2:172.21.15.60:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561cee007a40 con 0x561cedfda800 2024-05-03T15:05:01.339 INFO:journalctl@ceph.mds.a.smithi060.stdout:May 03 15:05:01 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-a[68311]: starting mds.a at 2024-05-03T15:05:01.374 INFO:journalctl@ceph.mds.e.smithi090.stdout:May 03 15:05:01 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-e[63100]: ignoring --setuser ceph since I am not root 2024-05-03T15:05:01.374 INFO:journalctl@ceph.mds.e.smithi090.stdout:May 03 15:05:01 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-e[63100]: ignoring --setgroup ceph since I am not root 2024-05-03T15:05:01.375 INFO:journalctl@ceph.mds.e.smithi090.stdout:May 03 15:05:01 smithi090 ceph-mds[63104]: -- 172.21.15.90:0/1568864144 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560682a6fa40 con 0x560682a42800 2024-05-03T15:05:01.375 INFO:journalctl@ceph.mds.e.smithi090.stdout:May 03 15:05:01 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-e[63100]: starting mds.e at 2024-05-03T15:05:01.376 INFO:journalctl@ceph.mds.k.smithi090.stdout:May 03 15:05:01 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-k[62785]: ignoring --setuser ceph since I am not root 2024-05-03T15:05:01.376 INFO:journalctl@ceph.mds.k.smithi090.stdout:May 03 15:05:01 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-k[62785]: ignoring --setgroup ceph since I am not root 2024-05-03T15:05:01.376 INFO:journalctl@ceph.mds.k.smithi090.stdout:May 03 15:05:01 smithi090 ceph-mds[62789]: -- 172.21.15.90:0/2184298729 <== mon.2 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5603479e9a40 con 0x5603479bc800 2024-05-03T15:05:01.376 INFO:journalctl@ceph.mds.k.smithi090.stdout:May 03 15:05:01 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-k[62785]: starting mds.k at 2024-05-03T15:05:01.376 INFO:journalctl@ceph.mds.b.smithi090.stdout:May 03 15:05:01 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-b[63731]: ignoring --setuser ceph since I am not root 2024-05-03T15:05:01.377 INFO:journalctl@ceph.mds.b.smithi090.stdout:May 03 15:05:01 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-b[63731]: ignoring --setgroup ceph since I am not root 2024-05-03T15:05:01.377 INFO:journalctl@ceph.mds.b.smithi090.stdout:May 03 15:05:01 smithi090 ceph-mds[63735]: -- 172.21.15.90:0/2753893677 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5560c84b1a40 con 0x5560c8484800 2024-05-03T15:05:01.377 INFO:journalctl@ceph.mds.b.smithi090.stdout:May 03 15:05:01 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-b[63731]: starting mds.b at 2024-05-03T15:05:01.378 INFO:journalctl@ceph.mds.h.smithi090.stdout:May 03 15:05:01 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-h[63425]: ignoring --setuser ceph since I am not root 2024-05-03T15:05:01.378 INFO:journalctl@ceph.mds.h.smithi090.stdout:May 03 15:05:01 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-h[63425]: ignoring --setgroup ceph since I am not root 2024-05-03T15:05:01.378 INFO:journalctl@ceph.mds.h.smithi090.stdout:May 03 15:05:01 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-h[63425]: starting mds.h at 2024-05-03T15:05:01.422 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 03 15:05:01 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-c[64220]: ignoring --setuser ceph since I am not root 2024-05-03T15:05:01.422 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 03 15:05:01 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-c[64220]: ignoring --setgroup ceph since I am not root 2024-05-03T15:05:01.423 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 03 15:05:01 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-c[64220]: starting mds.c at 2024-05-03T15:05:01.423 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 03 15:05:01 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-l[63595]: ignoring --setuser ceph since I am not root 2024-05-03T15:05:01.423 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 03 15:05:01 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-l[63595]: ignoring --setgroup ceph since I am not root 2024-05-03T15:05:01.423 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 03 15:05:01 smithi149 ceph-mds[63599]: -- 172.21.15.149:0/3080087299 <== mon.2 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56474591fa40 con 0x5647458f2800 2024-05-03T15:05:01.423 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 03 15:05:01 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-l[63595]: starting mds.l at 2024-05-03T15:05:01.424 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 03 15:05:01 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-f[63919]: ignoring --setuser ceph since I am not root 2024-05-03T15:05:01.424 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 03 15:05:01 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-f[63919]: ignoring --setgroup ceph since I am not root 2024-05-03T15:05:01.424 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 03 15:05:01 smithi149 ceph-mds[63923]: -- 172.21.15.149:0/2645695170 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x555768bf04e0 con 0x555768bc7000 2024-05-03T15:05:01.424 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 03 15:05:01 smithi149 ceph-mds[63923]: -- 172.21.15.149:0/2645695170 <== mon.2 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555768bf3a40 con 0x555768bc7000 2024-05-03T15:05:01.424 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 03 15:05:01 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-f[63919]: starting mds.f at 2024-05-03T15:05:01.425 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 03 15:05:01 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-i[64526]: ignoring --setuser ceph since I am not root 2024-05-03T15:05:01.425 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 03 15:05:01 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-i[64526]: ignoring --setgroup ceph since I am not root 2024-05-03T15:05:01.425 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 03 15:05:01 smithi149 ceph-mds[64530]: -- 172.21.15.149:0/2666580838 <== mon.2 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5591762f3a40 con 0x5591762c7000 2024-05-03T15:05:01.425 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 03 15:05:01 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-i[64526]: starting mds.i at 2024-05-03T15:05:01.943 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-05-03T15:05:02.275 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:02 smithi090 ceph-mon[33568]: pgmap v947: 129 pgs: 129 active+clean; 649 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 511 B/s wr, 12 op/s 2024-05-03T15:05:02.276 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:02 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/1578748625' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-03T15:05:02.276 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:02 smithi090 ceph-mon[33568]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-03T15:05:02.276 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:02 smithi090 ceph-mon[33568]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-03T15:05:02.276 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:02 smithi090 ceph-mon[33568]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-03T15:05:02.276 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:02 smithi090 ceph-mon[33568]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-03T15:05:02.276 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:02 smithi090 ceph-mon[33568]: osdmap e90: 12 total, 12 up, 12 in 2024-05-03T15:05:02.276 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:02 smithi090 ceph-mon[33568]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-03T15:05:02.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:02 smithi060 ceph-mon[28873]: pgmap v947: 129 pgs: 129 active+clean; 649 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 511 B/s wr, 12 op/s 2024-05-03T15:05:02.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:02 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/1578748625' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-03T15:05:02.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:02 smithi060 ceph-mon[28873]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-03T15:05:02.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:02 smithi060 ceph-mon[28873]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-03T15:05:02.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:02 smithi060 ceph-mon[28873]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-03T15:05:02.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:02 smithi060 ceph-mon[28873]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-03T15:05:02.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:02 smithi060 ceph-mon[28873]: osdmap e90: 12 total, 12 up, 12 in 2024-05-03T15:05:02.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:02 smithi060 ceph-mon[28873]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-03T15:05:02.419 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T15:05:02.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:02 smithi149 ceph-mon[33332]: pgmap v947: 129 pgs: 129 active+clean; 649 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 511 B/s wr, 12 op/s 2024-05-03T15:05:02.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:02 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/1578748625' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-03T15:05:02.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:02 smithi149 ceph-mon[33332]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-03T15:05:02.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:02 smithi149 ceph-mon[33332]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-03T15:05:02.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:02 smithi149 ceph-mon[33332]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-03T15:05:02.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:02 smithi149 ceph-mon[33332]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-03T15:05:02.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:02 smithi149 ceph-mon[33332]: osdmap e90: 12 total, 12 up, 12 in 2024-05-03T15:05:02.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:02 smithi149 ceph-mon[33332]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-03T15:05:03.297 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:03 smithi060 ceph-mon[28873]: mds.? [v2:172.21.15.90:6833/1790214955,v1:172.21.15.90:6835/1790214955] up:boot 2024-05-03T15:05:03.297 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:03 smithi060 ceph-mon[28873]: mds.? [v2:172.21.15.90:6832/233486054,v1:172.21.15.90:6834/233486054] up:boot 2024-05-03T15:05:03.298 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:03 smithi060 ceph-mon[28873]: mds.? [v2:172.21.15.149:6834/50709348,v1:172.21.15.149:6838/50709348] up:boot 2024-05-03T15:05:03.298 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:03 smithi060 ceph-mon[28873]: mds.? [v2:172.21.15.149:6832/3041052525,v1:172.21.15.149:6837/3041052525] up:boot 2024-05-03T15:05:03.298 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:03 smithi060 ceph-mon[28873]: mds.? [v2:172.21.15.149:6833/523430178,v1:172.21.15.149:6836/523430178] up:boot 2024-05-03T15:05:03.298 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:03 smithi060 ceph-mon[28873]: mds.? [v2:172.21.15.149:6835/4163424965,v1:172.21.15.149:6839/4163424965] up:boot 2024-05-03T15:05:03.299 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:03 smithi060 ceph-mon[28873]: mds.? [v2:172.21.15.90:6836/778709002,v1:172.21.15.90:6838/778709002] up:boot 2024-05-03T15:05:03.299 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:03 smithi060 ceph-mon[28873]: mds.? [v2:172.21.15.90:6837/2598565721,v1:172.21.15.90:6839/2598565721] up:boot 2024-05-03T15:05:03.299 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:03 smithi060 ceph-mon[28873]: mds.? [v2:172.21.15.60:6838/1167209231,v1:172.21.15.60:6839/1167209231] up:boot 2024-05-03T15:05:03.299 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:03 smithi060 ceph-mon[28873]: mds.? [v2:172.21.15.60:6840/720135337,v1:172.21.15.60:6841/720135337] up:boot 2024-05-03T15:05:03.299 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:03 smithi060 ceph-mon[28873]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-05-03T15:05:03.300 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:03 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-03T15:05:03.300 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:03 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-03T15:05:03.300 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:03 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-03T15:05:03.300 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:03 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-03T15:05:03.301 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:03 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-03T15:05:03.301 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:03 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-03T15:05:03.301 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:03 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-03T15:05:03.301 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:03 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-03T15:05:03.302 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:03 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-03T15:05:03.302 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:03 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-03T15:05:03.302 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:03 smithi060 ceph-mon[28873]: pgmap v949: 129 pgs: 129 active+clean; 649 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-05-03T15:05:03.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:03 smithi090 ceph-mon[33568]: mds.? [v2:172.21.15.90:6833/1790214955,v1:172.21.15.90:6835/1790214955] up:boot 2024-05-03T15:05:03.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:03 smithi090 ceph-mon[33568]: mds.? [v2:172.21.15.90:6832/233486054,v1:172.21.15.90:6834/233486054] up:boot 2024-05-03T15:05:03.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:03 smithi090 ceph-mon[33568]: mds.? [v2:172.21.15.149:6834/50709348,v1:172.21.15.149:6838/50709348] up:boot 2024-05-03T15:05:03.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:03 smithi090 ceph-mon[33568]: mds.? [v2:172.21.15.149:6832/3041052525,v1:172.21.15.149:6837/3041052525] up:boot 2024-05-03T15:05:03.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:03 smithi090 ceph-mon[33568]: mds.? [v2:172.21.15.149:6833/523430178,v1:172.21.15.149:6836/523430178] up:boot 2024-05-03T15:05:03.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:03 smithi090 ceph-mon[33568]: mds.? [v2:172.21.15.149:6835/4163424965,v1:172.21.15.149:6839/4163424965] up:boot 2024-05-03T15:05:03.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:03 smithi090 ceph-mon[33568]: mds.? [v2:172.21.15.90:6836/778709002,v1:172.21.15.90:6838/778709002] up:boot 2024-05-03T15:05:03.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:03 smithi090 ceph-mon[33568]: mds.? [v2:172.21.15.90:6837/2598565721,v1:172.21.15.90:6839/2598565721] up:boot 2024-05-03T15:05:03.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:03 smithi090 ceph-mon[33568]: mds.? [v2:172.21.15.60:6838/1167209231,v1:172.21.15.60:6839/1167209231] up:boot 2024-05-03T15:05:03.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:03 smithi090 ceph-mon[33568]: mds.? [v2:172.21.15.60:6840/720135337,v1:172.21.15.60:6841/720135337] up:boot 2024-05-03T15:05:03.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:03 smithi090 ceph-mon[33568]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-05-03T15:05:03.375 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:03 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-03T15:05:03.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:03 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-03T15:05:03.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:03 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-03T15:05:03.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:03 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-03T15:05:03.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:03 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-03T15:05:03.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:03 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-03T15:05:03.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:03 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-03T15:05:03.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:03 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-03T15:05:03.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:03 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-03T15:05:03.376 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:03 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-03T15:05:03.377 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:03 smithi090 ceph-mon[33568]: pgmap v949: 129 pgs: 129 active+clean; 649 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-05-03T15:05:03.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:03 smithi149 ceph-mon[33332]: mds.? [v2:172.21.15.90:6833/1790214955,v1:172.21.15.90:6835/1790214955] up:boot 2024-05-03T15:05:03.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:03 smithi149 ceph-mon[33332]: mds.? [v2:172.21.15.90:6832/233486054,v1:172.21.15.90:6834/233486054] up:boot 2024-05-03T15:05:03.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:03 smithi149 ceph-mon[33332]: mds.? [v2:172.21.15.149:6834/50709348,v1:172.21.15.149:6838/50709348] up:boot 2024-05-03T15:05:03.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:03 smithi149 ceph-mon[33332]: mds.? [v2:172.21.15.149:6832/3041052525,v1:172.21.15.149:6837/3041052525] up:boot 2024-05-03T15:05:03.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:03 smithi149 ceph-mon[33332]: mds.? [v2:172.21.15.149:6833/523430178,v1:172.21.15.149:6836/523430178] up:boot 2024-05-03T15:05:03.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:03 smithi149 ceph-mon[33332]: mds.? [v2:172.21.15.149:6835/4163424965,v1:172.21.15.149:6839/4163424965] up:boot 2024-05-03T15:05:03.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:03 smithi149 ceph-mon[33332]: mds.? [v2:172.21.15.90:6836/778709002,v1:172.21.15.90:6838/778709002] up:boot 2024-05-03T15:05:03.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:03 smithi149 ceph-mon[33332]: mds.? [v2:172.21.15.90:6837/2598565721,v1:172.21.15.90:6839/2598565721] up:boot 2024-05-03T15:05:03.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:03 smithi149 ceph-mon[33332]: mds.? [v2:172.21.15.60:6838/1167209231,v1:172.21.15.60:6839/1167209231] up:boot 2024-05-03T15:05:03.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:03 smithi149 ceph-mon[33332]: mds.? [v2:172.21.15.60:6840/720135337,v1:172.21.15.60:6841/720135337] up:boot 2024-05-03T15:05:03.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:03 smithi149 ceph-mon[33332]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-05-03T15:05:03.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:03 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-03T15:05:03.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:03 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-03T15:05:03.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:03 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-03T15:05:03.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:03 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-03T15:05:03.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:03 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-03T15:05:03.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:03 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-03T15:05:03.425 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:03 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-03T15:05:03.425 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:03 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-03T15:05:03.425 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:03 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-03T15:05:03.425 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:03 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-03T15:05:03.425 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:03 smithi149 ceph-mon[33332]: pgmap v949: 129 pgs: 129 active+clean; 649 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-05-03T15:05:04.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:04.473+0000 7f5dd3400700 1 -- 172.21.15.60:0/631163890 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5dcc06c510 msgr2=0x7f5dcc110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T15:05:04.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:04.473+0000 7f5dd3400700 1 --2- 172.21.15.60:0/631163890 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5dcc06c510 0x7f5dcc110330 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5dc400b0a0 tx=0x7f5dc400b3b0 comp rx=0 tx=0).stop 2024-05-03T15:05:04.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:04.473+0000 7f5dd3400700 1 -- 172.21.15.60:0/631163890 shutdown_connections 2024-05-03T15:05:04.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:04.473+0000 7f5dd3400700 1 --2- 172.21.15.60:0/631163890 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5dcc06c510 0x7f5dcc110330 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:05:04.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:04.473+0000 7f5dd3400700 1 --2- 172.21.15.60:0/631163890 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5dcc109510 0x7f5dcc1098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:05:04.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:04.473+0000 7f5dd3400700 1 --2- 172.21.15.60:0/631163890 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5dcc06bb50 0x7f5dcc06bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:05:04.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:04.473+0000 7f5dd3400700 1 -- 172.21.15.60:0/631163890 >> 172.21.15.60:0/631163890 conn(0x7f5dcc079310 msgr2=0x7f5dcc079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T15:05:04.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:04.473+0000 7f5dd3400700 1 -- 172.21.15.60:0/631163890 shutdown_connections 2024-05-03T15:05:04.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:04.473+0000 7f5dd3400700 1 -- 172.21.15.60:0/631163890 wait complete. 2024-05-03T15:05:04.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:04.475+0000 7f5dd3400700 1 Processor -- start 2024-05-03T15:05:04.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:04.475+0000 7f5dd3400700 1 -- start start 2024-05-03T15:05:04.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:04.476+0000 7f5dd3400700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5dcc06bb50 0x7f5dcc19f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T15:05:04.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:04.476+0000 7f5dd3400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5dcc06c510 0x7f5dcc19fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T15:05:04.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:04.476+0000 7f5dd3400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5dcc109510 0x7f5dcc199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T15:05:04.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:04.476+0000 7f5dd3400700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5dcc113920 con 0x7f5dcc06bb50 2024-05-03T15:05:04.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:04.476+0000 7f5dd3400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5dcc1137a0 con 0x7f5dcc06c510 2024-05-03T15:05:04.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:04.476+0000 7f5dd3400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5dcc113aa0 con 0x7f5dcc109510 2024-05-03T15:05:04.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:04.476+0000 7f5dd0a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5dcc06bb50 0x7f5dcc19f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T15:05:04.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:04.476+0000 7f5dd0a00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5dcc06bb50 0x7f5dcc19f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57550/0 (socket says 172.21.15.60:57550) 2024-05-03T15:05:04.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:04.476+0000 7f5dd0a00700 1 -- 172.21.15.60:0/4187829055 learned_addr learned my addr 172.21.15.60:0/4187829055 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T15:05:04.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:04.476+0000 7f5dd1400700 1 --2- 172.21.15.60:0/4187829055 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5dcc109510 0x7f5dcc199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T15:05:04.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:04.476+0000 7f5dc3e00700 1 --2- 172.21.15.60:0/4187829055 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5dcc06c510 0x7f5dcc19fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T15:05:04.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:04.477+0000 7f5dd1400700 1 -- 172.21.15.60:0/4187829055 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5dcc06c510 msgr2=0x7f5dcc19fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T15:05:04.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:04.477+0000 7f5dd1400700 1 --2- 172.21.15.60:0/4187829055 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5dcc06c510 0x7f5dcc19fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:05:04.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:04.477+0000 7f5dd1400700 1 -- 172.21.15.60:0/4187829055 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5dcc06bb50 msgr2=0x7f5dcc19f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T15:05:04.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:04.477+0000 7f5dd1400700 1 --2- 172.21.15.60:0/4187829055 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5dcc06bb50 0x7f5dcc19f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:05:04.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:04.477+0000 7f5dd1400700 1 -- 172.21.15.60:0/4187829055 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dcc19a260 con 0x7f5dcc109510 2024-05-03T15:05:04.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:04.477+0000 7f5dd0a00700 1 --2- 172.21.15.60:0/4187829055 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5dcc06bb50 0x7f5dcc19f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T15:05:04.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:04.477+0000 7f5dd1400700 1 --2- 172.21.15.60:0/4187829055 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5dcc109510 0x7f5dcc199b00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5dc400b070 tx=0x7f5dc4004c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T15:05:04.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:04.477+0000 7f5dca000700 1 -- 172.21.15.60:0/4187829055 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc4021070 con 0x7f5dcc109510 2024-05-03T15:05:04.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:04.477+0000 7f5dd3400700 1 -- 172.21.15.60:0/4187829055 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dcc19a4f0 con 0x7f5dcc109510 2024-05-03T15:05:04.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:04.477+0000 7f5dca000700 1 -- 172.21.15.60:0/4187829055 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f5dc4007b80 con 0x7f5dcc109510 2024-05-03T15:05:04.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:04.477+0000 7f5dca000700 1 -- 172.21.15.60:0/4187829055 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc40052a0 con 0x7f5dcc109510 2024-05-03T15:05:04.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:04.477+0000 7f5dd3400700 1 -- 172.21.15.60:0/4187829055 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dcc1a6810 con 0x7f5dcc109510 2024-05-03T15:05:04.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:04.479+0000 7f5dd3400700 1 -- 172.21.15.60:0/4187829055 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dcc10b9f0 con 0x7f5dcc109510 2024-05-03T15:05:04.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:04.481+0000 7f5dca000700 1 -- 172.21.15.60:0/4187829055 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f5dc4024070 con 0x7f5dcc109510 2024-05-03T15:05:04.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:04.483+0000 7f5dca000700 1 --2- 172.21.15.60:0/4187829055 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f5db00b2910 0x7f5db00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T15:05:04.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:04.483+0000 7f5dca000700 1 -- 172.21.15.60:0/4187829055 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9551+0+0 (secure 0 0 0) 0x7f5dc400f020 con 0x7f5dcc109510 2024-05-03T15:05:04.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:04.483+0000 7f5dd0a00700 1 --2- 172.21.15.60:0/4187829055 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f5db00b2910 0x7f5db00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T15:05:04.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:04.484+0000 7f5dd0a00700 1 --2- 172.21.15.60:0/4187829055 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f5db00b2910 0x7f5db00b4dd0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc011550 tx=0x7f5dbc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T15:05:04.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:04.485+0000 7f5dca000700 1 -- 172.21.15.60:0/4187829055 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f5dc402b020 con 0x7f5dcc109510 2024-05-03T15:05:04.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:04.771+0000 7f5dd3400700 1 -- 172.21.15.60:0/4187829055 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f5dcc19aff0 con 0x7f5dcc109510 2024-05-03T15:05:04.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:04.839+0000 7f5dca000700 1 -- 172.21.15.60:0/4187829055 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v33) ==== 104+0+0 (secure 0 0 0) 0x7f5dc40ab740 con 0x7f5dcc109510 2024-05-03T15:05:04.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:04.843+0000 7f5dd3400700 1 -- 172.21.15.60:0/4187829055 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f5db00b2910 msgr2=0x7f5db00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T15:05:04.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:04.843+0000 7f5dd3400700 1 --2- 172.21.15.60:0/4187829055 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f5db00b2910 0x7f5db00b4dd0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc011550 tx=0x7f5dbc009260 comp rx=0 tx=0).stop 2024-05-03T15:05:04.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:04.843+0000 7f5dd3400700 1 -- 172.21.15.60:0/4187829055 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5dcc109510 msgr2=0x7f5dcc199b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T15:05:04.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:04.843+0000 7f5dd3400700 1 --2- 172.21.15.60:0/4187829055 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5dcc109510 0x7f5dcc199b00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5dc400b070 tx=0x7f5dc4004c80 comp rx=0 tx=0).stop 2024-05-03T15:05:04.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:04.844+0000 7f5dd3400700 1 -- 172.21.15.60:0/4187829055 shutdown_connections 2024-05-03T15:05:04.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:04.844+0000 7f5dd3400700 1 --2- 172.21.15.60:0/4187829055 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f5db00b2910 0x7f5db00b4dd0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:05:04.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:04.844+0000 7f5dd3400700 1 --2- 172.21.15.60:0/4187829055 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5dcc109510 0x7f5dcc199b00 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:05:04.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:04.844+0000 7f5dd3400700 1 --2- 172.21.15.60:0/4187829055 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5dcc06bb50 0x7f5dcc19f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:05:04.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:04.844+0000 7f5dd3400700 1 --2- 172.21.15.60:0/4187829055 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5dcc06c510 0x7f5dcc19fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:05:04.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:04.844+0000 7f5dd3400700 1 -- 172.21.15.60:0/4187829055 >> 172.21.15.60:0/4187829055 conn(0x7f5dcc079310 msgr2=0x7f5dcc100eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T15:05:04.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:04.844+0000 7f5dd3400700 1 -- 172.21.15.60:0/4187829055 shutdown_connections 2024-05-03T15:05:04.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:04.844+0000 7f5dd3400700 1 -- 172.21.15.60:0/4187829055 wait complete. 2024-05-03T15:05:05.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:04 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/4187829055' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-03T15:05:05.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:04 smithi060 ceph-mon[28873]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-03T15:05:05.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:04 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/4187829055' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-03T15:05:05.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:04 smithi090 ceph-mon[33568]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-03T15:05:05.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:04 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/4187829055' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-03T15:05:05.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:04 smithi149 ceph-mon[33332]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-03T15:05:05.595 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-03T15:05:06.076 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T15:05:06.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:05 smithi060 ceph-mon[28873]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-03T15:05:06.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:05 smithi060 ceph-mon[28873]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-03T15:05:06.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:05 smithi060 ceph-mon[28873]: Cluster is now healthy 2024-05-03T15:05:06.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:05 smithi060 ceph-mon[28873]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-03T15:05:06.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:05 smithi060 ceph-mon[28873]: fsmap 12 up:standby 2024-05-03T15:05:06.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:05 smithi060 ceph-mon[28873]: pgmap v950: 129 pgs: 129 active+clean; 649 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-05-03T15:05:06.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:05 smithi090 ceph-mon[33568]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-03T15:05:06.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:05 smithi090 ceph-mon[33568]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-03T15:05:06.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:05 smithi090 ceph-mon[33568]: Cluster is now healthy 2024-05-03T15:05:06.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:05 smithi090 ceph-mon[33568]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-03T15:05:06.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:05 smithi090 ceph-mon[33568]: fsmap 12 up:standby 2024-05-03T15:05:06.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:05 smithi090 ceph-mon[33568]: pgmap v950: 129 pgs: 129 active+clean; 649 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-05-03T15:05:06.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:05 smithi149 ceph-mon[33332]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-03T15:05:06.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:05 smithi149 ceph-mon[33332]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-03T15:05:06.173 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:05 smithi149 ceph-mon[33332]: Cluster is now healthy 2024-05-03T15:05:06.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:05 smithi149 ceph-mon[33332]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-03T15:05:06.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:05 smithi149 ceph-mon[33332]: fsmap 12 up:standby 2024-05-03T15:05:06.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:05 smithi149 ceph-mon[33332]: pgmap v950: 129 pgs: 129 active+clean; 649 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-05-03T15:05:08.057 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:08 smithi060 ceph-mon[28873]: pgmap v951: 129 pgs: 129 active+clean; 649 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-05-03T15:05:08.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:08 smithi090 ceph-mon[33568]: pgmap v951: 129 pgs: 129 active+clean; 649 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-05-03T15:05:08.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:08.372+0000 7f933be00700 1 -- 172.21.15.60:0/3649637605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f933c06ba10 msgr2=0x7f933c06be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T15:05:08.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:08.372+0000 7f933be00700 1 --2- 172.21.15.60:0/3649637605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f933c06ba10 0x7f933c06be90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9324009990 tx=0x7f9324009ca0 comp rx=0 tx=0).stop 2024-05-03T15:05:08.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:08.373+0000 7f933be00700 1 -- 172.21.15.60:0/3649637605 shutdown_connections 2024-05-03T15:05:08.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:08.373+0000 7f933be00700 1 --2- 172.21.15.60:0/3649637605 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f933c06c3d0 0x7f933c1102a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:05:08.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:08.373+0000 7f933be00700 1 --2- 172.21.15.60:0/3649637605 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f933c109510 0x7f933c1098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:05:08.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:08.373+0000 7f933be00700 1 --2- 172.21.15.60:0/3649637605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f933c06ba10 0x7f933c06be90 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:05:08.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:08.373+0000 7f933be00700 1 -- 172.21.15.60:0/3649637605 >> 172.21.15.60:0/3649637605 conn(0x7f933c0791e0 msgr2=0x7f933c0795f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T15:05:08.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:08.373+0000 7f933be00700 1 -- 172.21.15.60:0/3649637605 shutdown_connections 2024-05-03T15:05:08.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:08.373+0000 7f933be00700 1 -- 172.21.15.60:0/3649637605 wait complete. 2024-05-03T15:05:08.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:08.374+0000 7f933be00700 1 Processor -- start 2024-05-03T15:05:08.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:08.374+0000 7f933be00700 1 -- start start 2024-05-03T15:05:08.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:08.375+0000 7f933be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f933c06ba10 0x7f933c19f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T15:05:08.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:08.375+0000 7f933be00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f933c06c3d0 0x7f933c19fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T15:05:08.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:08.375+0000 7f933be00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f933c109510 0x7f933c1999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T15:05:08.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:08.375+0000 7f933be00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f933c1137f0 con 0x7f933c06c3d0 2024-05-03T15:05:08.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:08.375+0000 7f933be00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f933c113670 con 0x7f933c06ba10 2024-05-03T15:05:08.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:08.375+0000 7f933be00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f933c113970 con 0x7f933c109510 2024-05-03T15:05:08.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:08.375+0000 7f9332000700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f933c06c3d0 0x7f933c19fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T15:05:08.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:08.375+0000 7f933aa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f933c06ba10 0x7f933c19f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T15:05:08.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:08.375+0000 7f9332000700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f933c06c3d0 0x7f933c19fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57572/0 (socket says 172.21.15.60:57572) 2024-05-03T15:05:08.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:08.375+0000 7f9332000700 1 -- 172.21.15.60:0/2430730640 learned_addr learned my addr 172.21.15.60:0/2430730640 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T15:05:08.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:08.375+0000 7f933b400700 1 --2- 172.21.15.60:0/2430730640 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f933c109510 0x7f933c1999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T15:05:08.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:08.376+0000 7f9332000700 1 -- 172.21.15.60:0/2430730640 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f933c109510 msgr2=0x7f933c1999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T15:05:08.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:08.376+0000 7f9332000700 1 --2- 172.21.15.60:0/2430730640 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f933c109510 0x7f933c1999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:05:08.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:08.376+0000 7f9332000700 1 -- 172.21.15.60:0/2430730640 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f933c06ba10 msgr2=0x7f933c19f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T15:05:08.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:08.376+0000 7f9332000700 1 --2- 172.21.15.60:0/2430730640 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f933c06ba10 0x7f933c19f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:05:08.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:08.376+0000 7f9332000700 1 -- 172.21.15.60:0/2430730640 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f933c19a150 con 0x7f933c06c3d0 2024-05-03T15:05:08.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:08.376+0000 7f933aa00700 1 --2- 172.21.15.60:0/2430730640 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f933c06ba10 0x7f933c19f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T15:05:08.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:08.376+0000 7f9332000700 1 --2- 172.21.15.60:0/2430730640 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f933c06c3d0 0x7f933c19fd20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f93240095c0 tx=0x7f9324005590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T15:05:08.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:08.376+0000 7f9333e00700 1 -- 172.21.15.60:0/2430730640 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9324026070 con 0x7f933c06c3d0 2024-05-03T15:05:08.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:08.376+0000 7f933be00700 1 -- 172.21.15.60:0/2430730640 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f933c19a3e0 con 0x7f933c06c3d0 2024-05-03T15:05:08.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:08.376+0000 7f9333e00700 1 -- 172.21.15.60:0/2430730640 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f932400fe30 con 0x7f933c06c3d0 2024-05-03T15:05:08.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:08.377+0000 7f933be00700 1 -- 172.21.15.60:0/2430730640 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f933c102560 con 0x7f933c06c3d0 2024-05-03T15:05:08.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:08.377+0000 7f9333e00700 1 -- 172.21.15.60:0/2430730640 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9324030060 con 0x7f933c06c3d0 2024-05-03T15:05:08.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:08.378+0000 7f933be00700 1 -- 172.21.15.60:0/2430730640 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f933c104b10 con 0x7f933c06c3d0 2024-05-03T15:05:08.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:08.380+0000 7f9333e00700 1 -- 172.21.15.60:0/2430730640 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f9324029070 con 0x7f933c06c3d0 2024-05-03T15:05:08.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:08.382+0000 7f9333e00700 1 --2- 172.21.15.60:0/2430730640 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f93200b2910 0x7f93200b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T15:05:08.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:08.382+0000 7f9333e00700 1 -- 172.21.15.60:0/2430730640 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9551+0+0 (secure 0 0 0) 0x7f93240e3730 con 0x7f933c06c3d0 2024-05-03T15:05:08.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:08.382+0000 7f933aa00700 1 --2- 172.21.15.60:0/2430730640 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f93200b2910 0x7f93200b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T15:05:08.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:08.383+0000 7f933aa00700 1 --2- 172.21.15.60:0/2430730640 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f93200b2910 0x7f93200b4dd0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f933c102460 tx=0x7f932c012040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T15:05:08.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:08.385+0000 7f9333e00700 1 -- 172.21.15.60:0/2430730640 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f9324013070 con 0x7f933c06c3d0 2024-05-03T15:05:08.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:08 smithi149 ceph-mon[33332]: pgmap v951: 129 pgs: 129 active+clean; 649 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-05-03T15:05:08.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:08.625+0000 7f933be00700 1 -- 172.21.15.60:0/2430730640 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f933c19aed0 con 0x7f933c06c3d0 2024-05-03T15:05:09.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:09.034+0000 7f9333e00700 1 -- 172.21.15.60:0/2430730640 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v91) ==== 181+0+0 (secure 0 0 0) 0x7f93240aae60 con 0x7f933c06c3d0 2024-05-03T15:05:09.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:09.039+0000 7f933be00700 1 -- 172.21.15.60:0/2430730640 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f93200b2910 msgr2=0x7f93200b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T15:05:09.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:09.039+0000 7f933be00700 1 --2- 172.21.15.60:0/2430730640 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f93200b2910 0x7f93200b4dd0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f933c102460 tx=0x7f932c012040 comp rx=0 tx=0).stop 2024-05-03T15:05:09.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:09.039+0000 7f933be00700 1 -- 172.21.15.60:0/2430730640 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f933c06c3d0 msgr2=0x7f933c19fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T15:05:09.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:09.039+0000 7f933be00700 1 --2- 172.21.15.60:0/2430730640 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f933c06c3d0 0x7f933c19fd20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f93240095c0 tx=0x7f9324005590 comp rx=0 tx=0).stop 2024-05-03T15:05:09.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:09.040+0000 7f933be00700 1 -- 172.21.15.60:0/2430730640 shutdown_connections 2024-05-03T15:05:09.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:09.040+0000 7f933be00700 1 --2- 172.21.15.60:0/2430730640 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f93200b2910 0x7f93200b4dd0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:05:09.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:09.040+0000 7f933be00700 1 --2- 172.21.15.60:0/2430730640 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f933c109510 0x7f933c1999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:05:09.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:09.040+0000 7f933be00700 1 --2- 172.21.15.60:0/2430730640 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f933c06ba10 0x7f933c19f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:05:09.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:09.040+0000 7f933be00700 1 --2- 172.21.15.60:0/2430730640 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f933c06c3d0 0x7f933c19fd20 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:05:09.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:09.040+0000 7f933be00700 1 -- 172.21.15.60:0/2430730640 >> 172.21.15.60:0/2430730640 conn(0x7f933c0791e0 msgr2=0x7f933c100510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T15:05:09.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:09.042+0000 7f933be00700 1 -- 172.21.15.60:0/2430730640 shutdown_connections 2024-05-03T15:05:09.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:09.042+0000 7f933be00700 1 -- 172.21.15.60:0/2430730640 wait complete. 2024-05-03T15:05:09.045 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' removed 2024-05-03T15:05:09.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:09 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/2430730640' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-03T15:05:09.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:09 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/2430730640' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-03T15:05:09.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:09 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/2430730640' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-03T15:05:09.814 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-03T15:05:10.277 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T15:05:10.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:10 smithi060 ceph-mon[28873]: pgmap v952: 129 pgs: 129 active+clean; 649 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T15:05:10.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:10 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/2430730640' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-03T15:05:10.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:10 smithi060 ceph-mon[28873]: osdmap e91: 12 total, 12 up, 12 in 2024-05-03T15:05:10.373 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:10 smithi090 ceph-mon[33568]: pgmap v952: 129 pgs: 129 active+clean; 649 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T15:05:10.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:10 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/2430730640' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-03T15:05:10.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:10 smithi090 ceph-mon[33568]: osdmap e91: 12 total, 12 up, 12 in 2024-05-03T15:05:10.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:10 smithi149 ceph-mon[33332]: pgmap v952: 129 pgs: 129 active+clean; 649 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-03T15:05:10.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:10 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/2430730640' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-03T15:05:10.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:10 smithi149 ceph-mon[33332]: osdmap e91: 12 total, 12 up, 12 in 2024-05-03T15:05:11.308 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:11 smithi060 ceph-mon[28873]: pgmap v954: 97 pgs: 97 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T15:05:11.373 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:11 smithi090 ceph-mon[33568]: pgmap v954: 97 pgs: 97 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T15:05:11.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:11 smithi149 ceph-mon[33332]: pgmap v954: 97 pgs: 97 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T15:05:12.275 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:12 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T15:05:12.373 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:12 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T15:05:12.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:12 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T15:05:12.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:12.453+0000 7f259da00700 1 -- 172.21.15.60:0/1451759064 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25981045d0 msgr2=0x7f25981049b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T15:05:12.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:12.453+0000 7f259da00700 1 --2- 172.21.15.60:0/1451759064 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25981045d0 0x7f25981049b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2588009990 tx=0x7f2588009ca0 comp rx=0 tx=0).stop 2024-05-03T15:05:12.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:12.454+0000 7f259da00700 1 -- 172.21.15.60:0/1451759064 shutdown_connections 2024-05-03T15:05:12.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:12.454+0000 7f259da00700 1 --2- 172.21.15.60:0/1451759064 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f259810da00 0x7f259810fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:05:12.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:12.454+0000 7f259da00700 1 --2- 172.21.15.60:0/1451759064 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25981045d0 0x7f25981049b0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:05:12.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:12.454+0000 7f259da00700 1 --2- 172.21.15.60:0/1451759064 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2598104ef0 0x7f259810d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:05:12.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:12.454+0000 7f259da00700 1 -- 172.21.15.60:0/1451759064 >> 172.21.15.60:0/1451759064 conn(0x7f25980fddc0 msgr2=0x7f25981001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T15:05:12.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:12.454+0000 7f259da00700 1 -- 172.21.15.60:0/1451759064 shutdown_connections 2024-05-03T15:05:12.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:12.454+0000 7f259da00700 1 -- 172.21.15.60:0/1451759064 wait complete. 2024-05-03T15:05:12.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:12.455+0000 7f259da00700 1 Processor -- start 2024-05-03T15:05:12.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:12.455+0000 7f259da00700 1 -- start start 2024-05-03T15:05:12.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:12.455+0000 7f259da00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f25981045d0 0x7f259819f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T15:05:12.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:12.455+0000 7f259da00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2598104ef0 0x7f259819fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T15:05:12.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:12.456+0000 7f259da00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f259810da00 0x7f2598199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T15:05:12.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:12.456+0000 7f259da00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f2598113830 con 0x7f2598104ef0 2024-05-03T15:05:12.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:12.456+0000 7f259da00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f25981136b0 con 0x7f25981045d0 2024-05-03T15:05:12.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:12.456+0000 7f259da00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f25981139b0 con 0x7f259810da00 2024-05-03T15:05:12.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:12.456+0000 7f2597e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f25981045d0 0x7f259819f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T15:05:12.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:12.456+0000 7f258fe00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2598104ef0 0x7f259819fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T15:05:12.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:12.456+0000 7f2597e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f25981045d0 0x7f259819f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.60:54378/0 (socket says 172.21.15.60:54378) 2024-05-03T15:05:12.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:12.456+0000 7f2597e00700 1 -- 172.21.15.60:0/3275456428 learned_addr learned my addr 172.21.15.60:0/3275456428 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T15:05:12.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:12.456+0000 7f259d000700 1 --2- 172.21.15.60:0/3275456428 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f259810da00 0x7f2598199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T15:05:12.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:12.456+0000 7f2597e00700 1 -- 172.21.15.60:0/3275456428 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f259810da00 msgr2=0x7f2598199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T15:05:12.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:12.456+0000 7f2597e00700 1 --2- 172.21.15.60:0/3275456428 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f259810da00 0x7f2598199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:05:12.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:12.456+0000 7f2597e00700 1 -- 172.21.15.60:0/3275456428 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2598104ef0 msgr2=0x7f259819fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T15:05:12.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:12.456+0000 7f2597e00700 1 --2- 172.21.15.60:0/3275456428 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2598104ef0 0x7f259819fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:05:12.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:12.456+0000 7f2597e00700 1 -- 172.21.15.60:0/3275456428 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f259819a1f0 con 0x7f25981045d0 2024-05-03T15:05:12.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:12.457+0000 7f2597e00700 1 --2- 172.21.15.60:0/3275456428 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f25981045d0 0x7f259819f8d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2588000c00 tx=0x7f2588004580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T15:05:12.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:12.457+0000 7f2595600700 1 -- 172.21.15.60:0/3275456428 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2588027070 con 0x7f25981045d0 2024-05-03T15:05:12.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:12.457+0000 7f2595600700 1 -- 172.21.15.60:0/3275456428 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f2588022470 con 0x7f25981045d0 2024-05-03T15:05:12.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:12.457+0000 7f2595600700 1 -- 172.21.15.60:0/3275456428 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2588017650 con 0x7f25981045d0 2024-05-03T15:05:12.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:12.457+0000 7f259da00700 1 -- 172.21.15.60:0/3275456428 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f259819a480 con 0x7f25981045d0 2024-05-03T15:05:12.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:12.457+0000 7f259da00700 1 -- 172.21.15.60:0/3275456428 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2598102160 con 0x7f25981045d0 2024-05-03T15:05:12.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:12.459+0000 7f259da00700 1 -- 172.21.15.60:0/3275456428 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f259806a190 con 0x7f25981045d0 2024-05-03T15:05:12.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:12.461+0000 7f2595600700 1 -- 172.21.15.60:0/3275456428 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f258802a070 con 0x7f25981045d0 2024-05-03T15:05:12.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:12.462+0000 7f2595600700 1 --2- 172.21.15.60:0/3275456428 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f257c0b2990 0x7f257c0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T15:05:12.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:12.462+0000 7f2595600700 1 -- 172.21.15.60:0/3275456428 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8997+0+0 (secure 0 0 0) 0x7f25880e3440 con 0x7f25981045d0 2024-05-03T15:05:12.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:12.463+0000 7f258fe00700 1 --2- 172.21.15.60:0/3275456428 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f257c0b2990 0x7f257c0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T15:05:12.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:12.463+0000 7f258fe00700 1 --2- 172.21.15.60:0/3275456428 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f257c0b2990 0x7f257c0b4e50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f259810f170 tx=0x7f2580012040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T15:05:12.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:12.470+0000 7f2595600700 1 -- 172.21.15.60:0/3275456428 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f2588013070 con 0x7f25981045d0 2024-05-03T15:05:12.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:12.736+0000 7f259da00700 1 -- 172.21.15.60:0/3275456428 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f259802e510 con 0x7f25981045d0 2024-05-03T15:05:13.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:13.052+0000 7f2595600700 1 -- 172.21.15.60:0/3275456428 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v92) ==== 169+0+0 (secure 0 0 0) 0x7f25880aad30 con 0x7f25981045d0 2024-05-03T15:05:13.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:13.057+0000 7f259da00700 1 -- 172.21.15.60:0/3275456428 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f257c0b2990 msgr2=0x7f257c0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T15:05:13.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:13.057+0000 7f259da00700 1 --2- 172.21.15.60:0/3275456428 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f257c0b2990 0x7f257c0b4e50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f259810f170 tx=0x7f2580012040 comp rx=0 tx=0).stop 2024-05-03T15:05:13.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:13.057+0000 7f259da00700 1 -- 172.21.15.60:0/3275456428 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f25981045d0 msgr2=0x7f259819f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T15:05:13.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:13.057+0000 7f259da00700 1 --2- 172.21.15.60:0/3275456428 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f25981045d0 0x7f259819f8d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2588000c00 tx=0x7f2588004580 comp rx=0 tx=0).stop 2024-05-03T15:05:13.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:13.058+0000 7f259da00700 1 -- 172.21.15.60:0/3275456428 shutdown_connections 2024-05-03T15:05:13.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:13.058+0000 7f259da00700 1 --2- 172.21.15.60:0/3275456428 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f257c0b2990 0x7f257c0b4e50 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:05:13.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:13.058+0000 7f259da00700 1 --2- 172.21.15.60:0/3275456428 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f259810da00 0x7f2598199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:05:13.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:13.058+0000 7f259da00700 1 --2- 172.21.15.60:0/3275456428 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f25981045d0 0x7f259819f8d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:05:13.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:13.058+0000 7f259da00700 1 --2- 172.21.15.60:0/3275456428 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2598104ef0 0x7f259819fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:05:13.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:13.058+0000 7f259da00700 1 -- 172.21.15.60:0/3275456428 >> 172.21.15.60:0/3275456428 conn(0x7f25980fddc0 msgr2=0x7f25980fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T15:05:13.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:13.059+0000 7f259da00700 1 -- 172.21.15.60:0/3275456428 shutdown_connections 2024-05-03T15:05:13.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:13.059+0000 7f259da00700 1 -- 172.21.15.60:0/3275456428 wait complete. 2024-05-03T15:05:13.062 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' removed 2024-05-03T15:05:13.337 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:13 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/3275456428' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-03T15:05:13.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:13 smithi060 ceph-mon[28873]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-03T15:05:13.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:13 smithi060 ceph-mon[28873]: pgmap v955: 97 pgs: 97 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T15:05:13.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:13 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/3275456428' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-03T15:05:13.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:13 smithi090 ceph-mon[33568]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-03T15:05:13.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:13 smithi090 ceph-mon[33568]: pgmap v955: 97 pgs: 97 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T15:05:13.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:13 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/3275456428' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-03T15:05:13.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:13 smithi149 ceph-mon[33332]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-03T15:05:13.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:13 smithi149 ceph-mon[33332]: pgmap v955: 97 pgs: 97 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T15:05:13.865 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 285d64b2-095a-11ef-bc97-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-05-03T15:05:14.322 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T15:05:14.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:14 smithi060 ceph-mon[28873]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-03T15:05:14.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:14 smithi060 ceph-mon[28873]: osdmap e92: 12 total, 12 up, 12 in 2024-05-03T15:05:14.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:14 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T15:05:14.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:14 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T15:05:14.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:14 smithi060 ceph-mon[28873]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T15:05:14.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:14 smithi090 ceph-mon[33568]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-03T15:05:14.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:14 smithi090 ceph-mon[33568]: osdmap e92: 12 total, 12 up, 12 in 2024-05-03T15:05:14.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:14 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T15:05:14.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:14 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T15:05:14.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:14 smithi090 ceph-mon[33568]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T15:05:14.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:14 smithi149 ceph-mon[33332]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-03T15:05:14.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:14 smithi149 ceph-mon[33332]: osdmap e92: 12 total, 12 up, 12 in 2024-05-03T15:05:14.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:14 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T15:05:14.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:14 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T15:05:14.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:14 smithi149 ceph-mon[33332]: from='mgr.14152 172.21.15.60:0/1178095988' entity='mgr.x' 2024-05-03T15:05:15.333 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:15 smithi060 ceph-mon[28873]: pgmap v957: 33 pgs: 33 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T15:05:15.373 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:15 smithi090 ceph-mon[33568]: pgmap v957: 33 pgs: 33 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T15:05:15.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:15 smithi149 ceph-mon[33332]: pgmap v957: 33 pgs: 33 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T15:05:16.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:16.505+0000 7f9e7da00700 1 -- 172.21.15.60:0/2142243461 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e7810b660 msgr2=0x7f9e7810ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T15:05:16.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:16.505+0000 7f9e7da00700 1 --2- 172.21.15.60:0/2142243461 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e7810b660 0x7f9e7810ba40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9e68009990 tx=0x7f9e68009ca0 comp rx=0 tx=0).stop 2024-05-03T15:05:16.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:16.505+0000 7f9e7da00700 1 -- 172.21.15.60:0/2142243461 shutdown_connections 2024-05-03T15:05:16.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:16.505+0000 7f9e7da00700 1 --2- 172.21.15.60:0/2142243461 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e78105f90 0x7f9e781124a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:05:16.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:16.505+0000 7f9e7da00700 1 --2- 172.21.15.60:0/2142243461 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e7810b660 0x7f9e7810ba40 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:05:16.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:16.505+0000 7f9e7da00700 1 --2- 172.21.15.60:0/2142243461 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9e781055d0 0x7f9e78105a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:05:16.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:16.505+0000 7f9e7da00700 1 -- 172.21.15.60:0/2142243461 >> 172.21.15.60:0/2142243461 conn(0x7f9e781010c0 msgr2=0x7f9e781034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T15:05:16.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:16.505+0000 7f9e7da00700 1 -- 172.21.15.60:0/2142243461 shutdown_connections 2024-05-03T15:05:16.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:16.505+0000 7f9e7da00700 1 -- 172.21.15.60:0/2142243461 wait complete. 2024-05-03T15:05:16.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:16.506+0000 7f9e7da00700 1 Processor -- start 2024-05-03T15:05:16.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:16.507+0000 7f9e7da00700 1 -- start start 2024-05-03T15:05:16.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:16.507+0000 7f9e7da00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9e781055d0 0x7f9e7819bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T15:05:16.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:16.507+0000 7f9e7da00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e78105f90 0x7f9e7819c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T15:05:16.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:16.507+0000 7f9e7da00700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e7810b660 0x7f9e781a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T15:05:16.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:16.507+0000 7f9e7da00700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f9e78115a70 con 0x7f9e7810b660 2024-05-03T15:05:16.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:16.507+0000 7f9e7da00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9e781158f0 con 0x7f9e78105f90 2024-05-03T15:05:16.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:16.507+0000 7f9e7da00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9e78115bf0 con 0x7f9e781055d0 2024-05-03T15:05:16.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:16.507+0000 7f9e77400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e78105f90 0x7f9e7819c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T15:05:16.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:16.507+0000 7f9e77e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9e781055d0 0x7f9e7819bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T15:05:16.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:16.507+0000 7f9e77400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e78105f90 0x7f9e7819c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.60:54394/0 (socket says 172.21.15.60:54394) 2024-05-03T15:05:16.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:16.508+0000 7f9e77400700 1 -- 172.21.15.60:0/2652203013 learned_addr learned my addr 172.21.15.60:0/2652203013 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-05-03T15:05:16.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:16.508+0000 7f9e7d000700 1 --2- 172.21.15.60:0/2652203013 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e7810b660 0x7f9e781a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T15:05:16.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:16.508+0000 7f9e77400700 1 -- 172.21.15.60:0/2652203013 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9e781055d0 msgr2=0x7f9e7819bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T15:05:16.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:16.508+0000 7f9e77400700 1 --2- 172.21.15.60:0/2652203013 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9e781055d0 0x7f9e7819bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:05:16.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:16.508+0000 7f9e77400700 1 -- 172.21.15.60:0/2652203013 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e7810b660 msgr2=0x7f9e781a0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T15:05:16.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:16.508+0000 7f9e77400700 1 --2- 172.21.15.60:0/2652203013 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e7810b660 0x7f9e781a0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:05:16.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:16.508+0000 7f9e77400700 1 -- 172.21.15.60:0/2652203013 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e781a0e60 con 0x7f9e78105f90 2024-05-03T15:05:16.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:16.508+0000 7f9e77400700 1 --2- 172.21.15.60:0/2652203013 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e78105f90 0x7f9e7819c3e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9e6400e9a0 tx=0x7f9e6400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T15:05:16.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:16.509+0000 7f9e74c00700 1 -- 172.21.15.60:0/2652203013 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6400cbd0 con 0x7f9e78105f90 2024-05-03T15:05:16.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:16.509+0000 7f9e7da00700 1 -- 172.21.15.60:0/2652203013 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e781a1150 con 0x7f9e78105f90 2024-05-03T15:05:16.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:16.509+0000 7f9e7da00700 1 -- 172.21.15.60:0/2652203013 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e781a89d0 con 0x7f9e78105f90 2024-05-03T15:05:16.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:16.509+0000 7f9e74c00700 1 -- 172.21.15.60:0/2652203013 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(28 keys) ==== 1170+0+0 (secure 0 0 0) 0x7f9e6400cd70 con 0x7f9e78105f90 2024-05-03T15:05:16.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:16.509+0000 7f9e74c00700 1 -- 172.21.15.60:0/2652203013 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e64018820 con 0x7f9e78105f90 2024-05-03T15:05:16.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:16.510+0000 7f9e7da00700 1 -- 172.21.15.60:0/2652203013 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e58005330 con 0x7f9e78105f90 2024-05-03T15:05:16.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:16.514+0000 7f9e74c00700 1 -- 172.21.15.60:0/2652203013 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f9e640189c0 con 0x7f9e78105f90 2024-05-03T15:05:16.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:16.515+0000 7f9e74c00700 1 --2- 172.21.15.60:0/2652203013 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f9e5c0b2990 0x7f9e5c0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T15:05:16.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:16.515+0000 7f9e74c00700 1 -- 172.21.15.60:0/2652203013 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8496+0+0 (secure 0 0 0) 0x7f9e640e2770 con 0x7f9e78105f90 2024-05-03T15:05:16.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:16.516+0000 7f9e77e00700 1 --2- 172.21.15.60:0/2652203013 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f9e5c0b2990 0x7f9e5c0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T15:05:16.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:16.516+0000 7f9e77e00700 1 --2- 172.21.15.60:0/2652203013 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f9e5c0b2990 0x7f9e5c0b4e50 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9e6800b2e0 tx=0x7f9e68005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T15:05:16.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:16.517+0000 7f9e74c00700 1 -- 172.21.15.60:0/2652203013 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f9e640aa1f0 con 0x7f9e78105f90 2024-05-03T15:05:16.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:16.761+0000 7f9e7da00700 1 -- 172.21.15.60:0/2652203013 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f9e58005620 con 0x7f9e78105f90 2024-05-03T15:05:16.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:16.823+0000 7f9e74c00700 1 -- 172.21.15.60:0/2652203013 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v93) ==== 178+0+0 (secure 0 0 0) 0x7f9e640b2270 con 0x7f9e78105f90 2024-05-03T15:05:16.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:16.830+0000 7f9e7da00700 1 -- 172.21.15.60:0/2652203013 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f9e5c0b2990 msgr2=0x7f9e5c0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T15:05:16.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:16.830+0000 7f9e7da00700 1 --2- 172.21.15.60:0/2652203013 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f9e5c0b2990 0x7f9e5c0b4e50 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9e6800b2e0 tx=0x7f9e68005fd0 comp rx=0 tx=0).stop 2024-05-03T15:05:16.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:16.830+0000 7f9e7da00700 1 -- 172.21.15.60:0/2652203013 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e78105f90 msgr2=0x7f9e7819c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T15:05:16.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:16.830+0000 7f9e7da00700 1 --2- 172.21.15.60:0/2652203013 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e78105f90 0x7f9e7819c3e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9e6400e9a0 tx=0x7f9e6400ecb0 comp rx=0 tx=0).stop 2024-05-03T15:05:16.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:16.831+0000 7f9e7da00700 1 -- 172.21.15.60:0/2652203013 shutdown_connections 2024-05-03T15:05:16.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:16.831+0000 7f9e7da00700 1 --2- 172.21.15.60:0/2652203013 >> [v2:172.21.15.60:6800/3066142956,v1:172.21.15.60:6801/3066142956] conn(0x7f9e5c0b2990 0x7f9e5c0b4e50 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:05:16.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:16.831+0000 7f9e7da00700 1 --2- 172.21.15.60:0/2652203013 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e7810b660 0x7f9e781a0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:05:16.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:16.831+0000 7f9e7da00700 1 --2- 172.21.15.60:0/2652203013 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9e781055d0 0x7f9e7819bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:05:16.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:16.831+0000 7f9e7da00700 1 --2- 172.21.15.60:0/2652203013 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e78105f90 0x7f9e7819c3e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T15:05:16.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:16.831+0000 7f9e7da00700 1 -- 172.21.15.60:0/2652203013 >> 172.21.15.60:0/2652203013 conn(0x7f9e781010c0 msgr2=0x7f9e78102640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T15:05:16.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:16.831+0000 7f9e7da00700 1 -- 172.21.15.60:0/2652203013 shutdown_connections 2024-05-03T15:05:16.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-05-03T15:05:16.832+0000 7f9e7da00700 1 -- 172.21.15.60:0/2652203013 wait complete. 2024-05-03T15:05:16.836 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data_ec' removed 2024-05-03T15:05:17.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:16 smithi060 ceph-mon[28873]: from='client.? 172.21.15.60:0/2652203013' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-03T15:05:17.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:16 smithi060 ceph-mon[28873]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-03T15:05:17.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:16 smithi090 ceph-mon[33568]: from='client.? 172.21.15.60:0/2652203013' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-03T15:05:17.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:16 smithi090 ceph-mon[33568]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-03T15:05:17.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:16 smithi149 ceph-mon[33332]: from='client.? 172.21.15.60:0/2652203013' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-03T15:05:17.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:16 smithi149 ceph-mon[33332]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-03T15:05:17.657 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-03T15:05:17.754 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-03T15:05:17.786 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-03T15:05:17.816 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-03T15:05:17.817 DEBUG:teuthology.orchestra.run.smithi060:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-03T15:05:17.875 DEBUG:teuthology.orchestra.run.smithi090:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-03T15:05:17.893 DEBUG:teuthology.orchestra.run.smithi149:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-03T15:05:17.909 INFO:tasks.cephadm:Stopping all daemons... 2024-05-03T15:05:17.909 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-05-03T15:05:17.910 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-285d64b2-095a-11ef-bc97-c7b262605968@mon.a 2024-05-03T15:05:18.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:17 smithi060 ceph-mon[28873]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-05-03T15:05:18.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:17 smithi060 ceph-mon[28873]: osdmap e93: 12 total, 12 up, 12 in 2024-05-03T15:05:18.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:17 smithi060 ceph-mon[28873]: pgmap v959: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T15:05:18.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:18 smithi060 systemd[1]: Stopping Ceph mon.a for 285d64b2-095a-11ef-bc97-c7b262605968... 2024-05-03T15:05:18.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:17 smithi090 ceph-mon[33568]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-05-03T15:05:18.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:17 smithi090 ceph-mon[33568]: osdmap e93: 12 total, 12 up, 12 in 2024-05-03T15:05:18.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:17 smithi090 ceph-mon[33568]: pgmap v959: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T15:05:18.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:17 smithi149 ceph-mon[33332]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-05-03T15:05:18.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:17 smithi149 ceph-mon[33332]: osdmap e93: 12 total, 12 up, 12 in 2024-05-03T15:05:18.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:17 smithi149 ceph-mon[33332]: pgmap v959: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T15:05:18.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:18 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mon-a[28869]: 2024-05-03T15:05:18.521+0000 7fc7ca800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T15:05:18.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:18 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mon-a[28869]: 2024-05-03T15:05:18.521+0000 7fc7ca800700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-05-03T15:05:18.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:18 smithi060 podman[90137]: 2024-05-03 15:05:18.70603916 +0000 UTC m=+0.308610703 container died 1149168d47be4367a47e8c3614bd2067ca5a7b86039f95aa814d0f49b8a0581d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-05-03T15:05:19.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:19 smithi060 podman[90137]: 2024-05-03 15:05:19.171849716 +0000 UTC m=+0.774421267 container cleanup 1149168d47be4367a47e8c3614bd2067ca5a7b86039f95aa814d0f49b8a0581d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mon-a, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-03T15:05:19.336 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:19 smithi060 bash[90137]: ceph-285d64b2-095a-11ef-bc97-c7b262605968-mon-a 2024-05-03T15:05:19.836 INFO:journalctl@ceph.mon.a.smithi060.stdout:May 03 15:05:19 smithi060 podman[90151]: 2024-05-03 15:05:19.456254639 +0000 UTC m=+0.732908515 container remove 1149168d47be4367a47e8c3614bd2067ca5a7b86039f95aa814d0f49b8a0581d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-03T15:05:19.925 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-285d64b2-095a-11ef-bc97-c7b262605968@mon.a.service' 2024-05-03T15:05:20.061 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T15:05:20.061 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-05-03T15:05:20.062 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-05-03T15:05:20.062 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-285d64b2-095a-11ef-bc97-c7b262605968@mon.b 2024-05-03T15:05:20.374 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:20 smithi090 systemd[1]: Stopping Ceph mon.b for 285d64b2-095a-11ef-bc97-c7b262605968... 2024-05-03T15:05:20.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:20 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mon-b[33564]: 2024-05-03T15:05:20.395+0000 7fb718bb6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T15:05:20.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:20 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mon-b[33564]: 2024-05-03T15:05:20.395+0000 7fb718bb6700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-05-03T15:05:20.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:20 smithi090 podman[68596]: 2024-05-03 15:05:20.59065742 +0000 UTC m=+0.329799733 container died 308ea0b9e17303fdf54eda1cdeca6021d969a8e5ed771797e9a6e5e12a710a35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mon-b, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-03T15:05:21.461 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:21 smithi090 podman[68596]: 2024-05-03 15:05:21.141958248 +0000 UTC m=+0.881100602 container cleanup 308ea0b9e17303fdf54eda1cdeca6021d969a8e5ed771797e9a6e5e12a710a35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430) 2024-05-03T15:05:21.462 INFO:journalctl@ceph.mon.b.smithi090.stdout:May 03 15:05:21 smithi090 bash[68596]: ceph-285d64b2-095a-11ef-bc97-c7b262605968-mon-b 2024-05-03T15:05:21.607 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-285d64b2-095a-11ef-bc97-c7b262605968@mon.b.service' 2024-05-03T15:05:21.649 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T15:05:21.649 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-05-03T15:05:21.650 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-05-03T15:05:21.650 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-285d64b2-095a-11ef-bc97-c7b262605968@mon.c 2024-05-03T15:05:21.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:21 smithi149 systemd[1]: Stopping Ceph mon.c for 285d64b2-095a-11ef-bc97-c7b262605968... 2024-05-03T15:05:22.231 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:21 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mon-c[33328]: 2024-05-03T15:05:21.963+0000 7f8daf07e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T15:05:22.231 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:21 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mon-c[33328]: 2024-05-03T15:05:21.963+0000 7f8daf07e700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-05-03T15:05:22.500 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:22 smithi149 podman[69412]: 2024-05-03 15:05:22.230181077 +0000 UTC m=+0.390181174 container died 2a9091c8d79b5595dca35e7ec7421628f58e612f25aaa6a5cff7876a0d652c0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mon-c, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-03T15:05:22.751 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:22 smithi149 podman[69412]: 2024-05-03 15:05:22.749482054 +0000 UTC m=+0.909482151 container cleanup 2a9091c8d79b5595dca35e7ec7421628f58e612f25aaa6a5cff7876a0d652c0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mon-c, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-03T15:05:22.752 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:22 smithi149 bash[69412]: ceph-285d64b2-095a-11ef-bc97-c7b262605968-mon-c 2024-05-03T15:05:23.227 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:22 smithi149 podman[69424]: 2024-05-03 15:05:22.976746005 +0000 UTC m=+0.751082027 container remove 2a9091c8d79b5595dca35e7ec7421628f58e612f25aaa6a5cff7876a0d652c0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mon-c, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T15:05:23.227 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:23 smithi149 systemd[1]: ceph-285d64b2-095a-11ef-bc97-c7b262605968@mon.c.service: Deactivated successfully. 2024-05-03T15:05:23.227 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:23 smithi149 systemd[1]: Stopped Ceph mon.c for 285d64b2-095a-11ef-bc97-c7b262605968. 2024-05-03T15:05:23.228 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 03 15:05:23 smithi149 systemd[1]: ceph-285d64b2-095a-11ef-bc97-c7b262605968@mon.c.service: Consumed 16.917s CPU time. 2024-05-03T15:05:23.228 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-285d64b2-095a-11ef-bc97-c7b262605968@mon.c.service' 2024-05-03T15:05:23.276 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T15:05:23.276 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-05-03T15:05:23.276 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-05-03T15:05:23.276 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-285d64b2-095a-11ef-bc97-c7b262605968@mgr.x 2024-05-03T15:05:23.836 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 15:05:23 smithi060 systemd[1]: Stopping Ceph mgr.x for 285d64b2-095a-11ef-bc97-c7b262605968... 2024-05-03T15:05:24.249 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 15:05:23 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T15:05:23.865+0000 7fd4b3a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T15:05:24.249 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 15:05:23 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x[29112]: 2024-05-03T15:05:23.866+0000 7fd4b3a00700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-05-03T15:05:24.249 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 15:05:23 smithi060 podman[90256]: 2024-05-03 15:05:23.959104803 +0000 UTC m=+0.251553854 container died 5813c13738d02eba3243414f18be1b0e883ff7e44e629fece7ae8498dcafe66a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux ) 2024-05-03T15:05:24.586 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 15:05:24 smithi060 podman[90256]: 2024-05-03 15:05:24.418085681 +0000 UTC m=+0.710534752 container cleanup 5813c13738d02eba3243414f18be1b0e883ff7e44e629fece7ae8498dcafe66a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-03T15:05:24.587 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 15:05:24 smithi060 bash[90256]: ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x 2024-05-03T15:05:25.086 INFO:journalctl@ceph.mgr.x.smithi060.stdout:May 03 15:05:24 smithi060 podman[90270]: 2024-05-03 15:05:24.686023924 +0000 UTC m=+0.713750884 container remove 5813c13738d02eba3243414f18be1b0e883ff7e44e629fece7ae8498dcafe66a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-x, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-03T15:05:25.242 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-285d64b2-095a-11ef-bc97-c7b262605968@mgr.x.service' 2024-05-03T15:05:25.361 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T15:05:25.361 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-05-03T15:05:25.361 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-05-03T15:05:25.361 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-285d64b2-095a-11ef-bc97-c7b262605968@mgr.y 2024-05-03T15:05:25.874 INFO:journalctl@ceph.mgr.y.smithi090.stdout:May 03 15:05:25 smithi090 systemd[1]: Stopping Ceph mgr.y for 285d64b2-095a-11ef-bc97-c7b262605968... 2024-05-03T15:05:26.152 INFO:journalctl@ceph.mgr.y.smithi090.stdout:May 03 15:05:25 smithi090 podman[68721]: 2024-05-03 15:05:25.878258665 +0000 UTC m=+0.321588909 container died d260c61b912b3f46c8ba3a613fc85edc1e3610330f6341af3c60b0b8915ec52c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2) 2024-05-03T15:05:26.512 INFO:journalctl@ceph.mgr.y.smithi090.stdout:May 03 15:05:26 smithi090 podman[68721]: 2024-05-03 15:05:26.251889567 +0000 UTC m=+0.695219810 container cleanup d260c61b912b3f46c8ba3a613fc85edc1e3610330f6341af3c60b0b8915ec52c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD) 2024-05-03T15:05:26.513 INFO:journalctl@ceph.mgr.y.smithi090.stdout:May 03 15:05:26 smithi090 bash[68721]: ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-y 2024-05-03T15:05:26.679 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-285d64b2-095a-11ef-bc97-c7b262605968@mgr.y.service' 2024-05-03T15:05:26.712 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T15:05:26.713 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-05-03T15:05:26.713 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-05-03T15:05:26.713 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-285d64b2-095a-11ef-bc97-c7b262605968@mgr.z 2024-05-03T15:05:27.043 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 03 15:05:26 smithi149 systemd[1]: Stopping Ceph mgr.z for 285d64b2-095a-11ef-bc97-c7b262605968... 2024-05-03T15:05:27.422 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 03 15:05:27 smithi149 podman[69545]: 2024-05-03 15:05:27.042968782 +0000 UTC m=+0.136429285 container died 5a2811f816a5be827827d19c52011014f1a3e4b1098f80f1aa91b055798e90a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-z, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240430) 2024-05-03T15:05:27.799 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 03 15:05:27 smithi149 podman[69545]: 2024-05-03 15:05:27.530552588 +0000 UTC m=+0.624013083 container cleanup 5a2811f816a5be827827d19c52011014f1a3e4b1098f80f1aa91b055798e90a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-z, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-03T15:05:27.799 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 03 15:05:27 smithi149 bash[69545]: ceph-285d64b2-095a-11ef-bc97-c7b262605968-mgr-z 2024-05-03T15:05:27.960 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-285d64b2-095a-11ef-bc97-c7b262605968@mgr.z.service' 2024-05-03T15:05:27.995 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T15:05:27.995 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-05-03T15:05:27.995 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-03T15:05:27.995 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-285d64b2-095a-11ef-bc97-c7b262605968@osd.0 2024-05-03T15:05:28.336 INFO:journalctl@ceph.osd.0.smithi060.stdout:May 03 15:05:28 smithi060 systemd[1]: Stopping Ceph osd.0 for 285d64b2-095a-11ef-bc97-c7b262605968... 2024-05-03T15:05:29.087 INFO:journalctl@ceph.osd.0.smithi060.stdout:May 03 15:05:28 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-0[40916]: 2024-05-03T15:05:28.644+0000 7fc1fa800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T15:05:29.087 INFO:journalctl@ceph.osd.0.smithi060.stdout:May 03 15:05:28 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-0[40916]: 2024-05-03T15:05:28.644+0000 7fc1fa800700 -1 osd.0 93 *** Got signal Terminated *** 2024-05-03T15:05:29.087 INFO:journalctl@ceph.osd.0.smithi060.stdout:May 03 15:05:28 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-0[40916]: 2024-05-03T15:05:28.644+0000 7fc1fa800700 -1 osd.0 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-03T15:05:34.336 INFO:journalctl@ceph.osd.0.smithi060.stdout:May 03 15:05:34 smithi060 podman[90378]: 2024-05-03 15:05:34.032971211 +0000 UTC m=+5.551530441 container died 230339d13b11ce91818f98de60b79ac9de689afda700379dea9bbf39a6d93baf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-05-03T15:05:34.758 INFO:journalctl@ceph.osd.0.smithi060.stdout:May 03 15:05:34 smithi060 podman[90378]: 2024-05-03 15:05:34.4890209 +0000 UTC m=+6.007580168 container cleanup 230339d13b11ce91818f98de60b79ac9de689afda700379dea9bbf39a6d93baf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-03T15:05:34.758 INFO:journalctl@ceph.osd.0.smithi060.stdout:May 03 15:05:34 smithi060 bash[90378]: ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-0 2024-05-03T15:05:35.086 INFO:journalctl@ceph.osd.0.smithi060.stdout:May 03 15:05:34 smithi060 podman[90399]: 2024-05-03 15:05:34.757074936 +0000 UTC m=+0.713122633 container remove 230339d13b11ce91818f98de60b79ac9de689afda700379dea9bbf39a6d93baf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-05-03T15:05:35.934 INFO:journalctl@ceph.osd.0.smithi060.stdout:May 03 15:05:35 smithi060 podman[90488]: 2024-05-03 15:05:35.583413609 +0000 UTC m=+0.033586742 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T15:05:35.935 INFO:journalctl@ceph.osd.0.smithi060.stdout:May 03 15:05:35 smithi060 podman[90488]: 2024-05-03 15:05:35.707963517 +0000 UTC m=+0.158136675 container create 6e51f07ab15bd09700af5b342d98e50a759b25a013017cd9ffdc01cd734e0435 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=) 2024-05-03T15:05:36.336 INFO:journalctl@ceph.osd.0.smithi060.stdout:May 03 15:05:36 smithi060 podman[90488]: 2024-05-03 15:05:36.04658584 +0000 UTC m=+0.496759038 container init 6e51f07ab15bd09700af5b342d98e50a759b25a013017cd9ffdc01cd734e0435 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-03T15:05:36.337 INFO:journalctl@ceph.osd.0.smithi060.stdout:May 03 15:05:36 smithi060 podman[90488]: 2024-05-03 15:05:36.059748049 +0000 UTC m=+0.509921214 container start 6e51f07ab15bd09700af5b342d98e50a759b25a013017cd9ffdc01cd734e0435 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T15:05:36.337 INFO:journalctl@ceph.osd.0.smithi060.stdout:May 03 15:05:36 smithi060 podman[90488]: 2024-05-03 15:05:36.160696457 +0000 UTC m=+0.610869629 container attach 6e51f07ab15bd09700af5b342d98e50a759b25a013017cd9ffdc01cd734e0435 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-0-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-05-03T15:05:36.836 INFO:journalctl@ceph.osd.0.smithi060.stdout:May 03 15:05:36 smithi060 podman[90488]: 2024-05-03 15:05:36.535956518 +0000 UTC m=+0.986129682 container died 6e51f07ab15bd09700af5b342d98e50a759b25a013017cd9ffdc01cd734e0435 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-03T15:05:37.317 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-285d64b2-095a-11ef-bc97-c7b262605968@osd.0.service' 2024-05-03T15:05:37.433 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T15:05:37.434 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-03T15:05:37.434 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-03T15:05:37.434 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-285d64b2-095a-11ef-bc97-c7b262605968@osd.1 2024-05-03T15:05:37.743 INFO:journalctl@ceph.osd.1.smithi090.stdout:May 03 15:05:37 smithi090 systemd[1]: Stopping Ceph osd.1 for 285d64b2-095a-11ef-bc97-c7b262605968... 2024-05-03T15:05:38.124 INFO:journalctl@ceph.osd.1.smithi090.stdout:May 03 15:05:37 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-1[38156]: 2024-05-03T15:05:37.740+0000 7f2b26326700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T15:05:38.124 INFO:journalctl@ceph.osd.1.smithi090.stdout:May 03 15:05:37 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-1[38156]: 2024-05-03T15:05:37.740+0000 7f2b26326700 -1 osd.1 93 *** Got signal Terminated *** 2024-05-03T15:05:38.124 INFO:journalctl@ceph.osd.1.smithi090.stdout:May 03 15:05:37 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-1[38156]: 2024-05-03T15:05:37.740+0000 7f2b26326700 -1 osd.1 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-03T15:05:43.374 INFO:journalctl@ceph.osd.1.smithi090.stdout:May 03 15:05:43 smithi090 podman[68857]: 2024-05-03 15:05:43.087806957 +0000 UTC m=+5.465714322 container died 0da45666f424e8dd5c2cdc7a5fc14112cb6f800732bb17983e20a8a62354235f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-03T15:05:43.747 INFO:journalctl@ceph.osd.1.smithi090.stdout:May 03 15:05:43 smithi090 podman[68857]: 2024-05-03 15:05:43.486329864 +0000 UTC m=+5.864237219 container cleanup 0da45666f424e8dd5c2cdc7a5fc14112cb6f800732bb17983e20a8a62354235f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-1, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T15:05:43.747 INFO:journalctl@ceph.osd.1.smithi090.stdout:May 03 15:05:43 smithi090 bash[68857]: ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-1 2024-05-03T15:05:44.045 INFO:journalctl@ceph.osd.1.smithi090.stdout:May 03 15:05:43 smithi090 podman[68874]: 2024-05-03 15:05:43.746585582 +0000 UTC m=+0.661564251 container remove 0da45666f424e8dd5c2cdc7a5fc14112cb6f800732bb17983e20a8a62354235f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-1, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-03T15:05:44.301 INFO:journalctl@ceph.osd.1.smithi090.stdout:May 03 15:05:44 smithi090 podman[68966]: 2024-05-03 15:05:43.947716718 +0000 UTC m=+0.017587033 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T15:05:44.301 INFO:journalctl@ceph.osd.1.smithi090.stdout:May 03 15:05:44 smithi090 podman[68966]: 2024-05-03 15:05:44.056206965 +0000 UTC m=+0.126077312 container create 6a184bf553e40badc71af14df335d70890059c2a8b6589742091e3e22689759a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-03T15:05:44.302 INFO:journalctl@ceph.osd.1.smithi090.stdout:May 03 15:05:44 smithi090 podman[68966]: 2024-05-03 15:05:44.301154981 +0000 UTC m=+0.371025287 container init 6a184bf553e40badc71af14df335d70890059c2a8b6589742091e3e22689759a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-05-03T15:05:44.596 INFO:journalctl@ceph.osd.1.smithi090.stdout:May 03 15:05:44 smithi090 podman[68966]: 2024-05-03 15:05:44.304791547 +0000 UTC m=+0.374661858 container start 6a184bf553e40badc71af14df335d70890059c2a8b6589742091e3e22689759a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, ceph=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-03T15:05:44.597 INFO:journalctl@ceph.osd.1.smithi090.stdout:May 03 15:05:44 smithi090 podman[68966]: 2024-05-03 15:05:44.411086076 +0000 UTC m=+0.480956397 container attach 6a184bf553e40badc71af14df335d70890059c2a8b6589742091e3e22689759a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-03T15:05:44.873 INFO:journalctl@ceph.osd.1.smithi090.stdout:May 03 15:05:44 smithi090 podman[68966]: 2024-05-03 15:05:44.596586071 +0000 UTC m=+0.666456376 container died 6a184bf553e40badc71af14df335d70890059c2a8b6589742091e3e22689759a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, io.buildah.version=1.33.5) 2024-05-03T15:05:45.170 INFO:journalctl@ceph.osd.1.smithi090.stdout:May 03 15:05:45 smithi090 podman[68992]: 2024-05-03 15:05:45.149776639 +0000 UTC m=+0.544137895 container remove 6a184bf553e40badc71af14df335d70890059c2a8b6589742091e3e22689759a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-1-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T15:05:45.175 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-285d64b2-095a-11ef-bc97-c7b262605968@osd.1.service' 2024-05-03T15:05:45.212 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T15:05:45.212 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-03T15:05:45.212 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-03T15:05:45.213 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-285d64b2-095a-11ef-bc97-c7b262605968@osd.2 2024-05-03T15:05:45.494 INFO:journalctl@ceph.osd.2.smithi149.stdout:May 03 15:05:45 smithi149 systemd[1]: Stopping Ceph osd.2 for 285d64b2-095a-11ef-bc97-c7b262605968... 2024-05-03T15:05:45.922 INFO:journalctl@ceph.osd.2.smithi149.stdout:May 03 15:05:45 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-2[38497]: 2024-05-03T15:05:45.493+0000 7f0fc97aa700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T15:05:45.922 INFO:journalctl@ceph.osd.2.smithi149.stdout:May 03 15:05:45 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-2[38497]: 2024-05-03T15:05:45.493+0000 7f0fc97aa700 -1 osd.2 93 *** Got signal Terminated *** 2024-05-03T15:05:45.922 INFO:journalctl@ceph.osd.2.smithi149.stdout:May 03 15:05:45 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-2[38497]: 2024-05-03T15:05:45.493+0000 7f0fc97aa700 -1 osd.2 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-03T15:05:51.361 INFO:journalctl@ceph.osd.2.smithi149.stdout:May 03 15:05:51 smithi149 podman[69671]: 2024-05-03 15:05:51.020663707 +0000 UTC m=+5.617891082 container died e1af27f902a91325ebc7267318fce70740da57d5ab21209fa72b04f152126f54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-03T15:05:51.619 INFO:journalctl@ceph.osd.2.smithi149.stdout:May 03 15:05:51 smithi149 podman[69671]: 2024-05-03 15:05:51.45182974 +0000 UTC m=+6.049057102 container cleanup e1af27f902a91325ebc7267318fce70740da57d5ab21209fa72b04f152126f54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-2, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-05-03T15:05:51.620 INFO:journalctl@ceph.osd.2.smithi149.stdout:May 03 15:05:51 smithi149 bash[69671]: ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-2 2024-05-03T15:05:51.922 INFO:journalctl@ceph.osd.2.smithi149.stdout:May 03 15:05:51 smithi149 podman[69688]: 2024-05-03 15:05:51.619640468 +0000 UTC m=+0.597468801 container remove e1af27f902a91325ebc7267318fce70740da57d5ab21209fa72b04f152126f54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-05-03T15:05:52.367 INFO:journalctl@ceph.osd.2.smithi149.stdout:May 03 15:05:52 smithi149 podman[69782]: 2024-05-03 15:05:51.965415875 +0000 UTC m=+0.018787213 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T15:05:52.367 INFO:journalctl@ceph.osd.2.smithi149.stdout:May 03 15:05:52 smithi149 podman[69782]: 2024-05-03 15:05:52.103848251 +0000 UTC m=+0.157219564 container create 4b323c6c071b4f614591c53a3cdabf1a7af3ccbf2dd2bc9b1a8d0be72c2145e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-03T15:05:52.667 INFO:journalctl@ceph.osd.2.smithi149.stdout:May 03 15:05:52 smithi149 podman[69782]: 2024-05-03 15:05:52.367254583 +0000 UTC m=+0.420625909 container init 4b323c6c071b4f614591c53a3cdabf1a7af3ccbf2dd2bc9b1a8d0be72c2145e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-03T15:05:52.667 INFO:journalctl@ceph.osd.2.smithi149.stdout:May 03 15:05:52 smithi149 podman[69782]: 2024-05-03 15:05:52.370489955 +0000 UTC m=+0.423861270 container start 4b323c6c071b4f614591c53a3cdabf1a7af3ccbf2dd2bc9b1a8d0be72c2145e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-05-03T15:05:52.667 INFO:journalctl@ceph.osd.2.smithi149.stdout:May 03 15:05:52 smithi149 podman[69782]: 2024-05-03 15:05:52.467458147 +0000 UTC m=+0.520829461 container attach 4b323c6c071b4f614591c53a3cdabf1a7af3ccbf2dd2bc9b1a8d0be72c2145e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-2-deactivate, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD) 2024-05-03T15:05:52.922 INFO:journalctl@ceph.osd.2.smithi149.stdout:May 03 15:05:52 smithi149 podman[69782]: 2024-05-03 15:05:52.666787732 +0000 UTC m=+0.720159064 container died 4b323c6c071b4f614591c53a3cdabf1a7af3ccbf2dd2bc9b1a8d0be72c2145e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430) 2024-05-03T15:05:53.347 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-285d64b2-095a-11ef-bc97-c7b262605968@osd.2.service' 2024-05-03T15:05:53.385 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T15:05:53.385 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-03T15:05:53.386 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-03T15:05:53.386 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-285d64b2-095a-11ef-bc97-c7b262605968@osd.3 2024-05-03T15:05:53.806 INFO:journalctl@ceph.osd.3.smithi060.stdout:May 03 15:05:53 smithi060 systemd[1]: Stopping Ceph osd.3 for 285d64b2-095a-11ef-bc97-c7b262605968... 2024-05-03T15:05:54.087 INFO:journalctl@ceph.osd.3.smithi060.stdout:May 03 15:05:53 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-3[47440]: 2024-05-03T15:05:53.980+0000 7fb579c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T15:05:54.087 INFO:journalctl@ceph.osd.3.smithi060.stdout:May 03 15:05:53 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-3[47440]: 2024-05-03T15:05:53.980+0000 7fb579c00700 -1 osd.3 93 *** Got signal Terminated *** 2024-05-03T15:05:54.087 INFO:journalctl@ceph.osd.3.smithi060.stdout:May 03 15:05:53 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-3[47440]: 2024-05-03T15:05:53.980+0000 7fb579c00700 -1 osd.3 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-03T15:05:55.422 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 03 15:05:55 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-5[45461]: 2024-05-03T15:05:55.089+0000 7f50e4577700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.375047+0000 front 2024-05-03T15:05:31.375384+0000 (oldest deadline 2024-05-03T15:05:54.274777+0000) 2024-05-03T15:05:55.534 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:05:55 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:05:55.352+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.184568+0000 front 2024-05-03T15:05:32.184943+0000 (oldest deadline 2024-05-03T15:05:55.084762+0000) 2024-05-03T15:05:56.118 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:05:55 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:05:55.790+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:05:56.422 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 03 15:05:56 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-5[45461]: 2024-05-03T15:05:56.117+0000 7f50e4577700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.375047+0000 front 2024-05-03T15:05:31.375384+0000 (oldest deadline 2024-05-03T15:05:54.274777+0000) 2024-05-03T15:05:56.586 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:05:56 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:05:56.303+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.184568+0000 front 2024-05-03T15:05:32.184943+0000 (oldest deadline 2024-05-03T15:05:55.084762+0000) 2024-05-03T15:05:57.134 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:05:56 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:05:56.776+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:05:57.422 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 03 15:05:57 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-5[45461]: 2024-05-03T15:05:57.133+0000 7f50e4577700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.375047+0000 front 2024-05-03T15:05:31.375384+0000 (oldest deadline 2024-05-03T15:05:54.274777+0000) 2024-05-03T15:05:57.589 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:05:57 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:05:57.256+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.184568+0000 front 2024-05-03T15:05:32.184943+0000 (oldest deadline 2024-05-03T15:05:55.084762+0000) 2024-05-03T15:05:58.099 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:05:57 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:05:57.813+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:05:58.422 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 03 15:05:58 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-5[45461]: 2024-05-03T15:05:58.098+0000 7f50e4577700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.375047+0000 front 2024-05-03T15:05:31.375384+0000 (oldest deadline 2024-05-03T15:05:54.274777+0000) 2024-05-03T15:05:58.586 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:05:58 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:05:58.275+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.184568+0000 front 2024-05-03T15:05:32.184943+0000 (oldest deadline 2024-05-03T15:05:55.084762+0000) 2024-05-03T15:05:59.109 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:05:58 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:05:58.798+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:05:59.257 INFO:journalctl@ceph.osd.3.smithi060.stdout:May 03 15:05:58 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-3[47440]: 2024-05-03T15:05:58.943+0000 7fb575400700 -1 osd.3 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.889441+0000 front 2024-05-03T15:05:32.888381+0000 (oldest deadline 2024-05-03T15:05:58.689849+0000) 2024-05-03T15:05:59.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:05:58 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:05:58.878+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.538644+0000 front 2024-05-03T15:05:32.538760+0000 (oldest deadline 2024-05-03T15:05:58.438503+0000) 2024-05-03T15:05:59.422 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 03 15:05:59 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-5[45461]: 2024-05-03T15:05:59.108+0000 7f50e4577700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.375047+0000 front 2024-05-03T15:05:31.375384+0000 (oldest deadline 2024-05-03T15:05:54.274777+0000) 2024-05-03T15:05:59.587 INFO:journalctl@ceph.osd.3.smithi060.stdout:May 03 15:05:59 smithi060 podman[90593]: 2024-05-03 15:05:59.408202902 +0000 UTC m=+5.571762423 container died c18a7a43690179dc791d47ffb2da93011636727f675d4a9c3119bd1296982e60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-3, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-03T15:05:59.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:05:59 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:05:59.255+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.184568+0000 front 2024-05-03T15:05:32.184943+0000 (oldest deadline 2024-05-03T15:05:55.084762+0000) 2024-05-03T15:06:00.085 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:05:59 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:05:59.753+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:06:00.086 INFO:journalctl@ceph.osd.3.smithi060.stdout:May 03 15:05:59 smithi060 podman[90593]: 2024-05-03 15:05:59.874608962 +0000 UTC m=+6.038168472 container cleanup c18a7a43690179dc791d47ffb2da93011636727f675d4a9c3119bd1296982e60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-3, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T15:06:00.086 INFO:journalctl@ceph.osd.3.smithi060.stdout:May 03 15:05:59 smithi060 bash[90593]: ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-3 2024-05-03T15:06:00.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:05:59 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:05:59.919+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.538644+0000 front 2024-05-03T15:05:32.538760+0000 (oldest deadline 2024-05-03T15:05:58.438503+0000) 2024-05-03T15:06:00.422 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 03 15:06:00 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-5[45461]: 2024-05-03T15:06:00.085+0000 7f50e4577700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.375047+0000 front 2024-05-03T15:05:31.375384+0000 (oldest deadline 2024-05-03T15:05:54.274777+0000) 2024-05-03T15:06:00.586 INFO:journalctl@ceph.osd.3.smithi060.stdout:May 03 15:06:00 smithi060 podman[90623]: 2024-05-03 15:06:00.092432835 +0000 UTC m=+0.675130855 container remove c18a7a43690179dc791d47ffb2da93011636727f675d4a9c3119bd1296982e60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-3, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-03T15:06:00.586 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:00 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:00.214+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.184568+0000 front 2024-05-03T15:05:32.184943+0000 (oldest deadline 2024-05-03T15:05:55.084762+0000) 2024-05-03T15:06:01.069 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:00 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:00.760+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:06:01.247 INFO:journalctl@ceph.osd.3.smithi060.stdout:May 03 15:06:00 smithi060 podman[90710]: 2024-05-03 15:06:00.837021429 +0000 UTC m=+0.039888790 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T15:06:01.248 INFO:journalctl@ceph.osd.3.smithi060.stdout:May 03 15:06:00 smithi060 podman[90710]: 2024-05-03 15:06:00.960103192 +0000 UTC m=+0.162970585 container create 23da8f769cf45e8154411f65f6bc994350d166ce5ebb47c380c9012503c6808f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True) 2024-05-03T15:06:01.373 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:00 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:00.958+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.538644+0000 front 2024-05-03T15:05:32.538760+0000 (oldest deadline 2024-05-03T15:05:58.438503+0000) 2024-05-03T15:06:01.422 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 03 15:06:01 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-5[45461]: 2024-05-03T15:06:01.068+0000 7f50e4577700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.375047+0000 front 2024-05-03T15:05:31.375384+0000 (oldest deadline 2024-05-03T15:05:54.274777+0000) 2024-05-03T15:06:01.586 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:01 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:01.245+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.184568+0000 front 2024-05-03T15:05:32.184943+0000 (oldest deadline 2024-05-03T15:05:55.084762+0000) 2024-05-03T15:06:01.587 INFO:journalctl@ceph.osd.3.smithi060.stdout:May 03 15:06:01 smithi060 podman[90710]: 2024-05-03 15:06:01.256032731 +0000 UTC m=+0.458900148 container init 23da8f769cf45e8154411f65f6bc994350d166ce5ebb47c380c9012503c6808f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS) 2024-05-03T15:06:01.587 INFO:journalctl@ceph.osd.3.smithi060.stdout:May 03 15:06:01 smithi060 podman[90710]: 2024-05-03 15:06:01.269155554 +0000 UTC m=+0.472022953 container start 23da8f769cf45e8154411f65f6bc994350d166ce5ebb47c380c9012503c6808f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-03T15:06:01.587 INFO:journalctl@ceph.osd.3.smithi060.stdout:May 03 15:06:01 smithi060 podman[90710]: 2024-05-03 15:06:01.354688639 +0000 UTC m=+0.557556059 container attach 23da8f769cf45e8154411f65f6bc994350d166ce5ebb47c380c9012503c6808f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_BRANCH=HEAD) 2024-05-03T15:06:02.086 INFO:journalctl@ceph.osd.3.smithi060.stdout:May 03 15:06:01 smithi060 podman[90710]: 2024-05-03 15:06:01.742847242 +0000 UTC m=+0.945714632 container died 23da8f769cf45e8154411f65f6bc994350d166ce5ebb47c380c9012503c6808f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.schema-version=1.0) 2024-05-03T15:06:02.100 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:01 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:01.717+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:06:02.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:01 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:01.935+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.538644+0000 front 2024-05-03T15:05:32.538760+0000 (oldest deadline 2024-05-03T15:05:58.438503+0000) 2024-05-03T15:06:02.422 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 03 15:06:02 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-5[45461]: 2024-05-03T15:06:02.099+0000 7f50e4577700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.375047+0000 front 2024-05-03T15:05:31.375384+0000 (oldest deadline 2024-05-03T15:05:54.274777+0000) 2024-05-03T15:06:02.426 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:02 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:02.252+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.184568+0000 front 2024-05-03T15:05:32.184943+0000 (oldest deadline 2024-05-03T15:05:55.084762+0000) 2024-05-03T15:06:02.475 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-285d64b2-095a-11ef-bc97-c7b262605968@osd.3.service' 2024-05-03T15:06:02.601 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T15:06:02.601 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-03T15:06:02.602 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-03T15:06:02.602 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-285d64b2-095a-11ef-bc97-c7b262605968@osd.4 2024-05-03T15:06:02.906 INFO:journalctl@ceph.osd.4.smithi090.stdout:May 03 15:06:02 smithi090 systemd[1]: Stopping Ceph osd.4 for 285d64b2-095a-11ef-bc97-c7b262605968... 2024-05-03T15:06:03.063 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:02 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:02.738+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:06:03.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:02 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:02.920+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.538644+0000 front 2024-05-03T15:05:32.538760+0000 (oldest deadline 2024-05-03T15:05:58.438503+0000) 2024-05-03T15:06:03.374 INFO:journalctl@ceph.osd.4.smithi090.stdout:May 03 15:06:02 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-4[45126]: 2024-05-03T15:06:02.905+0000 7ff3a6c82700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T15:06:03.374 INFO:journalctl@ceph.osd.4.smithi090.stdout:May 03 15:06:02 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-4[45126]: 2024-05-03T15:06:02.905+0000 7ff3a6c82700 -1 osd.4 93 *** Got signal Terminated *** 2024-05-03T15:06:03.374 INFO:journalctl@ceph.osd.4.smithi090.stdout:May 03 15:06:02 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-4[45126]: 2024-05-03T15:06:02.905+0000 7ff3a6c82700 -1 osd.4 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-03T15:06:03.422 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 03 15:06:03 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-5[45461]: 2024-05-03T15:06:03.062+0000 7f50e4577700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.375047+0000 front 2024-05-03T15:05:31.375384+0000 (oldest deadline 2024-05-03T15:05:54.274777+0000) 2024-05-03T15:06:03.586 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:03 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:03.294+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.184568+0000 front 2024-05-03T15:05:32.184943+0000 (oldest deadline 2024-05-03T15:05:55.084762+0000) 2024-05-03T15:06:04.028 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:03 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:03.724+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:06:04.275 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:03 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:03.926+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.538644+0000 front 2024-05-03T15:05:32.538760+0000 (oldest deadline 2024-05-03T15:05:58.438503+0000) 2024-05-03T15:06:04.422 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 03 15:06:04 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-5[45461]: 2024-05-03T15:06:04.104+0000 7f50e4577700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.375047+0000 front 2024-05-03T15:05:31.375384+0000 (oldest deadline 2024-05-03T15:05:54.274777+0000) 2024-05-03T15:06:04.547 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:04 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:04.307+0000 7f487aafd700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:41.153998+0000 front 2024-05-03T15:05:41.154137+0000 (oldest deadline 2024-05-03T15:06:03.454048+0000) 2024-05-03T15:06:04.836 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:04 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:04.344+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.184568+0000 front 2024-05-03T15:05:32.184943+0000 (oldest deadline 2024-05-03T15:05:55.084762+0000) 2024-05-03T15:06:04.836 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:04 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:04.344+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.788481+0000 front 2024-05-03T15:05:40.788594+0000 (oldest deadline 2024-05-03T15:06:04.288673+0000) 2024-05-03T15:06:04.922 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:04 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:04.729+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:06:04.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:04 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:04.729+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:37.896657+0000 front 2024-05-03T15:05:37.896645+0000 (oldest deadline 2024-05-03T15:06:03.796605+0000) 2024-05-03T15:06:05.332 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:04 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:04.903+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.538644+0000 front 2024-05-03T15:05:32.538760+0000 (oldest deadline 2024-05-03T15:05:58.438503+0000) 2024-05-03T15:06:05.422 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 03 15:06:05 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-5[45461]: 2024-05-03T15:06:05.100+0000 7f50e4577700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.375047+0000 front 2024-05-03T15:05:31.375384+0000 (oldest deadline 2024-05-03T15:05:54.274777+0000) 2024-05-03T15:06:05.422 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 03 15:06:05 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-5[45461]: 2024-05-03T15:06:05.100+0000 7f50e4577700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:41.677763+0000 front 2024-05-03T15:05:41.677677+0000 (oldest deadline 2024-05-03T15:06:05.078219+0000) 2024-05-03T15:06:05.423 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:05 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:05.105+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.098256+0000 front 2024-05-03T15:05:40.098316+0000 (oldest deadline 2024-05-03T15:06:04.098995+0000) 2024-05-03T15:06:05.624 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:05 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:05.331+0000 7f487aafd700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:41.153998+0000 front 2024-05-03T15:05:41.154137+0000 (oldest deadline 2024-05-03T15:06:03.454048+0000) 2024-05-03T15:06:05.836 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:05 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:05.367+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.184568+0000 front 2024-05-03T15:05:32.184943+0000 (oldest deadline 2024-05-03T15:05:55.084762+0000) 2024-05-03T15:06:05.836 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:05 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:05.367+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.788481+0000 front 2024-05-03T15:05:40.788594+0000 (oldest deadline 2024-05-03T15:06:04.288673+0000) 2024-05-03T15:06:06.118 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:05 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:05.779+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:06:06.119 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:05 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:05.779+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:37.896657+0000 front 2024-05-03T15:05:37.896645+0000 (oldest deadline 2024-05-03T15:06:03.796605+0000) 2024-05-03T15:06:06.123 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:05 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:05.862+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.538644+0000 front 2024-05-03T15:05:32.538760+0000 (oldest deadline 2024-05-03T15:05:58.438503+0000) 2024-05-03T15:06:06.422 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 03 15:06:06 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-5[45461]: 2024-05-03T15:06:06.146+0000 7f50e4577700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.375047+0000 front 2024-05-03T15:05:31.375384+0000 (oldest deadline 2024-05-03T15:05:54.274777+0000) 2024-05-03T15:06:06.422 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 03 15:06:06 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-5[45461]: 2024-05-03T15:06:06.146+0000 7f50e4577700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:41.677763+0000 front 2024-05-03T15:05:41.677677+0000 (oldest deadline 2024-05-03T15:06:05.078219+0000) 2024-05-03T15:06:06.423 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:06 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:06.118+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.098256+0000 front 2024-05-03T15:05:40.098316+0000 (oldest deadline 2024-05-03T15:06:04.098995+0000) 2024-05-03T15:06:06.836 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:06 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:06.394+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.184568+0000 front 2024-05-03T15:05:32.184943+0000 (oldest deadline 2024-05-03T15:05:55.084762+0000) 2024-05-03T15:06:06.836 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:06 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:06.394+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.788481+0000 front 2024-05-03T15:05:40.788594+0000 (oldest deadline 2024-05-03T15:06:04.288673+0000) 2024-05-03T15:06:06.855 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:06 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:06.378+0000 7f487aafd700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:41.153998+0000 front 2024-05-03T15:05:41.154137+0000 (oldest deadline 2024-05-03T15:06:03.454048+0000) 2024-05-03T15:06:06.856 INFO:journalctl@ceph.osd.4.smithi090.stdout:May 03 15:06:06 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-4[45126]: 2024-05-03T15:06:06.609+0000 7ff39f26e700 -1 osd.4 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:41.921138+0000 front 2024-05-03T15:05:41.921162+0000 (oldest deadline 2024-05-03T15:06:05.921902+0000) 2024-05-03T15:06:07.085 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:06 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:06.817+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:06:07.085 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:06 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:06.817+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:37.896657+0000 front 2024-05-03T15:05:37.896645+0000 (oldest deadline 2024-05-03T15:06:03.796605+0000) 2024-05-03T15:06:07.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:06 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:06.855+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.538644+0000 front 2024-05-03T15:05:32.538760+0000 (oldest deadline 2024-05-03T15:05:58.438503+0000) 2024-05-03T15:06:07.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:06 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:06.855+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:42.440172+0000 front 2024-05-03T15:05:42.440196+0000 (oldest deadline 2024-05-03T15:06:05.940332+0000) 2024-05-03T15:06:07.422 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 03 15:06:07 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-5[45461]: 2024-05-03T15:06:07.104+0000 7f50e4577700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.375047+0000 front 2024-05-03T15:05:31.375384+0000 (oldest deadline 2024-05-03T15:05:54.274777+0000) 2024-05-03T15:06:07.423 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 03 15:06:07 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-5[45461]: 2024-05-03T15:06:07.104+0000 7f50e4577700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:41.677763+0000 front 2024-05-03T15:05:41.677677+0000 (oldest deadline 2024-05-03T15:06:05.078219+0000) 2024-05-03T15:06:07.423 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:07 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:07.084+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.098256+0000 front 2024-05-03T15:05:40.098316+0000 (oldest deadline 2024-05-03T15:06:04.098995+0000) 2024-05-03T15:06:07.825 INFO:journalctl@ceph.osd.4.smithi090.stdout:May 03 15:06:07 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-4[45126]: 2024-05-03T15:06:07.572+0000 7ff39f26e700 -1 osd.4 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:41.921138+0000 front 2024-05-03T15:05:41.921162+0000 (oldest deadline 2024-05-03T15:06:05.921902+0000) 2024-05-03T15:06:07.826 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:07 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:07.422+0000 7f487aafd700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:41.153998+0000 front 2024-05-03T15:05:41.154137+0000 (oldest deadline 2024-05-03T15:06:03.454048+0000) 2024-05-03T15:06:07.836 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:07 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:07.354+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.184568+0000 front 2024-05-03T15:05:32.184943+0000 (oldest deadline 2024-05-03T15:05:55.084762+0000) 2024-05-03T15:06:07.836 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:07 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:07.354+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.788481+0000 front 2024-05-03T15:05:40.788594+0000 (oldest deadline 2024-05-03T15:06:04.288673+0000) 2024-05-03T15:06:08.106 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:07 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:07.805+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:06:08.106 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:07 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:07.805+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:37.896657+0000 front 2024-05-03T15:05:37.896645+0000 (oldest deadline 2024-05-03T15:06:03.796605+0000) 2024-05-03T15:06:08.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:07 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:07.825+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.538644+0000 front 2024-05-03T15:05:32.538760+0000 (oldest deadline 2024-05-03T15:05:58.438503+0000) 2024-05-03T15:06:08.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:07 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:07.825+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:42.440172+0000 front 2024-05-03T15:05:42.440196+0000 (oldest deadline 2024-05-03T15:06:05.940332+0000) 2024-05-03T15:06:08.422 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 03 15:06:08 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-5[45461]: 2024-05-03T15:06:08.105+0000 7f50e4577700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.375047+0000 front 2024-05-03T15:05:31.375384+0000 (oldest deadline 2024-05-03T15:05:54.274777+0000) 2024-05-03T15:06:08.423 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 03 15:06:08 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-5[45461]: 2024-05-03T15:06:08.105+0000 7f50e4577700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:41.677763+0000 front 2024-05-03T15:05:41.677677+0000 (oldest deadline 2024-05-03T15:06:05.078219+0000) 2024-05-03T15:06:08.423 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:08 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:08.109+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.098256+0000 front 2024-05-03T15:05:40.098316+0000 (oldest deadline 2024-05-03T15:06:04.098995+0000) 2024-05-03T15:06:08.508 INFO:journalctl@ceph.osd.4.smithi090.stdout:May 03 15:06:08 smithi090 podman[69073]: 2024-05-03 15:06:08.236867738 +0000 UTC m=+5.441555680 container died 212c59b75ed6a0fbc43046d1ed58e1ca679c62e8114cf3c98f4142fad5fbdce7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, io.buildah.version=1.33.5) 2024-05-03T15:06:08.508 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:08 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:08.447+0000 7f487aafd700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:41.153998+0000 front 2024-05-03T15:05:41.154137+0000 (oldest deadline 2024-05-03T15:06:03.454048+0000) 2024-05-03T15:06:08.586 INFO:journalctl@ceph.osd.6.smithi060.stdout:May 03 15:06:08 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-6[54395]: 2024-05-03T15:06:08.454+0000 7f6dc4c00700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:41.710862+0000 front 2024-05-03T15:05:41.710964+0000 (oldest deadline 2024-05-03T15:06:07.512685+0000) 2024-05-03T15:06:08.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:08 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:08.306+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.184568+0000 front 2024-05-03T15:05:32.184943+0000 (oldest deadline 2024-05-03T15:05:55.084762+0000) 2024-05-03T15:06:08.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:08 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:08.306+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.788481+0000 front 2024-05-03T15:05:40.788594+0000 (oldest deadline 2024-05-03T15:06:04.288673+0000) 2024-05-03T15:06:08.846 INFO:journalctl@ceph.osd.4.smithi090.stdout:May 03 15:06:08 smithi090 podman[69073]: 2024-05-03 15:06:08.607298602 +0000 UTC m=+5.811986536 container cleanup 212c59b75ed6a0fbc43046d1ed58e1ca679c62e8114cf3c98f4142fad5fbdce7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-4, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T15:06:08.846 INFO:journalctl@ceph.osd.4.smithi090.stdout:May 03 15:06:08 smithi090 bash[69073]: ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-4 2024-05-03T15:06:09.124 INFO:journalctl@ceph.osd.4.smithi090.stdout:May 03 15:06:08 smithi090 podman[69102]: 2024-05-03 15:06:08.892104084 +0000 UTC m=+0.650036360 container remove 212c59b75ed6a0fbc43046d1ed58e1ca679c62e8114cf3c98f4142fad5fbdce7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T15:06:09.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:08 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:08.845+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.538644+0000 front 2024-05-03T15:05:32.538760+0000 (oldest deadline 2024-05-03T15:05:58.438503+0000) 2024-05-03T15:06:09.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:08 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:08.845+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:42.440172+0000 front 2024-05-03T15:05:42.440196+0000 (oldest deadline 2024-05-03T15:06:05.940332+0000) 2024-05-03T15:06:09.141 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:08 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:08.839+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:06:09.142 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:08 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:08.839+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:37.896657+0000 front 2024-05-03T15:05:37.896645+0000 (oldest deadline 2024-05-03T15:06:03.796605+0000) 2024-05-03T15:06:09.422 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:09 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:09.141+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.098256+0000 front 2024-05-03T15:05:40.098316+0000 (oldest deadline 2024-05-03T15:06:04.098995+0000) 2024-05-03T15:06:09.423 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 03 15:06:09 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-5[45461]: 2024-05-03T15:06:09.146+0000 7f50e4577700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.375047+0000 front 2024-05-03T15:05:31.375384+0000 (oldest deadline 2024-05-03T15:05:54.274777+0000) 2024-05-03T15:06:09.423 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 03 15:06:09 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-5[45461]: 2024-05-03T15:06:09.146+0000 7f50e4577700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:41.677763+0000 front 2024-05-03T15:05:41.677677+0000 (oldest deadline 2024-05-03T15:06:05.078219+0000) 2024-05-03T15:06:09.540 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:09 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:09.397+0000 7f487aafd700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:41.153998+0000 front 2024-05-03T15:05:41.154137+0000 (oldest deadline 2024-05-03T15:06:03.454048+0000) 2024-05-03T15:06:09.540 INFO:journalctl@ceph.osd.4.smithi090.stdout:May 03 15:06:09 smithi090 podman[69195]: 2024-05-03 15:06:09.12018398 +0000 UTC m=+0.017737391 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T15:06:09.540 INFO:journalctl@ceph.osd.4.smithi090.stdout:May 03 15:06:09 smithi090 podman[69195]: 2024-05-03 15:06:09.234930763 +0000 UTC m=+0.132484174 container create 6f597147ad37454b8852dee452146fbb67edb179e83050792c3db13bce32bf87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T15:06:09.540 INFO:journalctl@ceph.osd.4.smithi090.stdout:May 03 15:06:09 smithi090 podman[69195]: 2024-05-03 15:06:09.44820092 +0000 UTC m=+0.345754345 container init 6f597147ad37454b8852dee452146fbb67edb179e83050792c3db13bce32bf87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-4-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, GIT_CLEAN=True) 2024-05-03T15:06:09.540 INFO:journalctl@ceph.osd.4.smithi090.stdout:May 03 15:06:09 smithi090 podman[69195]: 2024-05-03 15:06:09.451438826 +0000 UTC m=+0.348992238 container start 6f597147ad37454b8852dee452146fbb67edb179e83050792c3db13bce32bf87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-05-03T15:06:09.541 INFO:journalctl@ceph.osd.4.smithi090.stdout:May 03 15:06:09 smithi090 podman[69195]: 2024-05-03 15:06:09.539465635 +0000 UTC m=+0.437019074 container attach 6f597147ad37454b8852dee452146fbb67edb179e83050792c3db13bce32bf87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-05-03T15:06:09.586 INFO:journalctl@ceph.osd.6.smithi060.stdout:May 03 15:06:09 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-6[54395]: 2024-05-03T15:06:09.430+0000 7f6dc4c00700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:41.710862+0000 front 2024-05-03T15:05:41.710964+0000 (oldest deadline 2024-05-03T15:06:07.512685+0000) 2024-05-03T15:06:09.586 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:09 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:09.281+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.184568+0000 front 2024-05-03T15:05:32.184943+0000 (oldest deadline 2024-05-03T15:05:55.084762+0000) 2024-05-03T15:06:09.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:09 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:09.281+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.788481+0000 front 2024-05-03T15:05:40.788594+0000 (oldest deadline 2024-05-03T15:06:04.288673+0000) 2024-05-03T15:06:09.807 INFO:journalctl@ceph.osd.4.smithi090.stdout:May 03 15:06:09 smithi090 podman[69195]: 2024-05-03 15:06:09.751097084 +0000 UTC m=+0.648650495 container died 6f597147ad37454b8852dee452146fbb67edb179e83050792c3db13bce32bf87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T15:06:10.069 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:09 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:09.806+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.538644+0000 front 2024-05-03T15:05:32.538760+0000 (oldest deadline 2024-05-03T15:05:58.438503+0000) 2024-05-03T15:06:10.070 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:09 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:09.806+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:42.440172+0000 front 2024-05-03T15:05:42.440196+0000 (oldest deadline 2024-05-03T15:06:05.940332+0000) 2024-05-03T15:06:10.153 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:09 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:09.848+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:06:10.153 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:09 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:09.848+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:37.896657+0000 front 2024-05-03T15:05:37.896645+0000 (oldest deadline 2024-05-03T15:06:03.796605+0000) 2024-05-03T15:06:10.322 INFO:journalctl@ceph.osd.4.smithi090.stdout:May 03 15:06:10 smithi090 podman[69221]: 2024-05-03 15:06:10.319444244 +0000 UTC m=+0.562952382 container remove 6f597147ad37454b8852dee452146fbb67edb179e83050792c3db13bce32bf87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240430) 2024-05-03T15:06:10.340 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-285d64b2-095a-11ef-bc97-c7b262605968@osd.4.service' 2024-05-03T15:06:10.379 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T15:06:10.379 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-03T15:06:10.380 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-03T15:06:10.380 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-285d64b2-095a-11ef-bc97-c7b262605968@osd.5 2024-05-03T15:06:10.406 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 03 15:06:10 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-5[45461]: 2024-05-03T15:06:10.151+0000 7f50e4577700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.375047+0000 front 2024-05-03T15:05:31.375384+0000 (oldest deadline 2024-05-03T15:05:54.274777+0000) 2024-05-03T15:06:10.407 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 03 15:06:10 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-5[45461]: 2024-05-03T15:06:10.151+0000 7f50e4577700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:41.677763+0000 front 2024-05-03T15:05:41.677677+0000 (oldest deadline 2024-05-03T15:06:05.078219+0000) 2024-05-03T15:06:10.407 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:10 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:10.159+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.098256+0000 front 2024-05-03T15:05:40.098316+0000 (oldest deadline 2024-05-03T15:06:04.098995+0000) 2024-05-03T15:06:10.586 INFO:journalctl@ceph.osd.6.smithi060.stdout:May 03 15:06:10 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-6[54395]: 2024-05-03T15:06:10.441+0000 7f6dc4c00700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:41.710862+0000 front 2024-05-03T15:05:41.710964+0000 (oldest deadline 2024-05-03T15:06:07.512685+0000) 2024-05-03T15:06:10.586 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:10 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:10.238+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.184568+0000 front 2024-05-03T15:05:32.184943+0000 (oldest deadline 2024-05-03T15:05:55.084762+0000) 2024-05-03T15:06:10.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:10 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:10.238+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.788481+0000 front 2024-05-03T15:05:40.788594+0000 (oldest deadline 2024-05-03T15:06:04.288673+0000) 2024-05-03T15:06:10.623 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:10 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:10.401+0000 7f487aafd700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:41.153998+0000 front 2024-05-03T15:05:41.154137+0000 (oldest deadline 2024-05-03T15:06:03.454048+0000) 2024-05-03T15:06:10.672 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 03 15:06:10 smithi149 systemd[1]: Stopping Ceph osd.5 for 285d64b2-095a-11ef-bc97-c7b262605968... 2024-05-03T15:06:11.123 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:10 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:10.832+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.538644+0000 front 2024-05-03T15:05:32.538760+0000 (oldest deadline 2024-05-03T15:05:58.438503+0000) 2024-05-03T15:06:11.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:10 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:10.832+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:42.440172+0000 front 2024-05-03T15:05:42.440196+0000 (oldest deadline 2024-05-03T15:06:05.940332+0000) 2024-05-03T15:06:11.138 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:10 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:10.869+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:06:11.138 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:10 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:10.869+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:37.896657+0000 front 2024-05-03T15:05:37.896645+0000 (oldest deadline 2024-05-03T15:06:03.796605+0000) 2024-05-03T15:06:11.139 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 03 15:06:10 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-5[45461]: 2024-05-03T15:06:10.699+0000 7f50ebf8b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T15:06:11.139 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 03 15:06:10 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-5[45461]: 2024-05-03T15:06:10.699+0000 7f50ebf8b700 -1 osd.5 93 *** Got signal Terminated *** 2024-05-03T15:06:11.139 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 03 15:06:10 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-5[45461]: 2024-05-03T15:06:10.699+0000 7f50ebf8b700 -1 osd.5 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-03T15:06:11.422 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:11 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:11.137+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.098256+0000 front 2024-05-03T15:05:40.098316+0000 (oldest deadline 2024-05-03T15:06:04.098995+0000) 2024-05-03T15:06:11.423 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 03 15:06:11 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-5[45461]: 2024-05-03T15:06:11.172+0000 7f50e4577700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.375047+0000 front 2024-05-03T15:05:31.375384+0000 (oldest deadline 2024-05-03T15:05:54.274777+0000) 2024-05-03T15:06:11.423 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 03 15:06:11 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-5[45461]: 2024-05-03T15:06:11.172+0000 7f50e4577700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:41.677763+0000 front 2024-05-03T15:05:41.677677+0000 (oldest deadline 2024-05-03T15:06:05.078219+0000) 2024-05-03T15:06:11.586 INFO:journalctl@ceph.osd.6.smithi060.stdout:May 03 15:06:11 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-6[54395]: 2024-05-03T15:06:11.432+0000 7f6dc4c00700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:41.710862+0000 front 2024-05-03T15:05:41.710964+0000 (oldest deadline 2024-05-03T15:06:07.512685+0000) 2024-05-03T15:06:11.586 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:11 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:11.262+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.184568+0000 front 2024-05-03T15:05:32.184943+0000 (oldest deadline 2024-05-03T15:05:55.084762+0000) 2024-05-03T15:06:11.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:11 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:11.262+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.788481+0000 front 2024-05-03T15:05:40.788594+0000 (oldest deadline 2024-05-03T15:06:04.288673+0000) 2024-05-03T15:06:11.815 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:11 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:11.428+0000 7f487aafd700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:41.153998+0000 front 2024-05-03T15:05:41.154137+0000 (oldest deadline 2024-05-03T15:06:03.454048+0000) 2024-05-03T15:06:12.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:11 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:11.814+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.538644+0000 front 2024-05-03T15:05:32.538760+0000 (oldest deadline 2024-05-03T15:05:58.438503+0000) 2024-05-03T15:06:12.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:11 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:11.814+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:42.440172+0000 front 2024-05-03T15:05:42.440196+0000 (oldest deadline 2024-05-03T15:06:05.940332+0000) 2024-05-03T15:06:12.172 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:12 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:12.090+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.098256+0000 front 2024-05-03T15:05:40.098316+0000 (oldest deadline 2024-05-03T15:06:04.098995+0000) 2024-05-03T15:06:12.172 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:12 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:12.090+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.000252+0000 front 2024-05-03T15:05:50.000274+0000 (oldest deadline 2024-05-03T15:06:11.700413+0000) 2024-05-03T15:06:12.173 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 03 15:06:12 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-5[45461]: 2024-05-03T15:06:12.137+0000 7f50e4577700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.375047+0000 front 2024-05-03T15:05:31.375384+0000 (oldest deadline 2024-05-03T15:05:54.274777+0000) 2024-05-03T15:06:12.173 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 03 15:06:12 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-5[45461]: 2024-05-03T15:06:12.137+0000 7f50e4577700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:41.677763+0000 front 2024-05-03T15:05:41.677677+0000 (oldest deadline 2024-05-03T15:06:05.078219+0000) 2024-05-03T15:06:12.173 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:11 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:11.913+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:06:12.173 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:11 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:11.913+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:37.896657+0000 front 2024-05-03T15:05:37.896645+0000 (oldest deadline 2024-05-03T15:06:03.796605+0000) 2024-05-03T15:06:12.173 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:11 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:11.913+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.098837+0000 front 2024-05-03T15:05:50.098910+0000 (oldest deadline 2024-05-03T15:06:11.198967+0000) 2024-05-03T15:06:12.586 INFO:journalctl@ceph.osd.6.smithi060.stdout:May 03 15:06:12 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-6[54395]: 2024-05-03T15:06:12.436+0000 7f6dc4c00700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:41.710862+0000 front 2024-05-03T15:05:41.710964+0000 (oldest deadline 2024-05-03T15:06:07.512685+0000) 2024-05-03T15:06:12.586 INFO:journalctl@ceph.osd.6.smithi060.stdout:May 03 15:06:12 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-6[54395]: 2024-05-03T15:06:12.436+0000 7f6dc4c00700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:49.214951+0000 front 2024-05-03T15:05:49.215022+0000 (oldest deadline 2024-05-03T15:06:11.515185+0000) 2024-05-03T15:06:12.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:12 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:12.236+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.184568+0000 front 2024-05-03T15:05:32.184943+0000 (oldest deadline 2024-05-03T15:05:55.084762+0000) 2024-05-03T15:06:12.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:12 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:12.236+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.788481+0000 front 2024-05-03T15:05:40.788594+0000 (oldest deadline 2024-05-03T15:06:04.288673+0000) 2024-05-03T15:06:12.844 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:12 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:12.400+0000 7f487aafd700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:41.153998+0000 front 2024-05-03T15:05:41.154137+0000 (oldest deadline 2024-05-03T15:06:03.454048+0000) 2024-05-03T15:06:13.123 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:12 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:12.844+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.538644+0000 front 2024-05-03T15:05:32.538760+0000 (oldest deadline 2024-05-03T15:05:58.438503+0000) 2024-05-03T15:06:13.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:12 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:12.844+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:42.440172+0000 front 2024-05-03T15:05:42.440196+0000 (oldest deadline 2024-05-03T15:06:05.940332+0000) 2024-05-03T15:06:13.422 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:12 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:12.961+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:06:13.422 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:12 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:12.961+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:37.896657+0000 front 2024-05-03T15:05:37.896645+0000 (oldest deadline 2024-05-03T15:06:03.796605+0000) 2024-05-03T15:06:13.423 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:12 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:12.961+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.098837+0000 front 2024-05-03T15:05:50.098910+0000 (oldest deadline 2024-05-03T15:06:11.198967+0000) 2024-05-03T15:06:13.423 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 03 15:06:13 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-5[45461]: 2024-05-03T15:06:13.165+0000 7f50e4577700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.375047+0000 front 2024-05-03T15:05:31.375384+0000 (oldest deadline 2024-05-03T15:05:54.274777+0000) 2024-05-03T15:06:13.423 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 03 15:06:13 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-5[45461]: 2024-05-03T15:06:13.165+0000 7f50e4577700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:41.677763+0000 front 2024-05-03T15:05:41.677677+0000 (oldest deadline 2024-05-03T15:06:05.078219+0000) 2024-05-03T15:06:13.423 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:13 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:13.104+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.098256+0000 front 2024-05-03T15:05:40.098316+0000 (oldest deadline 2024-05-03T15:06:04.098995+0000) 2024-05-03T15:06:13.423 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:13 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:13.104+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.000252+0000 front 2024-05-03T15:05:50.000274+0000 (oldest deadline 2024-05-03T15:06:11.700413+0000) 2024-05-03T15:06:13.476 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:13 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:13.201+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.184568+0000 front 2024-05-03T15:05:32.184943+0000 (oldest deadline 2024-05-03T15:05:55.084762+0000) 2024-05-03T15:06:13.477 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:13 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:13.201+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.788481+0000 front 2024-05-03T15:05:40.788594+0000 (oldest deadline 2024-05-03T15:06:04.288673+0000) 2024-05-03T15:06:13.624 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:13 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:13.361+0000 7f487aafd700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:41.153998+0000 front 2024-05-03T15:05:41.154137+0000 (oldest deadline 2024-05-03T15:06:03.454048+0000) 2024-05-03T15:06:13.836 INFO:journalctl@ceph.osd.6.smithi060.stdout:May 03 15:06:13 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-6[54395]: 2024-05-03T15:06:13.475+0000 7f6dc4c00700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:41.710862+0000 front 2024-05-03T15:05:41.710964+0000 (oldest deadline 2024-05-03T15:06:07.512685+0000) 2024-05-03T15:06:13.836 INFO:journalctl@ceph.osd.6.smithi060.stdout:May 03 15:06:13 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-6[54395]: 2024-05-03T15:06:13.475+0000 7f6dc4c00700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:49.214951+0000 front 2024-05-03T15:05:49.215022+0000 (oldest deadline 2024-05-03T15:06:11.515185+0000) 2024-05-03T15:06:14.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:13 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:13.848+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.538644+0000 front 2024-05-03T15:05:32.538760+0000 (oldest deadline 2024-05-03T15:05:58.438503+0000) 2024-05-03T15:06:14.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:13 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:13.848+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:42.440172+0000 front 2024-05-03T15:05:42.440196+0000 (oldest deadline 2024-05-03T15:06:05.940332+0000) 2024-05-03T15:06:14.211 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:14 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:14.061+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.098256+0000 front 2024-05-03T15:05:40.098316+0000 (oldest deadline 2024-05-03T15:06:04.098995+0000) 2024-05-03T15:06:14.212 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:14 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:14.061+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.000252+0000 front 2024-05-03T15:05:50.000274+0000 (oldest deadline 2024-05-03T15:06:11.700413+0000) 2024-05-03T15:06:14.212 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:13 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:13.952+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:06:14.212 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:13 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:13.952+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:37.896657+0000 front 2024-05-03T15:05:37.896645+0000 (oldest deadline 2024-05-03T15:06:03.796605+0000) 2024-05-03T15:06:14.212 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:13 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:13.952+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.098837+0000 front 2024-05-03T15:05:50.098910+0000 (oldest deadline 2024-05-03T15:06:11.198967+0000) 2024-05-03T15:06:14.522 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:14 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:14.199+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.184568+0000 front 2024-05-03T15:05:32.184943+0000 (oldest deadline 2024-05-03T15:05:55.084762+0000) 2024-05-03T15:06:14.523 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:14 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:14.199+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.788481+0000 front 2024-05-03T15:05:40.788594+0000 (oldest deadline 2024-05-03T15:06:04.288673+0000) 2024-05-03T15:06:14.624 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:14 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:14.347+0000 7f487aafd700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:41.153998+0000 front 2024-05-03T15:05:41.154137+0000 (oldest deadline 2024-05-03T15:06:03.454048+0000) 2024-05-03T15:06:14.624 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:14 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:14.347+0000 7f487aafd700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.455657+0000 front 2024-05-03T15:05:50.455635+0000 (oldest deadline 2024-05-03T15:06:13.955669+0000) 2024-05-03T15:06:14.672 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 03 15:06:14 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-5[45461]: 2024-05-03T15:06:14.209+0000 7f50e4577700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.375047+0000 front 2024-05-03T15:05:31.375384+0000 (oldest deadline 2024-05-03T15:05:54.274777+0000) 2024-05-03T15:06:14.672 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 03 15:06:14 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-5[45461]: 2024-05-03T15:06:14.209+0000 7f50e4577700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:41.677763+0000 front 2024-05-03T15:05:41.677677+0000 (oldest deadline 2024-05-03T15:06:05.078219+0000) 2024-05-03T15:06:14.672 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 03 15:06:14 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-5[45461]: 2024-05-03T15:06:14.209+0000 7f50e4577700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:45.078651+0000 front 2024-05-03T15:05:45.078679+0000 (oldest deadline 2024-05-03T15:06:13.279314+0000) 2024-05-03T15:06:14.836 INFO:journalctl@ceph.osd.6.smithi060.stdout:May 03 15:06:14 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-6[54395]: 2024-05-03T15:06:14.520+0000 7f6dc4c00700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:41.710862+0000 front 2024-05-03T15:05:41.710964+0000 (oldest deadline 2024-05-03T15:06:07.512685+0000) 2024-05-03T15:06:14.836 INFO:journalctl@ceph.osd.6.smithi060.stdout:May 03 15:06:14 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-6[54395]: 2024-05-03T15:06:14.520+0000 7f6dc4c00700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:49.214951+0000 front 2024-05-03T15:05:49.215022+0000 (oldest deadline 2024-05-03T15:06:11.515185+0000) 2024-05-03T15:06:15.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:14 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:14.829+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.538644+0000 front 2024-05-03T15:05:32.538760+0000 (oldest deadline 2024-05-03T15:05:58.438503+0000) 2024-05-03T15:06:15.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:14 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:14.829+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:42.440172+0000 front 2024-05-03T15:05:42.440196+0000 (oldest deadline 2024-05-03T15:06:05.940332+0000) 2024-05-03T15:06:15.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:14 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:14.829+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:49.441284+0000 front 2024-05-03T15:05:49.441413+0000 (oldest deadline 2024-05-03T15:06:14.041739+0000) 2024-05-03T15:06:15.212 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:15 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:15.042+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.098256+0000 front 2024-05-03T15:05:40.098316+0000 (oldest deadline 2024-05-03T15:06:04.098995+0000) 2024-05-03T15:06:15.212 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:15 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:15.042+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.000252+0000 front 2024-05-03T15:05:50.000274+0000 (oldest deadline 2024-05-03T15:06:11.700413+0000) 2024-05-03T15:06:15.213 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:14 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:14.940+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:06:15.213 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:14 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:14.940+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:37.896657+0000 front 2024-05-03T15:05:37.896645+0000 (oldest deadline 2024-05-03T15:06:03.796605+0000) 2024-05-03T15:06:15.213 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:14 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:14.940+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.098837+0000 front 2024-05-03T15:05:50.098910+0000 (oldest deadline 2024-05-03T15:06:11.198967+0000) 2024-05-03T15:06:15.503 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:15 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:15.225+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.184568+0000 front 2024-05-03T15:05:32.184943+0000 (oldest deadline 2024-05-03T15:05:55.084762+0000) 2024-05-03T15:06:15.503 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:15 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:15.225+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.788481+0000 front 2024-05-03T15:05:40.788594+0000 (oldest deadline 2024-05-03T15:06:04.288673+0000) 2024-05-03T15:06:15.503 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:15 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:15.225+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:48.892458+0000 front 2024-05-03T15:05:48.892380+0000 (oldest deadline 2024-05-03T15:06:14.792663+0000) 2024-05-03T15:06:15.672 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 03 15:06:15 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-5[45461]: 2024-05-03T15:06:15.210+0000 7f50e4577700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.375047+0000 front 2024-05-03T15:05:31.375384+0000 (oldest deadline 2024-05-03T15:05:54.274777+0000) 2024-05-03T15:06:15.673 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 03 15:06:15 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-5[45461]: 2024-05-03T15:06:15.210+0000 7f50e4577700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:41.677763+0000 front 2024-05-03T15:05:41.677677+0000 (oldest deadline 2024-05-03T15:06:05.078219+0000) 2024-05-03T15:06:15.673 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 03 15:06:15 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-5[45461]: 2024-05-03T15:06:15.210+0000 7f50e4577700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:45.078651+0000 front 2024-05-03T15:05:45.078679+0000 (oldest deadline 2024-05-03T15:06:13.279314+0000) 2024-05-03T15:06:15.836 INFO:journalctl@ceph.osd.6.smithi060.stdout:May 03 15:06:15 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-6[54395]: 2024-05-03T15:06:15.501+0000 7f6dc4c00700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:41.710862+0000 front 2024-05-03T15:05:41.710964+0000 (oldest deadline 2024-05-03T15:06:07.512685+0000) 2024-05-03T15:06:15.836 INFO:journalctl@ceph.osd.6.smithi060.stdout:May 03 15:06:15 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-6[54395]: 2024-05-03T15:06:15.501+0000 7f6dc4c00700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:49.214951+0000 front 2024-05-03T15:05:49.215022+0000 (oldest deadline 2024-05-03T15:06:11.515185+0000) 2024-05-03T15:06:15.848 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:15 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:15.396+0000 7f487aafd700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:41.153998+0000 front 2024-05-03T15:05:41.154137+0000 (oldest deadline 2024-05-03T15:06:03.454048+0000) 2024-05-03T15:06:15.848 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:15 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:15.396+0000 7f487aafd700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.455657+0000 front 2024-05-03T15:05:50.455635+0000 (oldest deadline 2024-05-03T15:06:13.955669+0000) 2024-05-03T15:06:16.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:15 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:15.847+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.538644+0000 front 2024-05-03T15:05:32.538760+0000 (oldest deadline 2024-05-03T15:05:58.438503+0000) 2024-05-03T15:06:16.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:15 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:15.847+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:42.440172+0000 front 2024-05-03T15:05:42.440196+0000 (oldest deadline 2024-05-03T15:06:05.940332+0000) 2024-05-03T15:06:16.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:15 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:15.847+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:49.441284+0000 front 2024-05-03T15:05:49.441413+0000 (oldest deadline 2024-05-03T15:06:14.041739+0000) 2024-05-03T15:06:16.172 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:16 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:16.059+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.098256+0000 front 2024-05-03T15:05:40.098316+0000 (oldest deadline 2024-05-03T15:06:04.098995+0000) 2024-05-03T15:06:16.172 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:16 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:16.059+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.000252+0000 front 2024-05-03T15:05:50.000274+0000 (oldest deadline 2024-05-03T15:06:11.700413+0000) 2024-05-03T15:06:16.173 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:15 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:15.911+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:06:16.173 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:15 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:15.911+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:37.896657+0000 front 2024-05-03T15:05:37.896645+0000 (oldest deadline 2024-05-03T15:06:03.796605+0000) 2024-05-03T15:06:16.173 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:15 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:15.911+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.098837+0000 front 2024-05-03T15:05:50.098910+0000 (oldest deadline 2024-05-03T15:06:11.198967+0000) 2024-05-03T15:06:16.477 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 03 15:06:16 smithi149 podman[69900]: 2024-05-03 15:06:16.183842538 +0000 UTC m=+5.614176243 container died cfb4e95629f9ffed3d6f4298c22e6334919ffed996339e72c1358066fc677a10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-05-03T15:06:16.513 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:16 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:16.239+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.184568+0000 front 2024-05-03T15:05:32.184943+0000 (oldest deadline 2024-05-03T15:05:55.084762+0000) 2024-05-03T15:06:16.513 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:16 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:16.239+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.788481+0000 front 2024-05-03T15:05:40.788594+0000 (oldest deadline 2024-05-03T15:06:04.288673+0000) 2024-05-03T15:06:16.513 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:16 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:16.239+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:48.892458+0000 front 2024-05-03T15:05:48.892380+0000 (oldest deadline 2024-05-03T15:06:14.792663+0000) 2024-05-03T15:06:16.829 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 03 15:06:16 smithi149 podman[69900]: 2024-05-03 15:06:16.568264454 +0000 UTC m=+5.998598164 container cleanup cfb4e95629f9ffed3d6f4298c22e6334919ffed996339e72c1358066fc677a10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, io.buildah.version=1.33.5) 2024-05-03T15:06:16.829 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 03 15:06:16 smithi149 bash[69900]: ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-5 2024-05-03T15:06:16.836 INFO:journalctl@ceph.osd.6.smithi060.stdout:May 03 15:06:16 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-6[54395]: 2024-05-03T15:06:16.511+0000 7f6dc4c00700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:41.710862+0000 front 2024-05-03T15:05:41.710964+0000 (oldest deadline 2024-05-03T15:06:07.512685+0000) 2024-05-03T15:06:16.836 INFO:journalctl@ceph.osd.6.smithi060.stdout:May 03 15:06:16 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-6[54395]: 2024-05-03T15:06:16.511+0000 7f6dc4c00700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:49.214951+0000 front 2024-05-03T15:05:49.215022+0000 (oldest deadline 2024-05-03T15:06:11.515185+0000) 2024-05-03T15:06:16.873 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:16 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:16.391+0000 7f487aafd700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:41.153998+0000 front 2024-05-03T15:05:41.154137+0000 (oldest deadline 2024-05-03T15:06:03.454048+0000) 2024-05-03T15:06:16.873 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:16 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:16.391+0000 7f487aafd700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.455657+0000 front 2024-05-03T15:05:50.455635+0000 (oldest deadline 2024-05-03T15:06:13.955669+0000) 2024-05-03T15:06:17.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:16 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:16.871+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.538644+0000 front 2024-05-03T15:05:32.538760+0000 (oldest deadline 2024-05-03T15:05:58.438503+0000) 2024-05-03T15:06:17.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:16 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:16.871+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:42.440172+0000 front 2024-05-03T15:05:42.440196+0000 (oldest deadline 2024-05-03T15:06:05.940332+0000) 2024-05-03T15:06:17.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:16 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:16.871+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:49.441284+0000 front 2024-05-03T15:05:49.441413+0000 (oldest deadline 2024-05-03T15:06:14.041739+0000) 2024-05-03T15:06:17.138 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 03 15:06:16 smithi149 podman[69917]: 2024-05-03 15:06:16.828926508 +0000 UTC m=+0.648687421 container remove cfb4e95629f9ffed3d6f4298c22e6334919ffed996339e72c1358066fc677a10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-05-03T15:06:17.139 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:17 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:17.048+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.098256+0000 front 2024-05-03T15:05:40.098316+0000 (oldest deadline 2024-05-03T15:06:04.098995+0000) 2024-05-03T15:06:17.139 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:17 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:17.048+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.000252+0000 front 2024-05-03T15:05:50.000274+0000 (oldest deadline 2024-05-03T15:06:11.700413+0000) 2024-05-03T15:06:17.139 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:16 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:16.886+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:06:17.140 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:16 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:16.886+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:37.896657+0000 front 2024-05-03T15:05:37.896645+0000 (oldest deadline 2024-05-03T15:06:03.796605+0000) 2024-05-03T15:06:17.140 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:16 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:16.886+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.098837+0000 front 2024-05-03T15:05:50.098910+0000 (oldest deadline 2024-05-03T15:06:11.198967+0000) 2024-05-03T15:06:17.422 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 03 15:06:17 smithi149 podman[70009]: 2024-05-03 15:06:17.040990766 +0000 UTC m=+0.018371383 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T15:06:17.422 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 03 15:06:17 smithi149 podman[70009]: 2024-05-03 15:06:17.183803272 +0000 UTC m=+0.161183923 container create b586eba159b2a160344b5e24166059995429eb77a1b1e0d249cb74cb0291ce1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-5-deactivate, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-05-03T15:06:17.586 INFO:journalctl@ceph.osd.6.smithi060.stdout:May 03 15:06:17 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-6[54395]: 2024-05-03T15:06:17.491+0000 7f6dc4c00700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:41.710862+0000 front 2024-05-03T15:05:41.710964+0000 (oldest deadline 2024-05-03T15:06:07.512685+0000) 2024-05-03T15:06:17.587 INFO:journalctl@ceph.osd.6.smithi060.stdout:May 03 15:06:17 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-6[54395]: 2024-05-03T15:06:17.491+0000 7f6dc4c00700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:49.214951+0000 front 2024-05-03T15:05:49.215022+0000 (oldest deadline 2024-05-03T15:06:11.515185+0000) 2024-05-03T15:06:17.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:17 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:17.255+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.184568+0000 front 2024-05-03T15:05:32.184943+0000 (oldest deadline 2024-05-03T15:05:55.084762+0000) 2024-05-03T15:06:17.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:17 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:17.255+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.788481+0000 front 2024-05-03T15:05:40.788594+0000 (oldest deadline 2024-05-03T15:06:04.288673+0000) 2024-05-03T15:06:17.588 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:17 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:17.255+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:48.892458+0000 front 2024-05-03T15:05:48.892380+0000 (oldest deadline 2024-05-03T15:06:14.792663+0000) 2024-05-03T15:06:17.746 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 03 15:06:17 smithi149 podman[70009]: 2024-05-03 15:06:17.442877461 +0000 UTC m=+0.420258077 container init b586eba159b2a160344b5e24166059995429eb77a1b1e0d249cb74cb0291ce1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD) 2024-05-03T15:06:17.746 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 03 15:06:17 smithi149 podman[70009]: 2024-05-03 15:06:17.446482323 +0000 UTC m=+0.423862936 container start b586eba159b2a160344b5e24166059995429eb77a1b1e0d249cb74cb0291ce1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T15:06:17.746 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 03 15:06:17 smithi149 podman[70009]: 2024-05-03 15:06:17.542610765 +0000 UTC m=+0.519991375 container attach b586eba159b2a160344b5e24166059995429eb77a1b1e0d249cb74cb0291ce1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T15:06:17.874 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:17 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:17.427+0000 7f487aafd700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:41.153998+0000 front 2024-05-03T15:05:41.154137+0000 (oldest deadline 2024-05-03T15:06:03.454048+0000) 2024-05-03T15:06:17.874 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:17 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:17.427+0000 7f487aafd700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.455657+0000 front 2024-05-03T15:05:50.455635+0000 (oldest deadline 2024-05-03T15:06:13.955669+0000) 2024-05-03T15:06:18.006 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 03 15:06:17 smithi149 podman[70009]: 2024-05-03 15:06:17.745094425 +0000 UTC m=+0.722475038 container died b586eba159b2a160344b5e24166059995429eb77a1b1e0d249cb74cb0291ce1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-03T15:06:18.007 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:17 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:17.908+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:06:18.007 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:17 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:17.908+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:37.896657+0000 front 2024-05-03T15:05:37.896645+0000 (oldest deadline 2024-05-03T15:06:03.796605+0000) 2024-05-03T15:06:18.007 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:17 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:17.908+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.098837+0000 front 2024-05-03T15:05:50.098910+0000 (oldest deadline 2024-05-03T15:06:11.198967+0000) 2024-05-03T15:06:18.372 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:18 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:18.005+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.098256+0000 front 2024-05-03T15:05:40.098316+0000 (oldest deadline 2024-05-03T15:06:04.098995+0000) 2024-05-03T15:06:18.372 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:18 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:18.005+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.000252+0000 front 2024-05-03T15:05:50.000274+0000 (oldest deadline 2024-05-03T15:06:11.700413+0000) 2024-05-03T15:06:18.373 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:17 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:17.917+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.538644+0000 front 2024-05-03T15:05:32.538760+0000 (oldest deadline 2024-05-03T15:05:58.438503+0000) 2024-05-03T15:06:18.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:17 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:17.917+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:42.440172+0000 front 2024-05-03T15:05:42.440196+0000 (oldest deadline 2024-05-03T15:06:05.940332+0000) 2024-05-03T15:06:18.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:17 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:17.917+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:49.441284+0000 front 2024-05-03T15:05:49.441413+0000 (oldest deadline 2024-05-03T15:06:14.041739+0000) 2024-05-03T15:06:18.399 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-285d64b2-095a-11ef-bc97-c7b262605968@osd.5.service' 2024-05-03T15:06:18.477 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T15:06:18.477 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-03T15:06:18.477 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-05-03T15:06:18.478 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-285d64b2-095a-11ef-bc97-c7b262605968@osd.6 2024-05-03T15:06:18.558 INFO:journalctl@ceph.osd.6.smithi060.stdout:May 03 15:06:18 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-6[54395]: 2024-05-03T15:06:18.487+0000 7f6dc4c00700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:41.710862+0000 front 2024-05-03T15:05:41.710964+0000 (oldest deadline 2024-05-03T15:06:07.512685+0000) 2024-05-03T15:06:18.558 INFO:journalctl@ceph.osd.6.smithi060.stdout:May 03 15:06:18 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-6[54395]: 2024-05-03T15:06:18.487+0000 7f6dc4c00700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:49.214951+0000 front 2024-05-03T15:05:49.215022+0000 (oldest deadline 2024-05-03T15:06:11.515185+0000) 2024-05-03T15:06:18.558 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:18 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:18.293+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.184568+0000 front 2024-05-03T15:05:32.184943+0000 (oldest deadline 2024-05-03T15:05:55.084762+0000) 2024-05-03T15:06:18.558 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:18 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:18.293+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.788481+0000 front 2024-05-03T15:05:40.788594+0000 (oldest deadline 2024-05-03T15:06:04.288673+0000) 2024-05-03T15:06:18.558 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:18 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:18.293+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:48.892458+0000 front 2024-05-03T15:05:48.892380+0000 (oldest deadline 2024-05-03T15:06:14.792663+0000) 2024-05-03T15:06:18.836 INFO:journalctl@ceph.osd.6.smithi060.stdout:May 03 15:06:18 smithi060 systemd[1]: Stopping Ceph osd.6 for 285d64b2-095a-11ef-bc97-c7b262605968... 2024-05-03T15:06:18.873 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:18 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:18.433+0000 7f487aafd700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:41.153998+0000 front 2024-05-03T15:05:41.154137+0000 (oldest deadline 2024-05-03T15:06:03.454048+0000) 2024-05-03T15:06:18.874 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:18 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:18.433+0000 7f487aafd700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.455657+0000 front 2024-05-03T15:05:50.455635+0000 (oldest deadline 2024-05-03T15:06:13.955669+0000) 2024-05-03T15:06:19.173 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:18 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:18.956+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.098256+0000 front 2024-05-03T15:05:40.098316+0000 (oldest deadline 2024-05-03T15:06:04.098995+0000) 2024-05-03T15:06:19.174 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:18 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:18.956+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.000252+0000 front 2024-05-03T15:05:50.000274+0000 (oldest deadline 2024-05-03T15:06:11.700413+0000) 2024-05-03T15:06:19.174 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:18 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:18.868+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:06:19.174 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:18 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:18.868+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:37.896657+0000 front 2024-05-03T15:05:37.896645+0000 (oldest deadline 2024-05-03T15:06:03.796605+0000) 2024-05-03T15:06:19.174 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:18 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:18.868+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.098837+0000 front 2024-05-03T15:05:50.098910+0000 (oldest deadline 2024-05-03T15:06:11.198967+0000) 2024-05-03T15:06:19.336 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:19 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:19.322+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.184568+0000 front 2024-05-03T15:05:32.184943+0000 (oldest deadline 2024-05-03T15:05:55.084762+0000) 2024-05-03T15:06:19.336 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:19 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:19.322+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.788481+0000 front 2024-05-03T15:05:40.788594+0000 (oldest deadline 2024-05-03T15:06:04.288673+0000) 2024-05-03T15:06:19.337 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:19 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:19.322+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:48.892458+0000 front 2024-05-03T15:05:48.892380+0000 (oldest deadline 2024-05-03T15:06:14.792663+0000) 2024-05-03T15:06:19.337 INFO:journalctl@ceph.osd.6.smithi060.stdout:May 03 15:06:19 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-6[54395]: 2024-05-03T15:06:19.084+0000 7f6dce400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T15:06:19.337 INFO:journalctl@ceph.osd.6.smithi060.stdout:May 03 15:06:19 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-6[54395]: 2024-05-03T15:06:19.084+0000 7f6dce400700 -1 osd.6 93 *** Got signal Terminated *** 2024-05-03T15:06:19.337 INFO:journalctl@ceph.osd.6.smithi060.stdout:May 03 15:06:19 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-6[54395]: 2024-05-03T15:06:19.084+0000 7f6dce400700 -1 osd.6 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-03T15:06:19.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:18 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:18.880+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.538644+0000 front 2024-05-03T15:05:32.538760+0000 (oldest deadline 2024-05-03T15:05:58.438503+0000) 2024-05-03T15:06:19.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:18 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:18.880+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:42.440172+0000 front 2024-05-03T15:05:42.440196+0000 (oldest deadline 2024-05-03T15:06:05.940332+0000) 2024-05-03T15:06:19.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:18 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:18.880+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:49.441284+0000 front 2024-05-03T15:05:49.441413+0000 (oldest deadline 2024-05-03T15:06:14.041739+0000) 2024-05-03T15:06:19.836 INFO:journalctl@ceph.osd.6.smithi060.stdout:May 03 15:06:19 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-6[54395]: 2024-05-03T15:06:19.516+0000 7f6dc4c00700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:41.710862+0000 front 2024-05-03T15:05:41.710964+0000 (oldest deadline 2024-05-03T15:06:07.512685+0000) 2024-05-03T15:06:19.836 INFO:journalctl@ceph.osd.6.smithi060.stdout:May 03 15:06:19 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-6[54395]: 2024-05-03T15:06:19.516+0000 7f6dc4c00700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:49.214951+0000 front 2024-05-03T15:05:49.215022+0000 (oldest deadline 2024-05-03T15:06:11.515185+0000) 2024-05-03T15:06:19.856 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:19 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:19.481+0000 7f487aafd700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:41.153998+0000 front 2024-05-03T15:05:41.154137+0000 (oldest deadline 2024-05-03T15:06:03.454048+0000) 2024-05-03T15:06:19.856 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:19 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:19.481+0000 7f487aafd700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.455657+0000 front 2024-05-03T15:05:50.455635+0000 (oldest deadline 2024-05-03T15:06:13.955669+0000) 2024-05-03T15:06:20.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:19 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:19.855+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.538644+0000 front 2024-05-03T15:05:32.538760+0000 (oldest deadline 2024-05-03T15:05:58.438503+0000) 2024-05-03T15:06:20.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:19 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:19.855+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:42.440172+0000 front 2024-05-03T15:05:42.440196+0000 (oldest deadline 2024-05-03T15:06:05.940332+0000) 2024-05-03T15:06:20.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:19 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:19.855+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:49.441284+0000 front 2024-05-03T15:05:49.441413+0000 (oldest deadline 2024-05-03T15:06:14.041739+0000) 2024-05-03T15:06:20.172 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:19 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:19.961+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.098256+0000 front 2024-05-03T15:05:40.098316+0000 (oldest deadline 2024-05-03T15:06:04.098995+0000) 2024-05-03T15:06:20.172 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:19 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:19.961+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.000252+0000 front 2024-05-03T15:05:50.000274+0000 (oldest deadline 2024-05-03T15:06:11.700413+0000) 2024-05-03T15:06:20.173 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:19 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:19.837+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:06:20.173 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:19 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:19.837+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:37.896657+0000 front 2024-05-03T15:05:37.896645+0000 (oldest deadline 2024-05-03T15:06:03.796605+0000) 2024-05-03T15:06:20.173 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:19 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:19.837+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.098837+0000 front 2024-05-03T15:05:50.098910+0000 (oldest deadline 2024-05-03T15:06:11.198967+0000) 2024-05-03T15:06:20.586 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:20 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:20.306+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.184568+0000 front 2024-05-03T15:05:32.184943+0000 (oldest deadline 2024-05-03T15:05:55.084762+0000) 2024-05-03T15:06:20.586 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:20 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:20.306+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.788481+0000 front 2024-05-03T15:05:40.788594+0000 (oldest deadline 2024-05-03T15:06:04.288673+0000) 2024-05-03T15:06:20.586 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:20 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:20.306+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:48.892458+0000 front 2024-05-03T15:05:48.892380+0000 (oldest deadline 2024-05-03T15:06:14.792663+0000) 2024-05-03T15:06:20.587 INFO:journalctl@ceph.osd.6.smithi060.stdout:May 03 15:06:20 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-6[54395]: 2024-05-03T15:06:20.508+0000 7f6dc4c00700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:41.710862+0000 front 2024-05-03T15:05:41.710964+0000 (oldest deadline 2024-05-03T15:06:07.512685+0000) 2024-05-03T15:06:20.587 INFO:journalctl@ceph.osd.6.smithi060.stdout:May 03 15:06:20 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-6[54395]: 2024-05-03T15:06:20.508+0000 7f6dc4c00700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:49.214951+0000 front 2024-05-03T15:05:49.215022+0000 (oldest deadline 2024-05-03T15:06:11.515185+0000) 2024-05-03T15:06:20.865 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:20 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:20.505+0000 7f487aafd700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:41.153998+0000 front 2024-05-03T15:05:41.154137+0000 (oldest deadline 2024-05-03T15:06:03.454048+0000) 2024-05-03T15:06:20.866 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:20 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:20.505+0000 7f487aafd700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.455657+0000 front 2024-05-03T15:05:50.455635+0000 (oldest deadline 2024-05-03T15:06:13.955669+0000) 2024-05-03T15:06:21.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:20 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:20.864+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.538644+0000 front 2024-05-03T15:05:32.538760+0000 (oldest deadline 2024-05-03T15:05:58.438503+0000) 2024-05-03T15:06:21.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:20 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:20.864+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:42.440172+0000 front 2024-05-03T15:05:42.440196+0000 (oldest deadline 2024-05-03T15:06:05.940332+0000) 2024-05-03T15:06:21.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:20 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:20.864+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:49.441284+0000 front 2024-05-03T15:05:49.441413+0000 (oldest deadline 2024-05-03T15:06:14.041739+0000) 2024-05-03T15:06:21.172 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:20 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:20.939+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.098256+0000 front 2024-05-03T15:05:40.098316+0000 (oldest deadline 2024-05-03T15:06:04.098995+0000) 2024-05-03T15:06:21.172 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:20 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:20.939+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.000252+0000 front 2024-05-03T15:05:50.000274+0000 (oldest deadline 2024-05-03T15:06:11.700413+0000) 2024-05-03T15:06:21.172 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:20 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:20.939+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:57.903054+0000 front 2024-05-03T15:05:57.903153+0000 (oldest deadline 2024-05-03T15:06:20.802884+0000) 2024-05-03T15:06:21.173 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:20 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:20.839+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:06:21.173 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:20 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:20.839+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:37.896657+0000 front 2024-05-03T15:05:37.896645+0000 (oldest deadline 2024-05-03T15:06:03.796605+0000) 2024-05-03T15:06:21.173 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:20 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:20.839+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.098837+0000 front 2024-05-03T15:05:50.098910+0000 (oldest deadline 2024-05-03T15:06:11.198967+0000) 2024-05-03T15:06:21.586 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:21 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:21.283+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.184568+0000 front 2024-05-03T15:05:32.184943+0000 (oldest deadline 2024-05-03T15:05:55.084762+0000) 2024-05-03T15:06:21.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:21 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:21.283+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.788481+0000 front 2024-05-03T15:05:40.788594+0000 (oldest deadline 2024-05-03T15:06:04.288673+0000) 2024-05-03T15:06:21.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:21 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:21.283+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:48.892458+0000 front 2024-05-03T15:05:48.892380+0000 (oldest deadline 2024-05-03T15:06:14.792663+0000) 2024-05-03T15:06:21.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:21 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:21.283+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:58.894896+0000 front 2024-05-03T15:05:58.894950+0000 (oldest deadline 2024-05-03T15:06:21.194384+0000) 2024-05-03T15:06:21.587 INFO:journalctl@ceph.osd.6.smithi060.stdout:May 03 15:06:21 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-6[54395]: 2024-05-03T15:06:21.502+0000 7f6dc4c00700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:41.710862+0000 front 2024-05-03T15:05:41.710964+0000 (oldest deadline 2024-05-03T15:06:07.512685+0000) 2024-05-03T15:06:21.587 INFO:journalctl@ceph.osd.6.smithi060.stdout:May 03 15:06:21 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-6[54395]: 2024-05-03T15:06:21.502+0000 7f6dc4c00700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:49.214951+0000 front 2024-05-03T15:05:49.215022+0000 (oldest deadline 2024-05-03T15:06:11.515185+0000) 2024-05-03T15:06:21.587 INFO:journalctl@ceph.osd.6.smithi060.stdout:May 03 15:06:21 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-6[54395]: 2024-05-03T15:06:21.502+0000 7f6dc4c00700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:57.919011+0000 front 2024-05-03T15:05:57.919567+0000 (oldest deadline 2024-05-03T15:06:21.419337+0000) 2024-05-03T15:06:21.847 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:21 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:21.549+0000 7f487aafd700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:41.153998+0000 front 2024-05-03T15:05:41.154137+0000 (oldest deadline 2024-05-03T15:06:03.454048+0000) 2024-05-03T15:06:21.847 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:21 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:21.549+0000 7f487aafd700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.455657+0000 front 2024-05-03T15:05:50.455635+0000 (oldest deadline 2024-05-03T15:06:13.955669+0000) 2024-05-03T15:06:22.088 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:21 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:21.913+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.098256+0000 front 2024-05-03T15:05:40.098316+0000 (oldest deadline 2024-05-03T15:06:04.098995+0000) 2024-05-03T15:06:22.088 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:21 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:21.913+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.000252+0000 front 2024-05-03T15:05:50.000274+0000 (oldest deadline 2024-05-03T15:06:11.700413+0000) 2024-05-03T15:06:22.088 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:21 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:21.913+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:57.903054+0000 front 2024-05-03T15:05:57.903153+0000 (oldest deadline 2024-05-03T15:06:20.802884+0000) 2024-05-03T15:06:22.088 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:21 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:21.831+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:06:22.089 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:21 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:21.831+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:37.896657+0000 front 2024-05-03T15:05:37.896645+0000 (oldest deadline 2024-05-03T15:06:03.796605+0000) 2024-05-03T15:06:22.089 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:21 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:21.831+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.098837+0000 front 2024-05-03T15:05:50.098910+0000 (oldest deadline 2024-05-03T15:06:11.198967+0000) 2024-05-03T15:06:22.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:21 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:21.845+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.538644+0000 front 2024-05-03T15:05:32.538760+0000 (oldest deadline 2024-05-03T15:05:58.438503+0000) 2024-05-03T15:06:22.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:21 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:21.845+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:42.440172+0000 front 2024-05-03T15:05:42.440196+0000 (oldest deadline 2024-05-03T15:06:05.940332+0000) 2024-05-03T15:06:22.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:21 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:21.845+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:49.441284+0000 front 2024-05-03T15:05:49.441413+0000 (oldest deadline 2024-05-03T15:06:14.041739+0000) 2024-05-03T15:06:22.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:21 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:21.845+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:55.743016+0000 front 2024-05-03T15:05:55.743076+0000 (oldest deadline 2024-05-03T15:06:21.642629+0000) 2024-05-03T15:06:22.586 INFO:journalctl@ceph.osd.6.smithi060.stdout:May 03 15:06:22 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-6[54395]: 2024-05-03T15:06:22.487+0000 7f6dc4c00700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:41.710862+0000 front 2024-05-03T15:05:41.710964+0000 (oldest deadline 2024-05-03T15:06:07.512685+0000) 2024-05-03T15:06:22.587 INFO:journalctl@ceph.osd.6.smithi060.stdout:May 03 15:06:22 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-6[54395]: 2024-05-03T15:06:22.487+0000 7f6dc4c00700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:49.214951+0000 front 2024-05-03T15:05:49.215022+0000 (oldest deadline 2024-05-03T15:06:11.515185+0000) 2024-05-03T15:06:22.587 INFO:journalctl@ceph.osd.6.smithi060.stdout:May 03 15:06:22 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-6[54395]: 2024-05-03T15:06:22.487+0000 7f6dc4c00700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:57.919011+0000 front 2024-05-03T15:05:57.919567+0000 (oldest deadline 2024-05-03T15:06:21.419337+0000) 2024-05-03T15:06:22.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:22 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:22.329+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.184568+0000 front 2024-05-03T15:05:32.184943+0000 (oldest deadline 2024-05-03T15:05:55.084762+0000) 2024-05-03T15:06:22.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:22 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:22.329+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.788481+0000 front 2024-05-03T15:05:40.788594+0000 (oldest deadline 2024-05-03T15:06:04.288673+0000) 2024-05-03T15:06:22.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:22 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:22.329+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:48.892458+0000 front 2024-05-03T15:05:48.892380+0000 (oldest deadline 2024-05-03T15:06:14.792663+0000) 2024-05-03T15:06:22.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:22 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:22.329+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:58.894896+0000 front 2024-05-03T15:05:58.894950+0000 (oldest deadline 2024-05-03T15:06:21.194384+0000) 2024-05-03T15:06:22.873 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:22 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:22.534+0000 7f487aafd700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:41.153998+0000 front 2024-05-03T15:05:41.154137+0000 (oldest deadline 2024-05-03T15:06:03.454048+0000) 2024-05-03T15:06:22.873 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:22 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:22.534+0000 7f487aafd700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.455657+0000 front 2024-05-03T15:05:50.455635+0000 (oldest deadline 2024-05-03T15:06:13.955669+0000) 2024-05-03T15:06:22.873 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:22 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:22.534+0000 7f487aafd700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:56.857259+0000 front 2024-05-03T15:05:56.857109+0000 (oldest deadline 2024-05-03T15:06:22.156806+0000) 2024-05-03T15:06:23.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:22 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:22.872+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.538644+0000 front 2024-05-03T15:05:32.538760+0000 (oldest deadline 2024-05-03T15:05:58.438503+0000) 2024-05-03T15:06:23.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:22 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:22.872+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:42.440172+0000 front 2024-05-03T15:05:42.440196+0000 (oldest deadline 2024-05-03T15:06:05.940332+0000) 2024-05-03T15:06:23.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:22 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:22.872+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:49.441284+0000 front 2024-05-03T15:05:49.441413+0000 (oldest deadline 2024-05-03T15:06:14.041739+0000) 2024-05-03T15:06:23.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:22 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:22.872+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:55.743016+0000 front 2024-05-03T15:05:55.743076+0000 (oldest deadline 2024-05-03T15:06:21.642629+0000) 2024-05-03T15:06:23.172 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:22 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:22.951+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.098256+0000 front 2024-05-03T15:05:40.098316+0000 (oldest deadline 2024-05-03T15:06:04.098995+0000) 2024-05-03T15:06:23.213 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:22 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:22.951+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.000252+0000 front 2024-05-03T15:05:50.000274+0000 (oldest deadline 2024-05-03T15:06:11.700413+0000) 2024-05-03T15:06:23.213 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:22 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:22.951+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:57.903054+0000 front 2024-05-03T15:05:57.903153+0000 (oldest deadline 2024-05-03T15:06:20.802884+0000) 2024-05-03T15:06:23.213 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:22 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:22.787+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:06:23.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:22 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:22.787+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:37.896657+0000 front 2024-05-03T15:05:37.896645+0000 (oldest deadline 2024-05-03T15:06:03.796605+0000) 2024-05-03T15:06:23.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:22 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:22.787+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.098837+0000 front 2024-05-03T15:05:50.098910+0000 (oldest deadline 2024-05-03T15:06:11.198967+0000) 2024-05-03T15:06:23.586 INFO:journalctl@ceph.osd.6.smithi060.stdout:May 03 15:06:23 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-6[54395]: 2024-05-03T15:06:23.529+0000 7f6dc4c00700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:41.710862+0000 front 2024-05-03T15:05:41.710964+0000 (oldest deadline 2024-05-03T15:06:07.512685+0000) 2024-05-03T15:06:23.586 INFO:journalctl@ceph.osd.6.smithi060.stdout:May 03 15:06:23 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-6[54395]: 2024-05-03T15:06:23.529+0000 7f6dc4c00700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:49.214951+0000 front 2024-05-03T15:05:49.215022+0000 (oldest deadline 2024-05-03T15:06:11.515185+0000) 2024-05-03T15:06:23.587 INFO:journalctl@ceph.osd.6.smithi060.stdout:May 03 15:06:23 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-6[54395]: 2024-05-03T15:06:23.529+0000 7f6dc4c00700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:57.919011+0000 front 2024-05-03T15:05:57.919567+0000 (oldest deadline 2024-05-03T15:06:21.419337+0000) 2024-05-03T15:06:23.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:23 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:23.305+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.184568+0000 front 2024-05-03T15:05:32.184943+0000 (oldest deadline 2024-05-03T15:05:55.084762+0000) 2024-05-03T15:06:23.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:23 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:23.305+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.788481+0000 front 2024-05-03T15:05:40.788594+0000 (oldest deadline 2024-05-03T15:06:04.288673+0000) 2024-05-03T15:06:23.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:23 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:23.305+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:48.892458+0000 front 2024-05-03T15:05:48.892380+0000 (oldest deadline 2024-05-03T15:06:14.792663+0000) 2024-05-03T15:06:23.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:23 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:23.305+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:58.894896+0000 front 2024-05-03T15:05:58.894950+0000 (oldest deadline 2024-05-03T15:06:21.194384+0000) 2024-05-03T15:06:23.854 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:23 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:23.564+0000 7f487aafd700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:41.153998+0000 front 2024-05-03T15:05:41.154137+0000 (oldest deadline 2024-05-03T15:06:03.454048+0000) 2024-05-03T15:06:23.854 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:23 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:23.564+0000 7f487aafd700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.455657+0000 front 2024-05-03T15:05:50.455635+0000 (oldest deadline 2024-05-03T15:06:13.955669+0000) 2024-05-03T15:06:23.854 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:23 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:23.564+0000 7f487aafd700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:56.857259+0000 front 2024-05-03T15:05:56.857109+0000 (oldest deadline 2024-05-03T15:06:22.156806+0000) 2024-05-03T15:06:24.123 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:23 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:23.852+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.538644+0000 front 2024-05-03T15:05:32.538760+0000 (oldest deadline 2024-05-03T15:05:58.438503+0000) 2024-05-03T15:06:24.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:23 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:23.852+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:42.440172+0000 front 2024-05-03T15:05:42.440196+0000 (oldest deadline 2024-05-03T15:06:05.940332+0000) 2024-05-03T15:06:24.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:23 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:23.852+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:49.441284+0000 front 2024-05-03T15:05:49.441413+0000 (oldest deadline 2024-05-03T15:06:14.041739+0000) 2024-05-03T15:06:24.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:23 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:23.852+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:55.743016+0000 front 2024-05-03T15:05:55.743076+0000 (oldest deadline 2024-05-03T15:06:21.642629+0000) 2024-05-03T15:06:24.172 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:23 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:23.751+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:06:24.172 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:23 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:23.751+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:37.896657+0000 front 2024-05-03T15:05:37.896645+0000 (oldest deadline 2024-05-03T15:06:03.796605+0000) 2024-05-03T15:06:24.172 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:23 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:23.751+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.098837+0000 front 2024-05-03T15:05:50.098910+0000 (oldest deadline 2024-05-03T15:06:11.198967+0000) 2024-05-03T15:06:24.173 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:23 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:23.984+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.098256+0000 front 2024-05-03T15:05:40.098316+0000 (oldest deadline 2024-05-03T15:06:04.098995+0000) 2024-05-03T15:06:24.173 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:23 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:23.984+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.000252+0000 front 2024-05-03T15:05:50.000274+0000 (oldest deadline 2024-05-03T15:06:11.700413+0000) 2024-05-03T15:06:24.173 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:23 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:23.984+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:57.903054+0000 front 2024-05-03T15:05:57.903153+0000 (oldest deadline 2024-05-03T15:06:20.802884+0000) 2024-05-03T15:06:24.586 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:24 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:24.268+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.184568+0000 front 2024-05-03T15:05:32.184943+0000 (oldest deadline 2024-05-03T15:05:55.084762+0000) 2024-05-03T15:06:24.586 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:24 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:24.269+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.788481+0000 front 2024-05-03T15:05:40.788594+0000 (oldest deadline 2024-05-03T15:06:04.288673+0000) 2024-05-03T15:06:24.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:24 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:24.269+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:48.892458+0000 front 2024-05-03T15:05:48.892380+0000 (oldest deadline 2024-05-03T15:06:14.792663+0000) 2024-05-03T15:06:24.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:24 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:24.269+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:58.894896+0000 front 2024-05-03T15:05:58.894950+0000 (oldest deadline 2024-05-03T15:06:21.194384+0000) 2024-05-03T15:06:24.874 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:24 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:24.579+0000 7f487aafd700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:41.153998+0000 front 2024-05-03T15:05:41.154137+0000 (oldest deadline 2024-05-03T15:06:03.454048+0000) 2024-05-03T15:06:24.874 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:24 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:24.579+0000 7f487aafd700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.455657+0000 front 2024-05-03T15:05:50.455635+0000 (oldest deadline 2024-05-03T15:06:13.955669+0000) 2024-05-03T15:06:24.874 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:24 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:24.579+0000 7f487aafd700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:56.857259+0000 front 2024-05-03T15:05:56.857109+0000 (oldest deadline 2024-05-03T15:06:22.156806+0000) 2024-05-03T15:06:24.874 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:24 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:24.824+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.538644+0000 front 2024-05-03T15:05:32.538760+0000 (oldest deadline 2024-05-03T15:05:58.438503+0000) 2024-05-03T15:06:24.875 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:24 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:24.824+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:42.440172+0000 front 2024-05-03T15:05:42.440196+0000 (oldest deadline 2024-05-03T15:06:05.940332+0000) 2024-05-03T15:06:24.875 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:24 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:24.824+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:49.441284+0000 front 2024-05-03T15:05:49.441413+0000 (oldest deadline 2024-05-03T15:06:14.041739+0000) 2024-05-03T15:06:24.875 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:24 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:24.824+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:55.743016+0000 front 2024-05-03T15:05:55.743076+0000 (oldest deadline 2024-05-03T15:06:21.642629+0000) 2024-05-03T15:06:24.899 INFO:journalctl@ceph.osd.6.smithi060.stdout:May 03 15:06:24 smithi060 podman[90814]: 2024-05-03 15:06:24.629234637 +0000 UTC m=+5.693307170 container died 61c3db2ab9f33e0564c20c8f163f40f2b6a6a2ac311c62e4781b206867692c73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-03T15:06:25.035 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:24 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:24.755+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:06:25.035 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:24 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:24.755+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:37.896657+0000 front 2024-05-03T15:05:37.896645+0000 (oldest deadline 2024-05-03T15:06:03.796605+0000) 2024-05-03T15:06:25.036 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:24 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:24.755+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.098837+0000 front 2024-05-03T15:05:50.098910+0000 (oldest deadline 2024-05-03T15:06:11.198967+0000) 2024-05-03T15:06:25.263 INFO:journalctl@ceph.osd.6.smithi060.stdout:May 03 15:06:25 smithi060 podman[90814]: 2024-05-03 15:06:25.010463395 +0000 UTC m=+6.074535933 container cleanup 61c3db2ab9f33e0564c20c8f163f40f2b6a6a2ac311c62e4781b206867692c73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-6, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T15:06:25.264 INFO:journalctl@ceph.osd.6.smithi060.stdout:May 03 15:06:25 smithi060 bash[90814]: ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-6 2024-05-03T15:06:25.422 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:25 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:25.034+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.098256+0000 front 2024-05-03T15:05:40.098316+0000 (oldest deadline 2024-05-03T15:06:04.098995+0000) 2024-05-03T15:06:25.422 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:25 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:25.034+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.000252+0000 front 2024-05-03T15:05:50.000274+0000 (oldest deadline 2024-05-03T15:06:11.700413+0000) 2024-05-03T15:06:25.422 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:25 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:25.034+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:57.903054+0000 front 2024-05-03T15:05:57.903153+0000 (oldest deadline 2024-05-03T15:06:20.802884+0000) 2024-05-03T15:06:25.586 INFO:journalctl@ceph.osd.6.smithi060.stdout:May 03 15:06:25 smithi060 podman[90842]: 2024-05-03 15:06:25.262058574 +0000 UTC m=+0.618665178 container remove 61c3db2ab9f33e0564c20c8f163f40f2b6a6a2ac311c62e4781b206867692c73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-6, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-03T15:06:25.586 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:25 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:25.315+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.184568+0000 front 2024-05-03T15:05:32.184943+0000 (oldest deadline 2024-05-03T15:05:55.084762+0000) 2024-05-03T15:06:25.586 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:25 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:25.315+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.788481+0000 front 2024-05-03T15:05:40.788594+0000 (oldest deadline 2024-05-03T15:06:04.288673+0000) 2024-05-03T15:06:25.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:25 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:25.315+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:48.892458+0000 front 2024-05-03T15:05:48.892380+0000 (oldest deadline 2024-05-03T15:06:14.792663+0000) 2024-05-03T15:06:25.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:25 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:25.315+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:58.894896+0000 front 2024-05-03T15:05:58.894950+0000 (oldest deadline 2024-05-03T15:06:21.194384+0000) 2024-05-03T15:06:25.874 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:25 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:25.617+0000 7f487aafd700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:41.153998+0000 front 2024-05-03T15:05:41.154137+0000 (oldest deadline 2024-05-03T15:06:03.454048+0000) 2024-05-03T15:06:25.874 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:25 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:25.617+0000 7f487aafd700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.455657+0000 front 2024-05-03T15:05:50.455635+0000 (oldest deadline 2024-05-03T15:06:13.955669+0000) 2024-05-03T15:06:25.875 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:25 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:25.617+0000 7f487aafd700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:56.857259+0000 front 2024-05-03T15:05:56.857109+0000 (oldest deadline 2024-05-03T15:06:22.156806+0000) 2024-05-03T15:06:25.875 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:25 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:25.831+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.538644+0000 front 2024-05-03T15:05:32.538760+0000 (oldest deadline 2024-05-03T15:05:58.438503+0000) 2024-05-03T15:06:25.875 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:25 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:25.831+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:42.440172+0000 front 2024-05-03T15:05:42.440196+0000 (oldest deadline 2024-05-03T15:06:05.940332+0000) 2024-05-03T15:06:25.875 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:25 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:25.831+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:49.441284+0000 front 2024-05-03T15:05:49.441413+0000 (oldest deadline 2024-05-03T15:06:14.041739+0000) 2024-05-03T15:06:25.876 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:25 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:25.831+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:55.743016+0000 front 2024-05-03T15:05:55.743076+0000 (oldest deadline 2024-05-03T15:06:21.642629+0000) 2024-05-03T15:06:26.061 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:25 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:25.745+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:06:26.061 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:25 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:25.745+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:37.896657+0000 front 2024-05-03T15:05:37.896645+0000 (oldest deadline 2024-05-03T15:06:03.796605+0000) 2024-05-03T15:06:26.061 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:25 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:25.745+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.098837+0000 front 2024-05-03T15:05:50.098910+0000 (oldest deadline 2024-05-03T15:06:11.198967+0000) 2024-05-03T15:06:26.422 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:26 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:26.060+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.098256+0000 front 2024-05-03T15:05:40.098316+0000 (oldest deadline 2024-05-03T15:06:04.098995+0000) 2024-05-03T15:06:26.423 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:26 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:26.060+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.000252+0000 front 2024-05-03T15:05:50.000274+0000 (oldest deadline 2024-05-03T15:06:11.700413+0000) 2024-05-03T15:06:26.423 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:26 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:26.060+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:57.903054+0000 front 2024-05-03T15:05:57.903153+0000 (oldest deadline 2024-05-03T15:06:20.802884+0000) 2024-05-03T15:06:26.426 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:26 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:26.294+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.184568+0000 front 2024-05-03T15:05:32.184943+0000 (oldest deadline 2024-05-03T15:05:55.084762+0000) 2024-05-03T15:06:26.426 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:26 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:26.294+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.788481+0000 front 2024-05-03T15:05:40.788594+0000 (oldest deadline 2024-05-03T15:06:04.288673+0000) 2024-05-03T15:06:26.426 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:26 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:26.294+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:48.892458+0000 front 2024-05-03T15:05:48.892380+0000 (oldest deadline 2024-05-03T15:06:14.792663+0000) 2024-05-03T15:06:26.427 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:26 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:26.294+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:58.894896+0000 front 2024-05-03T15:05:58.894950+0000 (oldest deadline 2024-05-03T15:06:21.194384+0000) 2024-05-03T15:06:26.427 INFO:journalctl@ceph.osd.6.smithi060.stdout:May 03 15:06:26 smithi060 podman[90930]: 2024-05-03 15:06:26.025096276 +0000 UTC m=+0.039059610 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T15:06:26.427 INFO:journalctl@ceph.osd.6.smithi060.stdout:May 03 15:06:26 smithi060 podman[90930]: 2024-05-03 15:06:26.139207334 +0000 UTC m=+0.153170683 container create dcd951415da5783490e67bb58a3ff566ddbe0f19e3351364dc830e6febc28f0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T15:06:26.428 INFO:journalctl@ceph.osd.6.smithi060.stdout:May 03 15:06:26 smithi060 podman[90930]: 2024-05-03 15:06:26.424748204 +0000 UTC m=+0.438711576 container init dcd951415da5783490e67bb58a3ff566ddbe0f19e3351364dc830e6febc28f0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True) 2024-05-03T15:06:26.836 INFO:journalctl@ceph.osd.6.smithi060.stdout:May 03 15:06:26 smithi060 podman[90930]: 2024-05-03 15:06:26.437389495 +0000 UTC m=+0.451352817 container start dcd951415da5783490e67bb58a3ff566ddbe0f19e3351364dc830e6febc28f0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-6-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-05-03T15:06:26.836 INFO:journalctl@ceph.osd.6.smithi060.stdout:May 03 15:06:26 smithi060 podman[90930]: 2024-05-03 15:06:26.540339883 +0000 UTC m=+0.554303275 container attach dcd951415da5783490e67bb58a3ff566ddbe0f19e3351364dc830e6febc28f0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-03T15:06:27.074 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:26 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:26.713+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:06:27.075 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:26 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:26.713+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:37.896657+0000 front 2024-05-03T15:05:37.896645+0000 (oldest deadline 2024-05-03T15:06:03.796605+0000) 2024-05-03T15:06:27.075 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:26 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:26.713+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.098837+0000 front 2024-05-03T15:05:50.098910+0000 (oldest deadline 2024-05-03T15:06:11.198967+0000) 2024-05-03T15:06:27.124 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:26 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:26.650+0000 7f487aafd700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:41.153998+0000 front 2024-05-03T15:05:41.154137+0000 (oldest deadline 2024-05-03T15:06:03.454048+0000) 2024-05-03T15:06:27.124 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:26 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:26.650+0000 7f487aafd700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.455657+0000 front 2024-05-03T15:05:50.455635+0000 (oldest deadline 2024-05-03T15:06:13.955669+0000) 2024-05-03T15:06:27.124 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:26 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:26.650+0000 7f487aafd700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:56.857259+0000 front 2024-05-03T15:05:56.857109+0000 (oldest deadline 2024-05-03T15:06:22.156806+0000) 2024-05-03T15:06:27.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:26 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:26.804+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.538644+0000 front 2024-05-03T15:05:32.538760+0000 (oldest deadline 2024-05-03T15:05:58.438503+0000) 2024-05-03T15:06:27.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:26 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:26.804+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:42.440172+0000 front 2024-05-03T15:05:42.440196+0000 (oldest deadline 2024-05-03T15:06:05.940332+0000) 2024-05-03T15:06:27.125 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:26 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:26.804+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:49.441284+0000 front 2024-05-03T15:05:49.441413+0000 (oldest deadline 2024-05-03T15:06:14.041739+0000) 2024-05-03T15:06:27.125 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:26 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:26.804+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:55.743016+0000 front 2024-05-03T15:05:55.743076+0000 (oldest deadline 2024-05-03T15:06:21.642629+0000) 2024-05-03T15:06:27.234 INFO:journalctl@ceph.osd.6.smithi060.stdout:May 03 15:06:26 smithi060 podman[90930]: 2024-05-03 15:06:26.910785904 +0000 UTC m=+0.924749216 container died dcd951415da5783490e67bb58a3ff566ddbe0f19e3351364dc830e6febc28f0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-05-03T15:06:27.422 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:27 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:27.073+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.098256+0000 front 2024-05-03T15:05:40.098316+0000 (oldest deadline 2024-05-03T15:06:04.098995+0000) 2024-05-03T15:06:27.422 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:27 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:27.073+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.000252+0000 front 2024-05-03T15:05:50.000274+0000 (oldest deadline 2024-05-03T15:06:11.700413+0000) 2024-05-03T15:06:27.422 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:27 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:27.073+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:57.903054+0000 front 2024-05-03T15:05:57.903153+0000 (oldest deadline 2024-05-03T15:06:20.802884+0000) 2024-05-03T15:06:27.579 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:27 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:27.330+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.184568+0000 front 2024-05-03T15:05:32.184943+0000 (oldest deadline 2024-05-03T15:05:55.084762+0000) 2024-05-03T15:06:27.579 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:27 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:27.330+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.788481+0000 front 2024-05-03T15:05:40.788594+0000 (oldest deadline 2024-05-03T15:06:04.288673+0000) 2024-05-03T15:06:27.579 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:27 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:27.330+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:48.892458+0000 front 2024-05-03T15:05:48.892380+0000 (oldest deadline 2024-05-03T15:06:14.792663+0000) 2024-05-03T15:06:27.580 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:27 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:27.330+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:58.894896+0000 front 2024-05-03T15:05:58.894950+0000 (oldest deadline 2024-05-03T15:06:21.194384+0000) 2024-05-03T15:06:27.625 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-285d64b2-095a-11ef-bc97-c7b262605968@osd.6.service' 2024-05-03T15:06:27.745 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T15:06:27.745 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-05-03T15:06:27.745 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-05-03T15:06:27.746 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-285d64b2-095a-11ef-bc97-c7b262605968@osd.7 2024-05-03T15:06:28.034 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:27 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:27.630+0000 7f487aafd700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:41.153998+0000 front 2024-05-03T15:05:41.154137+0000 (oldest deadline 2024-05-03T15:06:03.454048+0000) 2024-05-03T15:06:28.034 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:27 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:27.630+0000 7f487aafd700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.455657+0000 front 2024-05-03T15:05:50.455635+0000 (oldest deadline 2024-05-03T15:06:13.955669+0000) 2024-05-03T15:06:28.034 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:27 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:27.630+0000 7f487aafd700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:56.857259+0000 front 2024-05-03T15:05:56.857109+0000 (oldest deadline 2024-05-03T15:06:22.156806+0000) 2024-05-03T15:06:28.034 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:27 smithi090 systemd[1]: Stopping Ceph osd.7 for 285d64b2-095a-11ef-bc97-c7b262605968... 2024-05-03T15:06:28.035 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:27 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:27.852+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.538644+0000 front 2024-05-03T15:05:32.538760+0000 (oldest deadline 2024-05-03T15:05:58.438503+0000) 2024-05-03T15:06:28.035 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:27 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:27.852+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:42.440172+0000 front 2024-05-03T15:05:42.440196+0000 (oldest deadline 2024-05-03T15:06:05.940332+0000) 2024-05-03T15:06:28.035 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:27 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:27.852+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:49.441284+0000 front 2024-05-03T15:05:49.441413+0000 (oldest deadline 2024-05-03T15:06:14.041739+0000) 2024-05-03T15:06:28.035 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:27 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:27.852+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:55.743016+0000 front 2024-05-03T15:05:55.743076+0000 (oldest deadline 2024-05-03T15:06:21.642629+0000) 2024-05-03T15:06:28.045 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:27 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:27.699+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:06:28.046 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:27 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:27.699+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:37.896657+0000 front 2024-05-03T15:05:37.896645+0000 (oldest deadline 2024-05-03T15:06:03.796605+0000) 2024-05-03T15:06:28.046 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:27 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:27.699+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.098837+0000 front 2024-05-03T15:05:50.098910+0000 (oldest deadline 2024-05-03T15:06:11.198967+0000) 2024-05-03T15:06:28.374 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:28 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:28.032+0000 7f487e509700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T15:06:28.374 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:28 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:28.032+0000 7f487e509700 -1 osd.7 93 *** Got signal Terminated *** 2024-05-03T15:06:28.374 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:28 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:28.032+0000 7f487e509700 -1 osd.7 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-03T15:06:28.422 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:28 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:28.044+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.098256+0000 front 2024-05-03T15:05:40.098316+0000 (oldest deadline 2024-05-03T15:06:04.098995+0000) 2024-05-03T15:06:28.422 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:28 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:28.044+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.000252+0000 front 2024-05-03T15:05:50.000274+0000 (oldest deadline 2024-05-03T15:06:11.700413+0000) 2024-05-03T15:06:28.423 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:28 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:28.044+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:57.903054+0000 front 2024-05-03T15:05:57.903153+0000 (oldest deadline 2024-05-03T15:06:20.802884+0000) 2024-05-03T15:06:28.586 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:28 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:28.283+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.184568+0000 front 2024-05-03T15:05:32.184943+0000 (oldest deadline 2024-05-03T15:05:55.084762+0000) 2024-05-03T15:06:28.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:28 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:28.283+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.788481+0000 front 2024-05-03T15:05:40.788594+0000 (oldest deadline 2024-05-03T15:06:04.288673+0000) 2024-05-03T15:06:28.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:28 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:28.283+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:48.892458+0000 front 2024-05-03T15:05:48.892380+0000 (oldest deadline 2024-05-03T15:06:14.792663+0000) 2024-05-03T15:06:28.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:28 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:28.283+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:58.894896+0000 front 2024-05-03T15:05:58.894950+0000 (oldest deadline 2024-05-03T15:06:21.194384+0000) 2024-05-03T15:06:28.998 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:28 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:28.675+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:06:28.998 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:28 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:28.675+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:37.896657+0000 front 2024-05-03T15:05:37.896645+0000 (oldest deadline 2024-05-03T15:06:03.796605+0000) 2024-05-03T15:06:28.998 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:28 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:28.675+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.098837+0000 front 2024-05-03T15:05:50.098910+0000 (oldest deadline 2024-05-03T15:06:11.198967+0000) 2024-05-03T15:06:29.124 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:28 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:28.648+0000 7f487aafd700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:41.153998+0000 front 2024-05-03T15:05:41.154137+0000 (oldest deadline 2024-05-03T15:06:03.454048+0000) 2024-05-03T15:06:29.124 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:28 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:28.648+0000 7f487aafd700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.455657+0000 front 2024-05-03T15:05:50.455635+0000 (oldest deadline 2024-05-03T15:06:13.955669+0000) 2024-05-03T15:06:29.124 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:28 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:28.648+0000 7f487aafd700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:56.857259+0000 front 2024-05-03T15:05:56.857109+0000 (oldest deadline 2024-05-03T15:06:22.156806+0000) 2024-05-03T15:06:29.125 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:28 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:28.808+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.538644+0000 front 2024-05-03T15:05:32.538760+0000 (oldest deadline 2024-05-03T15:05:58.438503+0000) 2024-05-03T15:06:29.125 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:28 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:28.808+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:42.440172+0000 front 2024-05-03T15:05:42.440196+0000 (oldest deadline 2024-05-03T15:06:05.940332+0000) 2024-05-03T15:06:29.125 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:28 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:28.808+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:49.441284+0000 front 2024-05-03T15:05:49.441413+0000 (oldest deadline 2024-05-03T15:06:14.041739+0000) 2024-05-03T15:06:29.125 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:28 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:28.808+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:55.743016+0000 front 2024-05-03T15:05:55.743076+0000 (oldest deadline 2024-05-03T15:06:21.642629+0000) 2024-05-03T15:06:29.422 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:28 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:28.997+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.098256+0000 front 2024-05-03T15:05:40.098316+0000 (oldest deadline 2024-05-03T15:06:04.098995+0000) 2024-05-03T15:06:29.422 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:28 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:28.997+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.000252+0000 front 2024-05-03T15:05:50.000274+0000 (oldest deadline 2024-05-03T15:06:11.700413+0000) 2024-05-03T15:06:29.422 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:28 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:28.997+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:57.903054+0000 front 2024-05-03T15:05:57.903153+0000 (oldest deadline 2024-05-03T15:06:20.802884+0000) 2024-05-03T15:06:29.586 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:29 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:29.249+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.184568+0000 front 2024-05-03T15:05:32.184943+0000 (oldest deadline 2024-05-03T15:05:55.084762+0000) 2024-05-03T15:06:29.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:29 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:29.249+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.788481+0000 front 2024-05-03T15:05:40.788594+0000 (oldest deadline 2024-05-03T15:06:04.288673+0000) 2024-05-03T15:06:29.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:29 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:29.249+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:48.892458+0000 front 2024-05-03T15:05:48.892380+0000 (oldest deadline 2024-05-03T15:06:14.792663+0000) 2024-05-03T15:06:29.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:29 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:29.249+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:58.894896+0000 front 2024-05-03T15:05:58.894950+0000 (oldest deadline 2024-05-03T15:06:21.194384+0000) 2024-05-03T15:06:30.027 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:29 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:29.680+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:06:30.027 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:29 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:29.680+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:37.896657+0000 front 2024-05-03T15:05:37.896645+0000 (oldest deadline 2024-05-03T15:06:03.796605+0000) 2024-05-03T15:06:30.027 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:29 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:29.680+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.098837+0000 front 2024-05-03T15:05:50.098910+0000 (oldest deadline 2024-05-03T15:06:11.198967+0000) 2024-05-03T15:06:30.124 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:29 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:29.650+0000 7f487aafd700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:41.153998+0000 front 2024-05-03T15:05:41.154137+0000 (oldest deadline 2024-05-03T15:06:03.454048+0000) 2024-05-03T15:06:30.124 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:29 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:29.650+0000 7f487aafd700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.455657+0000 front 2024-05-03T15:05:50.455635+0000 (oldest deadline 2024-05-03T15:06:13.955669+0000) 2024-05-03T15:06:30.124 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:29 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:29.650+0000 7f487aafd700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:56.857259+0000 front 2024-05-03T15:05:56.857109+0000 (oldest deadline 2024-05-03T15:06:22.156806+0000) 2024-05-03T15:06:30.125 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:29 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:29.841+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.538644+0000 front 2024-05-03T15:05:32.538760+0000 (oldest deadline 2024-05-03T15:05:58.438503+0000) 2024-05-03T15:06:30.125 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:29 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:29.841+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:42.440172+0000 front 2024-05-03T15:05:42.440196+0000 (oldest deadline 2024-05-03T15:06:05.940332+0000) 2024-05-03T15:06:30.125 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:29 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:29.841+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:49.441284+0000 front 2024-05-03T15:05:49.441413+0000 (oldest deadline 2024-05-03T15:06:14.041739+0000) 2024-05-03T15:06:30.125 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:29 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:29.841+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:55.743016+0000 front 2024-05-03T15:05:55.743076+0000 (oldest deadline 2024-05-03T15:06:21.642629+0000) 2024-05-03T15:06:30.125 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:29 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:29.841+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:07.443866+0000 front 2024-05-03T15:06:07.443897+0000 (oldest deadline 2024-05-03T15:06:29.744048+0000) 2024-05-03T15:06:30.422 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:30 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:30.026+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.098256+0000 front 2024-05-03T15:05:40.098316+0000 (oldest deadline 2024-05-03T15:06:04.098995+0000) 2024-05-03T15:06:30.422 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:30 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:30.026+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.000252+0000 front 2024-05-03T15:05:50.000274+0000 (oldest deadline 2024-05-03T15:06:11.700413+0000) 2024-05-03T15:06:30.422 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:30 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:30.026+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:57.903054+0000 front 2024-05-03T15:05:57.903153+0000 (oldest deadline 2024-05-03T15:06:20.802884+0000) 2024-05-03T15:06:30.586 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:30 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:30.271+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.184568+0000 front 2024-05-03T15:05:32.184943+0000 (oldest deadline 2024-05-03T15:05:55.084762+0000) 2024-05-03T15:06:30.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:30 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:30.271+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.788481+0000 front 2024-05-03T15:05:40.788594+0000 (oldest deadline 2024-05-03T15:06:04.288673+0000) 2024-05-03T15:06:30.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:30 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:30.271+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:48.892458+0000 front 2024-05-03T15:05:48.892380+0000 (oldest deadline 2024-05-03T15:06:14.792663+0000) 2024-05-03T15:06:30.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:30 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:30.271+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:58.894896+0000 front 2024-05-03T15:05:58.894950+0000 (oldest deadline 2024-05-03T15:06:21.194384+0000) 2024-05-03T15:06:30.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:30 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:30.271+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:05.099256+0000 front 2024-05-03T15:06:05.099353+0000 (oldest deadline 2024-05-03T15:06:29.799510+0000) 2024-05-03T15:06:30.922 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:30 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:30.667+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:06:30.922 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:30 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:30.667+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:37.896657+0000 front 2024-05-03T15:05:37.896645+0000 (oldest deadline 2024-05-03T15:06:03.796605+0000) 2024-05-03T15:06:30.922 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:30 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:30.667+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.098837+0000 front 2024-05-03T15:05:50.098910+0000 (oldest deadline 2024-05-03T15:06:11.198967+0000) 2024-05-03T15:06:31.124 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:30 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:30.654+0000 7f487aafd700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:41.153998+0000 front 2024-05-03T15:05:41.154137+0000 (oldest deadline 2024-05-03T15:06:03.454048+0000) 2024-05-03T15:06:31.124 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:30 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:30.654+0000 7f487aafd700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.455657+0000 front 2024-05-03T15:05:50.455635+0000 (oldest deadline 2024-05-03T15:06:13.955669+0000) 2024-05-03T15:06:31.124 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:30 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:30.654+0000 7f487aafd700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:56.857259+0000 front 2024-05-03T15:05:56.857109+0000 (oldest deadline 2024-05-03T15:06:22.156806+0000) 2024-05-03T15:06:31.124 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:30 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:30.654+0000 7f487aafd700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:05.057831+0000 front 2024-05-03T15:06:05.057849+0000 (oldest deadline 2024-05-03T15:06:29.757722+0000) 2024-05-03T15:06:31.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:30 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:30.850+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.538644+0000 front 2024-05-03T15:05:32.538760+0000 (oldest deadline 2024-05-03T15:05:58.438503+0000) 2024-05-03T15:06:31.125 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:30 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:30.850+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:42.440172+0000 front 2024-05-03T15:05:42.440196+0000 (oldest deadline 2024-05-03T15:06:05.940332+0000) 2024-05-03T15:06:31.125 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:30 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:30.850+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:49.441284+0000 front 2024-05-03T15:05:49.441413+0000 (oldest deadline 2024-05-03T15:06:14.041739+0000) 2024-05-03T15:06:31.125 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:30 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:30.850+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:55.743016+0000 front 2024-05-03T15:05:55.743076+0000 (oldest deadline 2024-05-03T15:06:21.642629+0000) 2024-05-03T15:06:31.125 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:30 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:30.850+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:07.443866+0000 front 2024-05-03T15:06:07.443897+0000 (oldest deadline 2024-05-03T15:06:29.744048+0000) 2024-05-03T15:06:31.422 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:30 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:30.995+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.098256+0000 front 2024-05-03T15:05:40.098316+0000 (oldest deadline 2024-05-03T15:06:04.098995+0000) 2024-05-03T15:06:31.423 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:30 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:30.995+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.000252+0000 front 2024-05-03T15:05:50.000274+0000 (oldest deadline 2024-05-03T15:06:11.700413+0000) 2024-05-03T15:06:31.423 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:30 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:30.995+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:57.903054+0000 front 2024-05-03T15:05:57.903153+0000 (oldest deadline 2024-05-03T15:06:20.802884+0000) 2024-05-03T15:06:31.423 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:30 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:30.995+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:06.104131+0000 front 2024-05-03T15:06:06.104022+0000 (oldest deadline 2024-05-03T15:06:30.803920+0000) 2024-05-03T15:06:31.586 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:31 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:31.241+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.184568+0000 front 2024-05-03T15:05:32.184943+0000 (oldest deadline 2024-05-03T15:05:55.084762+0000) 2024-05-03T15:06:31.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:31 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:31.241+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.788481+0000 front 2024-05-03T15:05:40.788594+0000 (oldest deadline 2024-05-03T15:06:04.288673+0000) 2024-05-03T15:06:31.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:31 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:31.241+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:48.892458+0000 front 2024-05-03T15:05:48.892380+0000 (oldest deadline 2024-05-03T15:06:14.792663+0000) 2024-05-03T15:06:31.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:31 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:31.241+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:58.894896+0000 front 2024-05-03T15:05:58.894950+0000 (oldest deadline 2024-05-03T15:06:21.194384+0000) 2024-05-03T15:06:31.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:31 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:31.241+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:05.099256+0000 front 2024-05-03T15:06:05.099353+0000 (oldest deadline 2024-05-03T15:06:29.799510+0000) 2024-05-03T15:06:31.874 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:31 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:31.609+0000 7f487aafd700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:41.153998+0000 front 2024-05-03T15:05:41.154137+0000 (oldest deadline 2024-05-03T15:06:03.454048+0000) 2024-05-03T15:06:31.874 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:31 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:31.609+0000 7f487aafd700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.455657+0000 front 2024-05-03T15:05:50.455635+0000 (oldest deadline 2024-05-03T15:06:13.955669+0000) 2024-05-03T15:06:31.874 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:31 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:31.609+0000 7f487aafd700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:56.857259+0000 front 2024-05-03T15:05:56.857109+0000 (oldest deadline 2024-05-03T15:06:22.156806+0000) 2024-05-03T15:06:31.874 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:31 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:31.609+0000 7f487aafd700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:05.057831+0000 front 2024-05-03T15:06:05.057849+0000 (oldest deadline 2024-05-03T15:06:29.757722+0000) 2024-05-03T15:06:31.875 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:31 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:31.814+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.538644+0000 front 2024-05-03T15:05:32.538760+0000 (oldest deadline 2024-05-03T15:05:58.438503+0000) 2024-05-03T15:06:31.875 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:31 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:31.814+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:42.440172+0000 front 2024-05-03T15:05:42.440196+0000 (oldest deadline 2024-05-03T15:06:05.940332+0000) 2024-05-03T15:06:31.875 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:31 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:31.814+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:49.441284+0000 front 2024-05-03T15:05:49.441413+0000 (oldest deadline 2024-05-03T15:06:14.041739+0000) 2024-05-03T15:06:31.875 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:31 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:31.814+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:55.743016+0000 front 2024-05-03T15:05:55.743076+0000 (oldest deadline 2024-05-03T15:06:21.642629+0000) 2024-05-03T15:06:31.875 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:31 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:31.814+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:07.443866+0000 front 2024-05-03T15:06:07.443897+0000 (oldest deadline 2024-05-03T15:06:29.744048+0000) 2024-05-03T15:06:31.922 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:31 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:31.624+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:06:31.922 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:31 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:31.624+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:37.896657+0000 front 2024-05-03T15:05:37.896645+0000 (oldest deadline 2024-05-03T15:06:03.796605+0000) 2024-05-03T15:06:31.922 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:31 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:31.624+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.098837+0000 front 2024-05-03T15:05:50.098910+0000 (oldest deadline 2024-05-03T15:06:11.198967+0000) 2024-05-03T15:06:31.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:31 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:31.624+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:05.801228+0000 front 2024-05-03T15:06:05.801255+0000 (oldest deadline 2024-05-03T15:06:31.101050+0000) 2024-05-03T15:06:32.422 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:31 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:31.986+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.098256+0000 front 2024-05-03T15:05:40.098316+0000 (oldest deadline 2024-05-03T15:06:04.098995+0000) 2024-05-03T15:06:32.423 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:31 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:31.986+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.000252+0000 front 2024-05-03T15:05:50.000274+0000 (oldest deadline 2024-05-03T15:06:11.700413+0000) 2024-05-03T15:06:32.423 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:31 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:31.986+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:57.903054+0000 front 2024-05-03T15:05:57.903153+0000 (oldest deadline 2024-05-03T15:06:20.802884+0000) 2024-05-03T15:06:32.423 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:31 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:31.986+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:06.104131+0000 front 2024-05-03T15:06:06.104022+0000 (oldest deadline 2024-05-03T15:06:30.803920+0000) 2024-05-03T15:06:32.586 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:32 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:32.241+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.184568+0000 front 2024-05-03T15:05:32.184943+0000 (oldest deadline 2024-05-03T15:05:55.084762+0000) 2024-05-03T15:06:32.586 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:32 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:32.241+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.788481+0000 front 2024-05-03T15:05:40.788594+0000 (oldest deadline 2024-05-03T15:06:04.288673+0000) 2024-05-03T15:06:32.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:32 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:32.241+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:48.892458+0000 front 2024-05-03T15:05:48.892380+0000 (oldest deadline 2024-05-03T15:06:14.792663+0000) 2024-05-03T15:06:32.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:32 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:32.241+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:58.894896+0000 front 2024-05-03T15:05:58.894950+0000 (oldest deadline 2024-05-03T15:06:21.194384+0000) 2024-05-03T15:06:32.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:32 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:32.241+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:05.099256+0000 front 2024-05-03T15:06:05.099353+0000 (oldest deadline 2024-05-03T15:06:29.799510+0000) 2024-05-03T15:06:32.846 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:32 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:32.583+0000 7f487aafd700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:41.153998+0000 front 2024-05-03T15:05:41.154137+0000 (oldest deadline 2024-05-03T15:06:03.454048+0000) 2024-05-03T15:06:32.847 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:32 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:32.583+0000 7f487aafd700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.455657+0000 front 2024-05-03T15:05:50.455635+0000 (oldest deadline 2024-05-03T15:06:13.955669+0000) 2024-05-03T15:06:32.847 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:32 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:32.583+0000 7f487aafd700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:56.857259+0000 front 2024-05-03T15:05:56.857109+0000 (oldest deadline 2024-05-03T15:06:22.156806+0000) 2024-05-03T15:06:32.847 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:32 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7[51892]: 2024-05-03T15:06:32.583+0000 7f487aafd700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:05.057831+0000 front 2024-05-03T15:06:05.057849+0000 (oldest deadline 2024-05-03T15:06:29.757722+0000) 2024-05-03T15:06:32.922 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:32 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:32.631+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:06:32.922 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:32 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:32.631+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:37.896657+0000 front 2024-05-03T15:05:37.896645+0000 (oldest deadline 2024-05-03T15:06:03.796605+0000) 2024-05-03T15:06:32.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:32 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:32.631+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.098837+0000 front 2024-05-03T15:05:50.098910+0000 (oldest deadline 2024-05-03T15:06:11.198967+0000) 2024-05-03T15:06:32.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:32 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:32.631+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:05.801228+0000 front 2024-05-03T15:06:05.801255+0000 (oldest deadline 2024-05-03T15:06:31.101050+0000) 2024-05-03T15:06:33.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:32 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:32.845+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.538644+0000 front 2024-05-03T15:05:32.538760+0000 (oldest deadline 2024-05-03T15:05:58.438503+0000) 2024-05-03T15:06:33.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:32 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:32.845+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:42.440172+0000 front 2024-05-03T15:05:42.440196+0000 (oldest deadline 2024-05-03T15:06:05.940332+0000) 2024-05-03T15:06:33.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:32 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:32.845+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:49.441284+0000 front 2024-05-03T15:05:49.441413+0000 (oldest deadline 2024-05-03T15:06:14.041739+0000) 2024-05-03T15:06:33.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:32 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:32.845+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:55.743016+0000 front 2024-05-03T15:05:55.743076+0000 (oldest deadline 2024-05-03T15:06:21.642629+0000) 2024-05-03T15:06:33.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:32 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:32.845+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:07.443866+0000 front 2024-05-03T15:06:07.443897+0000 (oldest deadline 2024-05-03T15:06:29.744048+0000) 2024-05-03T15:06:33.354 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:32 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:32.945+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.098256+0000 front 2024-05-03T15:05:40.098316+0000 (oldest deadline 2024-05-03T15:06:04.098995+0000) 2024-05-03T15:06:33.354 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:32 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:32.945+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.000252+0000 front 2024-05-03T15:05:50.000274+0000 (oldest deadline 2024-05-03T15:06:11.700413+0000) 2024-05-03T15:06:33.354 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:32 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:32.945+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:57.903054+0000 front 2024-05-03T15:05:57.903153+0000 (oldest deadline 2024-05-03T15:06:20.802884+0000) 2024-05-03T15:06:33.354 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:32 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:32.945+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:06.104131+0000 front 2024-05-03T15:06:06.104022+0000 (oldest deadline 2024-05-03T15:06:30.803920+0000) 2024-05-03T15:06:33.586 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:33 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:33.203+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.184568+0000 front 2024-05-03T15:05:32.184943+0000 (oldest deadline 2024-05-03T15:05:55.084762+0000) 2024-05-03T15:06:33.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:33 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:33.203+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.788481+0000 front 2024-05-03T15:05:40.788594+0000 (oldest deadline 2024-05-03T15:06:04.288673+0000) 2024-05-03T15:06:33.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:33 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:33.203+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:48.892458+0000 front 2024-05-03T15:05:48.892380+0000 (oldest deadline 2024-05-03T15:06:14.792663+0000) 2024-05-03T15:06:33.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:33 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:33.203+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:58.894896+0000 front 2024-05-03T15:05:58.894950+0000 (oldest deadline 2024-05-03T15:06:21.194384+0000) 2024-05-03T15:06:33.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:33 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:33.203+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:05.099256+0000 front 2024-05-03T15:06:05.099353+0000 (oldest deadline 2024-05-03T15:06:29.799510+0000) 2024-05-03T15:06:33.672 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:33 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:33.591+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:06:33.672 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:33 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:33.591+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:37.896657+0000 front 2024-05-03T15:05:37.896645+0000 (oldest deadline 2024-05-03T15:06:03.796605+0000) 2024-05-03T15:06:33.672 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:33 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:33.591+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.098837+0000 front 2024-05-03T15:05:50.098910+0000 (oldest deadline 2024-05-03T15:06:11.198967+0000) 2024-05-03T15:06:33.673 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:33 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:33.591+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:05.801228+0000 front 2024-05-03T15:06:05.801255+0000 (oldest deadline 2024-05-03T15:06:31.101050+0000) 2024-05-03T15:06:33.835 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:33 smithi090 podman[69302]: 2024-05-03 15:06:33.546141381 +0000 UTC m=+5.615391914 container died 9d22c76a3a2ea2b759dc8fcf073928f6acb9c576b7e5d0df3ae4a95f9e61ef1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-05-03T15:06:34.124 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:33 smithi090 podman[69302]: 2024-05-03 15:06:33.926232669 +0000 UTC m=+5.995483200 container cleanup 9d22c76a3a2ea2b759dc8fcf073928f6acb9c576b7e5d0df3ae4a95f9e61ef1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux ) 2024-05-03T15:06:34.124 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:33 smithi090 bash[69302]: ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7 2024-05-03T15:06:34.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:33 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:33.856+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.538644+0000 front 2024-05-03T15:05:32.538760+0000 (oldest deadline 2024-05-03T15:05:58.438503+0000) 2024-05-03T15:06:34.125 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:33 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:33.856+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:42.440172+0000 front 2024-05-03T15:05:42.440196+0000 (oldest deadline 2024-05-03T15:06:05.940332+0000) 2024-05-03T15:06:34.125 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:33 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:33.856+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:49.441284+0000 front 2024-05-03T15:05:49.441413+0000 (oldest deadline 2024-05-03T15:06:14.041739+0000) 2024-05-03T15:06:34.125 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:33 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:33.856+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:55.743016+0000 front 2024-05-03T15:05:55.743076+0000 (oldest deadline 2024-05-03T15:06:21.642629+0000) 2024-05-03T15:06:34.125 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:33 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:33.856+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:07.443866+0000 front 2024-05-03T15:06:07.443897+0000 (oldest deadline 2024-05-03T15:06:29.744048+0000) 2024-05-03T15:06:34.172 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:33 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:33.912+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.098256+0000 front 2024-05-03T15:05:40.098316+0000 (oldest deadline 2024-05-03T15:06:04.098995+0000) 2024-05-03T15:06:34.173 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:33 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:33.912+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.000252+0000 front 2024-05-03T15:05:50.000274+0000 (oldest deadline 2024-05-03T15:06:11.700413+0000) 2024-05-03T15:06:34.173 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:33 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:33.912+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:57.903054+0000 front 2024-05-03T15:05:57.903153+0000 (oldest deadline 2024-05-03T15:06:20.802884+0000) 2024-05-03T15:06:34.173 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:33 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:33.912+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:06.104131+0000 front 2024-05-03T15:06:06.104022+0000 (oldest deadline 2024-05-03T15:06:30.803920+0000) 2024-05-03T15:06:34.550 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:34 smithi090 podman[69319]: 2024-05-03 15:06:34.211198879 +0000 UTC m=+0.658653758 container remove 9d22c76a3a2ea2b759dc8fcf073928f6acb9c576b7e5d0df3ae4a95f9e61ef1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True) 2024-05-03T15:06:34.566 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:34 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:34.177+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.184568+0000 front 2024-05-03T15:05:32.184943+0000 (oldest deadline 2024-05-03T15:05:55.084762+0000) 2024-05-03T15:06:34.566 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:34 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:34.177+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.788481+0000 front 2024-05-03T15:05:40.788594+0000 (oldest deadline 2024-05-03T15:06:04.288673+0000) 2024-05-03T15:06:34.566 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:34 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:34.177+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:48.892458+0000 front 2024-05-03T15:05:48.892380+0000 (oldest deadline 2024-05-03T15:06:14.792663+0000) 2024-05-03T15:06:34.566 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:34 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:34.177+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:58.894896+0000 front 2024-05-03T15:05:58.894950+0000 (oldest deadline 2024-05-03T15:06:21.194384+0000) 2024-05-03T15:06:34.567 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:34 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:34.177+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:05.099256+0000 front 2024-05-03T15:06:05.099353+0000 (oldest deadline 2024-05-03T15:06:29.799510+0000) 2024-05-03T15:06:34.809 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:34 smithi090 podman[69410]: 2024-05-03 15:06:34.453305599 +0000 UTC m=+0.018279022 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T15:06:34.809 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:34 smithi090 podman[69410]: 2024-05-03 15:06:34.563571474 +0000 UTC m=+0.128544944 container create 55cafaadcd25f3736681c56e597e983d27f1483fde07d993ce8653db9cc37eaa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240430) 2024-05-03T15:06:34.922 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:34 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:34.604+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:06:34.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:34 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:34.604+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:37.896657+0000 front 2024-05-03T15:05:37.896645+0000 (oldest deadline 2024-05-03T15:06:03.796605+0000) 2024-05-03T15:06:34.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:34 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:34.604+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.098837+0000 front 2024-05-03T15:05:50.098910+0000 (oldest deadline 2024-05-03T15:06:11.198967+0000) 2024-05-03T15:06:34.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:34 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:34.604+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:05.801228+0000 front 2024-05-03T15:06:05.801255+0000 (oldest deadline 2024-05-03T15:06:31.101050+0000) 2024-05-03T15:06:35.099 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:34 smithi090 podman[69410]: 2024-05-03 15:06:34.808665106 +0000 UTC m=+0.373638521 container init 55cafaadcd25f3736681c56e597e983d27f1483fde07d993ce8653db9cc37eaa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-05-03T15:06:35.099 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:34 smithi090 podman[69410]: 2024-05-03 15:06:34.81194659 +0000 UTC m=+0.376919999 container start 55cafaadcd25f3736681c56e597e983d27f1483fde07d993ce8653db9cc37eaa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-03T15:06:35.100 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:34 smithi090 podman[69410]: 2024-05-03 15:06:34.883555749 +0000 UTC m=+0.448529157 container attach 55cafaadcd25f3736681c56e597e983d27f1483fde07d993ce8653db9cc37eaa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True) 2024-05-03T15:06:35.100 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:34 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:34.900+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.538644+0000 front 2024-05-03T15:05:32.538760+0000 (oldest deadline 2024-05-03T15:05:58.438503+0000) 2024-05-03T15:06:35.100 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:34 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:34.900+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:42.440172+0000 front 2024-05-03T15:05:42.440196+0000 (oldest deadline 2024-05-03T15:06:05.940332+0000) 2024-05-03T15:06:35.100 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:34 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:34.900+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:49.441284+0000 front 2024-05-03T15:05:49.441413+0000 (oldest deadline 2024-05-03T15:06:14.041739+0000) 2024-05-03T15:06:35.100 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:34 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:34.900+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:55.743016+0000 front 2024-05-03T15:05:55.743076+0000 (oldest deadline 2024-05-03T15:06:21.642629+0000) 2024-05-03T15:06:35.100 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:34 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:34.900+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:07.443866+0000 front 2024-05-03T15:06:07.443897+0000 (oldest deadline 2024-05-03T15:06:29.744048+0000) 2024-05-03T15:06:35.374 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:35 smithi090 podman[69410]: 2024-05-03 15:06:35.099189094 +0000 UTC m=+0.664162495 container died 55cafaadcd25f3736681c56e597e983d27f1483fde07d993ce8653db9cc37eaa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7-deactivate, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-05-03T15:06:35.422 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:34 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:34.960+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.098256+0000 front 2024-05-03T15:05:40.098316+0000 (oldest deadline 2024-05-03T15:06:04.098995+0000) 2024-05-03T15:06:35.422 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:34 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:34.960+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.000252+0000 front 2024-05-03T15:05:50.000274+0000 (oldest deadline 2024-05-03T15:06:11.700413+0000) 2024-05-03T15:06:35.423 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:34 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:34.960+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:57.903054+0000 front 2024-05-03T15:05:57.903153+0000 (oldest deadline 2024-05-03T15:06:20.802884+0000) 2024-05-03T15:06:35.423 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:34 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:34.960+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:06.104131+0000 front 2024-05-03T15:06:06.104022+0000 (oldest deadline 2024-05-03T15:06:30.803920+0000) 2024-05-03T15:06:35.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:35 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:35.168+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.184568+0000 front 2024-05-03T15:05:32.184943+0000 (oldest deadline 2024-05-03T15:05:55.084762+0000) 2024-05-03T15:06:35.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:35 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:35.169+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.788481+0000 front 2024-05-03T15:05:40.788594+0000 (oldest deadline 2024-05-03T15:06:04.288673+0000) 2024-05-03T15:06:35.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:35 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:35.169+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:48.892458+0000 front 2024-05-03T15:05:48.892380+0000 (oldest deadline 2024-05-03T15:06:14.792663+0000) 2024-05-03T15:06:35.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:35 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:35.169+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:58.894896+0000 front 2024-05-03T15:05:58.894950+0000 (oldest deadline 2024-05-03T15:06:21.194384+0000) 2024-05-03T15:06:35.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:35 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:35.169+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:05.099256+0000 front 2024-05-03T15:06:05.099353+0000 (oldest deadline 2024-05-03T15:06:29.799510+0000) 2024-05-03T15:06:35.628 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:35 smithi090 podman[69436]: 2024-05-03 15:06:35.613046369 +0000 UTC m=+0.503990846 container remove 55cafaadcd25f3736681c56e597e983d27f1483fde07d993ce8653db9cc37eaa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-03T15:06:35.628 INFO:journalctl@ceph.osd.7.smithi090.stdout:May 03 15:06:35 smithi090 systemd[1]: ceph-285d64b2-095a-11ef-bc97-c7b262605968@osd.7.service: Deactivated successfully. 2024-05-03T15:06:35.632 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-285d64b2-095a-11ef-bc97-c7b262605968@osd.7.service' 2024-05-03T15:06:35.669 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T15:06:35.669 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-05-03T15:06:35.670 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-05-03T15:06:35.670 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-285d64b2-095a-11ef-bc97-c7b262605968@osd.8 2024-05-03T15:06:35.920 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:35 smithi149 systemd[1]: Stopping Ceph osd.8 for 285d64b2-095a-11ef-bc97-c7b262605968... 2024-05-03T15:06:35.920 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:35 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:35.580+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:06:35.920 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:35 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:35.580+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:37.896657+0000 front 2024-05-03T15:05:37.896645+0000 (oldest deadline 2024-05-03T15:06:03.796605+0000) 2024-05-03T15:06:35.920 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:35 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:35.580+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.098837+0000 front 2024-05-03T15:05:50.098910+0000 (oldest deadline 2024-05-03T15:06:11.198967+0000) 2024-05-03T15:06:35.921 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:35 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:35.580+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:05.801228+0000 front 2024-05-03T15:06:05.801255+0000 (oldest deadline 2024-05-03T15:06:31.101050+0000) 2024-05-03T15:06:36.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:35 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:35.863+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.538644+0000 front 2024-05-03T15:05:32.538760+0000 (oldest deadline 2024-05-03T15:05:58.438503+0000) 2024-05-03T15:06:36.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:35 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:35.863+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:42.440172+0000 front 2024-05-03T15:05:42.440196+0000 (oldest deadline 2024-05-03T15:06:05.940332+0000) 2024-05-03T15:06:36.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:35 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:35.863+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:49.441284+0000 front 2024-05-03T15:05:49.441413+0000 (oldest deadline 2024-05-03T15:06:14.041739+0000) 2024-05-03T15:06:36.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:35 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:35.863+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:55.743016+0000 front 2024-05-03T15:05:55.743076+0000 (oldest deadline 2024-05-03T15:06:21.642629+0000) 2024-05-03T15:06:36.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:35 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:35.863+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:07.443866+0000 front 2024-05-03T15:06:07.443897+0000 (oldest deadline 2024-05-03T15:06:29.744048+0000) 2024-05-03T15:06:36.172 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:35 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:35.918+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.098256+0000 front 2024-05-03T15:05:40.098316+0000 (oldest deadline 2024-05-03T15:06:04.098995+0000) 2024-05-03T15:06:36.172 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:35 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:35.918+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.000252+0000 front 2024-05-03T15:05:50.000274+0000 (oldest deadline 2024-05-03T15:06:11.700413+0000) 2024-05-03T15:06:36.172 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:35 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:35.918+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:57.903054+0000 front 2024-05-03T15:05:57.903153+0000 (oldest deadline 2024-05-03T15:06:20.802884+0000) 2024-05-03T15:06:36.172 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:35 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:35.918+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:06.104131+0000 front 2024-05-03T15:06:06.104022+0000 (oldest deadline 2024-05-03T15:06:30.803920+0000) 2024-05-03T15:06:36.586 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:36 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:36.132+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.184568+0000 front 2024-05-03T15:05:32.184943+0000 (oldest deadline 2024-05-03T15:05:55.084762+0000) 2024-05-03T15:06:36.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:36 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:36.132+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.788481+0000 front 2024-05-03T15:05:40.788594+0000 (oldest deadline 2024-05-03T15:06:04.288673+0000) 2024-05-03T15:06:36.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:36 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:36.132+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:48.892458+0000 front 2024-05-03T15:05:48.892380+0000 (oldest deadline 2024-05-03T15:06:14.792663+0000) 2024-05-03T15:06:36.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:36 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:36.132+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:58.894896+0000 front 2024-05-03T15:05:58.894950+0000 (oldest deadline 2024-05-03T15:06:21.194384+0000) 2024-05-03T15:06:36.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:36 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:36.132+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:05.099256+0000 front 2024-05-03T15:06:05.099353+0000 (oldest deadline 2024-05-03T15:06:29.799510+0000) 2024-05-03T15:06:36.616 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:36 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:36.258+0000 7f25570d1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T15:06:36.616 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:36 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:36.258+0000 7f25570d1700 -1 osd.8 93 *** Got signal Terminated *** 2024-05-03T15:06:36.616 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:36 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:36.258+0000 7f25570d1700 -1 osd.8 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-03T15:06:36.922 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:36 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:36.615+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:06:36.922 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:36 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:36.615+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:37.896657+0000 front 2024-05-03T15:05:37.896645+0000 (oldest deadline 2024-05-03T15:06:03.796605+0000) 2024-05-03T15:06:36.922 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:36 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:36.615+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.098837+0000 front 2024-05-03T15:05:50.098910+0000 (oldest deadline 2024-05-03T15:06:11.198967+0000) 2024-05-03T15:06:36.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:36 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:36.615+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:05.801228+0000 front 2024-05-03T15:06:05.801255+0000 (oldest deadline 2024-05-03T15:06:31.101050+0000) 2024-05-03T15:06:37.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:36 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:36.900+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.538644+0000 front 2024-05-03T15:05:32.538760+0000 (oldest deadline 2024-05-03T15:05:58.438503+0000) 2024-05-03T15:06:37.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:36 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:36.900+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:42.440172+0000 front 2024-05-03T15:05:42.440196+0000 (oldest deadline 2024-05-03T15:06:05.940332+0000) 2024-05-03T15:06:37.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:36 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:36.900+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:49.441284+0000 front 2024-05-03T15:05:49.441413+0000 (oldest deadline 2024-05-03T15:06:14.041739+0000) 2024-05-03T15:06:37.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:36 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:36.900+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:55.743016+0000 front 2024-05-03T15:05:55.743076+0000 (oldest deadline 2024-05-03T15:06:21.642629+0000) 2024-05-03T15:06:37.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:36 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:36.900+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:07.443866+0000 front 2024-05-03T15:06:07.443897+0000 (oldest deadline 2024-05-03T15:06:29.744048+0000) 2024-05-03T15:06:37.422 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:36 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:36.927+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.098256+0000 front 2024-05-03T15:05:40.098316+0000 (oldest deadline 2024-05-03T15:06:04.098995+0000) 2024-05-03T15:06:37.422 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:36 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:36.927+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.000252+0000 front 2024-05-03T15:05:50.000274+0000 (oldest deadline 2024-05-03T15:06:11.700413+0000) 2024-05-03T15:06:37.422 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:36 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:36.927+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:57.903054+0000 front 2024-05-03T15:05:57.903153+0000 (oldest deadline 2024-05-03T15:06:20.802884+0000) 2024-05-03T15:06:37.423 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:36 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:36.927+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:06.104131+0000 front 2024-05-03T15:06:06.104022+0000 (oldest deadline 2024-05-03T15:06:30.803920+0000) 2024-05-03T15:06:37.586 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:37 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:37.163+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.184568+0000 front 2024-05-03T15:05:32.184943+0000 (oldest deadline 2024-05-03T15:05:55.084762+0000) 2024-05-03T15:06:37.586 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:37 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:37.163+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.788481+0000 front 2024-05-03T15:05:40.788594+0000 (oldest deadline 2024-05-03T15:06:04.288673+0000) 2024-05-03T15:06:37.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:37 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:37.163+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:48.892458+0000 front 2024-05-03T15:05:48.892380+0000 (oldest deadline 2024-05-03T15:06:14.792663+0000) 2024-05-03T15:06:37.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:37 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:37.163+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:58.894896+0000 front 2024-05-03T15:05:58.894950+0000 (oldest deadline 2024-05-03T15:06:21.194384+0000) 2024-05-03T15:06:37.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:37 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:37.163+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:05.099256+0000 front 2024-05-03T15:06:05.099353+0000 (oldest deadline 2024-05-03T15:06:29.799510+0000) 2024-05-03T15:06:37.922 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:37 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:37.631+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:06:37.922 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:37 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:37.631+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:37.896657+0000 front 2024-05-03T15:05:37.896645+0000 (oldest deadline 2024-05-03T15:06:03.796605+0000) 2024-05-03T15:06:37.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:37 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:37.631+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.098837+0000 front 2024-05-03T15:05:50.098910+0000 (oldest deadline 2024-05-03T15:06:11.198967+0000) 2024-05-03T15:06:37.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:37 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:37.631+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:05.801228+0000 front 2024-05-03T15:06:05.801255+0000 (oldest deadline 2024-05-03T15:06:31.101050+0000) 2024-05-03T15:06:38.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:37 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:37.859+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.538644+0000 front 2024-05-03T15:05:32.538760+0000 (oldest deadline 2024-05-03T15:05:58.438503+0000) 2024-05-03T15:06:38.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:37 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:37.859+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:42.440172+0000 front 2024-05-03T15:05:42.440196+0000 (oldest deadline 2024-05-03T15:06:05.940332+0000) 2024-05-03T15:06:38.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:37 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:37.859+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:49.441284+0000 front 2024-05-03T15:05:49.441413+0000 (oldest deadline 2024-05-03T15:06:14.041739+0000) 2024-05-03T15:06:38.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:37 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:37.859+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:55.743016+0000 front 2024-05-03T15:05:55.743076+0000 (oldest deadline 2024-05-03T15:06:21.642629+0000) 2024-05-03T15:06:38.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:37 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:37.859+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:07.443866+0000 front 2024-05-03T15:06:07.443897+0000 (oldest deadline 2024-05-03T15:06:29.744048+0000) 2024-05-03T15:06:38.422 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:37 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:37.922+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.098256+0000 front 2024-05-03T15:05:40.098316+0000 (oldest deadline 2024-05-03T15:06:04.098995+0000) 2024-05-03T15:06:38.422 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:37 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:37.922+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.000252+0000 front 2024-05-03T15:05:50.000274+0000 (oldest deadline 2024-05-03T15:06:11.700413+0000) 2024-05-03T15:06:38.423 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:37 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:37.922+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:57.903054+0000 front 2024-05-03T15:05:57.903153+0000 (oldest deadline 2024-05-03T15:06:20.802884+0000) 2024-05-03T15:06:38.423 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:37 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:37.922+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:06.104131+0000 front 2024-05-03T15:06:06.104022+0000 (oldest deadline 2024-05-03T15:06:30.803920+0000) 2024-05-03T15:06:38.586 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:38 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:38.118+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.184568+0000 front 2024-05-03T15:05:32.184943+0000 (oldest deadline 2024-05-03T15:05:55.084762+0000) 2024-05-03T15:06:38.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:38 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:38.118+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.788481+0000 front 2024-05-03T15:05:40.788594+0000 (oldest deadline 2024-05-03T15:06:04.288673+0000) 2024-05-03T15:06:38.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:38 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:38.118+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:48.892458+0000 front 2024-05-03T15:05:48.892380+0000 (oldest deadline 2024-05-03T15:06:14.792663+0000) 2024-05-03T15:06:38.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:38 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:38.118+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:58.894896+0000 front 2024-05-03T15:05:58.894950+0000 (oldest deadline 2024-05-03T15:06:21.194384+0000) 2024-05-03T15:06:38.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:38 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:38.118+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:05.099256+0000 front 2024-05-03T15:06:05.099353+0000 (oldest deadline 2024-05-03T15:06:29.799510+0000) 2024-05-03T15:06:38.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:38 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:38.118+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-05-03T15:06:11.501358+0000 front 2024-05-03T15:06:11.501226+0000 (oldest deadline 2024-05-03T15:06:37.401492+0000) 2024-05-03T15:06:38.922 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:38 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:38.648+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:06:38.922 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:38 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:38.648+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:37.896657+0000 front 2024-05-03T15:05:37.896645+0000 (oldest deadline 2024-05-03T15:06:03.796605+0000) 2024-05-03T15:06:38.922 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:38 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:38.648+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.098837+0000 front 2024-05-03T15:05:50.098910+0000 (oldest deadline 2024-05-03T15:06:11.198967+0000) 2024-05-03T15:06:38.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:38 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:38.648+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:05.801228+0000 front 2024-05-03T15:06:05.801255+0000 (oldest deadline 2024-05-03T15:06:31.101050+0000) 2024-05-03T15:06:39.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:38 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:38.871+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.538644+0000 front 2024-05-03T15:05:32.538760+0000 (oldest deadline 2024-05-03T15:05:58.438503+0000) 2024-05-03T15:06:39.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:38 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:38.871+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:42.440172+0000 front 2024-05-03T15:05:42.440196+0000 (oldest deadline 2024-05-03T15:06:05.940332+0000) 2024-05-03T15:06:39.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:38 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:38.871+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:49.441284+0000 front 2024-05-03T15:05:49.441413+0000 (oldest deadline 2024-05-03T15:06:14.041739+0000) 2024-05-03T15:06:39.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:38 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:38.871+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:55.743016+0000 front 2024-05-03T15:05:55.743076+0000 (oldest deadline 2024-05-03T15:06:21.642629+0000) 2024-05-03T15:06:39.125 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:38 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:38.871+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:07.443866+0000 front 2024-05-03T15:06:07.443897+0000 (oldest deadline 2024-05-03T15:06:29.744048+0000) 2024-05-03T15:06:39.172 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:38 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:38.921+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.098256+0000 front 2024-05-03T15:05:40.098316+0000 (oldest deadline 2024-05-03T15:06:04.098995+0000) 2024-05-03T15:06:39.172 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:38 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:38.921+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.000252+0000 front 2024-05-03T15:05:50.000274+0000 (oldest deadline 2024-05-03T15:06:11.700413+0000) 2024-05-03T15:06:39.173 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:38 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:38.921+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:57.903054+0000 front 2024-05-03T15:05:57.903153+0000 (oldest deadline 2024-05-03T15:06:20.802884+0000) 2024-05-03T15:06:39.173 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:38 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:38.921+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:06.104131+0000 front 2024-05-03T15:06:06.104022+0000 (oldest deadline 2024-05-03T15:06:30.803920+0000) 2024-05-03T15:06:39.173 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:38 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:38.921+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-05-03T15:06:14.304769+0000 front 2024-05-03T15:06:14.304805+0000 (oldest deadline 2024-05-03T15:06:38.404937+0000) 2024-05-03T15:06:39.586 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:39 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:39.130+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.184568+0000 front 2024-05-03T15:05:32.184943+0000 (oldest deadline 2024-05-03T15:05:55.084762+0000) 2024-05-03T15:06:39.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:39 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:39.130+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.788481+0000 front 2024-05-03T15:05:40.788594+0000 (oldest deadline 2024-05-03T15:06:04.288673+0000) 2024-05-03T15:06:39.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:39 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:39.130+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:48.892458+0000 front 2024-05-03T15:05:48.892380+0000 (oldest deadline 2024-05-03T15:06:14.792663+0000) 2024-05-03T15:06:39.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:39 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:39.130+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:58.894896+0000 front 2024-05-03T15:05:58.894950+0000 (oldest deadline 2024-05-03T15:06:21.194384+0000) 2024-05-03T15:06:39.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:39 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:39.130+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:05.099256+0000 front 2024-05-03T15:06:05.099353+0000 (oldest deadline 2024-05-03T15:06:29.799510+0000) 2024-05-03T15:06:39.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:39 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:39.130+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-05-03T15:06:11.501358+0000 front 2024-05-03T15:06:11.501226+0000 (oldest deadline 2024-05-03T15:06:37.401492+0000) 2024-05-03T15:06:39.946 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:39 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:39.688+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:06:39.946 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:39 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:39.688+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:37.896657+0000 front 2024-05-03T15:05:37.896645+0000 (oldest deadline 2024-05-03T15:06:03.796605+0000) 2024-05-03T15:06:39.946 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:39 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:39.689+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.098837+0000 front 2024-05-03T15:05:50.098910+0000 (oldest deadline 2024-05-03T15:06:11.198967+0000) 2024-05-03T15:06:39.946 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:39 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:39.689+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:05.801228+0000 front 2024-05-03T15:06:05.801255+0000 (oldest deadline 2024-05-03T15:06:31.101050+0000) 2024-05-03T15:06:40.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:39 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:39.840+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.538644+0000 front 2024-05-03T15:05:32.538760+0000 (oldest deadline 2024-05-03T15:05:58.438503+0000) 2024-05-03T15:06:40.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:39 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:39.840+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:42.440172+0000 front 2024-05-03T15:05:42.440196+0000 (oldest deadline 2024-05-03T15:06:05.940332+0000) 2024-05-03T15:06:40.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:39 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:39.840+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:49.441284+0000 front 2024-05-03T15:05:49.441413+0000 (oldest deadline 2024-05-03T15:06:14.041739+0000) 2024-05-03T15:06:40.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:39 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:39.840+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:55.743016+0000 front 2024-05-03T15:05:55.743076+0000 (oldest deadline 2024-05-03T15:06:21.642629+0000) 2024-05-03T15:06:40.125 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:39 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:39.840+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:07.443866+0000 front 2024-05-03T15:06:07.443897+0000 (oldest deadline 2024-05-03T15:06:29.744048+0000) 2024-05-03T15:06:40.125 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:39 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:39.840+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-05-03T15:06:14.845949+0000 front 2024-05-03T15:06:14.845978+0000 (oldest deadline 2024-05-03T15:06:39.545914+0000) 2024-05-03T15:06:40.422 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:39 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:39.945+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.098256+0000 front 2024-05-03T15:05:40.098316+0000 (oldest deadline 2024-05-03T15:06:04.098995+0000) 2024-05-03T15:06:40.422 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:39 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:39.945+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.000252+0000 front 2024-05-03T15:05:50.000274+0000 (oldest deadline 2024-05-03T15:06:11.700413+0000) 2024-05-03T15:06:40.423 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:39 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:39.945+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:57.903054+0000 front 2024-05-03T15:05:57.903153+0000 (oldest deadline 2024-05-03T15:06:20.802884+0000) 2024-05-03T15:06:40.423 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:39 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:39.945+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:06.104131+0000 front 2024-05-03T15:06:06.104022+0000 (oldest deadline 2024-05-03T15:06:30.803920+0000) 2024-05-03T15:06:40.423 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:39 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:39.945+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-05-03T15:06:14.304769+0000 front 2024-05-03T15:06:14.304805+0000 (oldest deadline 2024-05-03T15:06:38.404937+0000) 2024-05-03T15:06:40.586 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:40 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:40.162+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.184568+0000 front 2024-05-03T15:05:32.184943+0000 (oldest deadline 2024-05-03T15:05:55.084762+0000) 2024-05-03T15:06:40.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:40 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:40.162+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.788481+0000 front 2024-05-03T15:05:40.788594+0000 (oldest deadline 2024-05-03T15:06:04.288673+0000) 2024-05-03T15:06:40.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:40 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:40.162+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:48.892458+0000 front 2024-05-03T15:05:48.892380+0000 (oldest deadline 2024-05-03T15:06:14.792663+0000) 2024-05-03T15:06:40.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:40 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:40.162+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:58.894896+0000 front 2024-05-03T15:05:58.894950+0000 (oldest deadline 2024-05-03T15:06:21.194384+0000) 2024-05-03T15:06:40.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:40 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:40.162+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:05.099256+0000 front 2024-05-03T15:06:05.099353+0000 (oldest deadline 2024-05-03T15:06:29.799510+0000) 2024-05-03T15:06:40.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:40 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:40.162+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-05-03T15:06:11.501358+0000 front 2024-05-03T15:06:11.501226+0000 (oldest deadline 2024-05-03T15:06:37.401492+0000) 2024-05-03T15:06:40.922 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:40 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:40.650+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:06:40.922 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:40 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:40.650+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:37.896657+0000 front 2024-05-03T15:05:37.896645+0000 (oldest deadline 2024-05-03T15:06:03.796605+0000) 2024-05-03T15:06:40.922 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:40 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:40.650+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.098837+0000 front 2024-05-03T15:05:50.098910+0000 (oldest deadline 2024-05-03T15:06:11.198967+0000) 2024-05-03T15:06:40.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:40 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:40.650+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:05.801228+0000 front 2024-05-03T15:06:05.801255+0000 (oldest deadline 2024-05-03T15:06:31.101050+0000) 2024-05-03T15:06:41.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:40 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:40.795+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.538644+0000 front 2024-05-03T15:05:32.538760+0000 (oldest deadline 2024-05-03T15:05:58.438503+0000) 2024-05-03T15:06:41.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:40 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:40.795+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:42.440172+0000 front 2024-05-03T15:05:42.440196+0000 (oldest deadline 2024-05-03T15:06:05.940332+0000) 2024-05-03T15:06:41.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:40 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:40.795+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:49.441284+0000 front 2024-05-03T15:05:49.441413+0000 (oldest deadline 2024-05-03T15:06:14.041739+0000) 2024-05-03T15:06:41.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:40 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:40.795+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:55.743016+0000 front 2024-05-03T15:05:55.743076+0000 (oldest deadline 2024-05-03T15:06:21.642629+0000) 2024-05-03T15:06:41.125 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:40 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:40.795+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:07.443866+0000 front 2024-05-03T15:06:07.443897+0000 (oldest deadline 2024-05-03T15:06:29.744048+0000) 2024-05-03T15:06:41.125 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:40 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:40.795+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-05-03T15:06:14.845949+0000 front 2024-05-03T15:06:14.845978+0000 (oldest deadline 2024-05-03T15:06:39.545914+0000) 2024-05-03T15:06:41.422 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:40 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:40.984+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.098256+0000 front 2024-05-03T15:05:40.098316+0000 (oldest deadline 2024-05-03T15:06:04.098995+0000) 2024-05-03T15:06:41.423 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:40 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:40.984+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.000252+0000 front 2024-05-03T15:05:50.000274+0000 (oldest deadline 2024-05-03T15:06:11.700413+0000) 2024-05-03T15:06:41.423 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:40 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:40.984+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:57.903054+0000 front 2024-05-03T15:05:57.903153+0000 (oldest deadline 2024-05-03T15:06:20.802884+0000) 2024-05-03T15:06:41.423 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:40 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:40.984+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:06.104131+0000 front 2024-05-03T15:06:06.104022+0000 (oldest deadline 2024-05-03T15:06:30.803920+0000) 2024-05-03T15:06:41.423 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:40 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8[52185]: 2024-05-03T15:06:40.984+0000 7f254f6bd700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-05-03T15:06:14.304769+0000 front 2024-05-03T15:06:14.304805+0000 (oldest deadline 2024-05-03T15:06:38.404937+0000) 2024-05-03T15:06:41.586 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:41 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:41.113+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.184568+0000 front 2024-05-03T15:05:32.184943+0000 (oldest deadline 2024-05-03T15:05:55.084762+0000) 2024-05-03T15:06:41.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:41 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:41.113+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.788481+0000 front 2024-05-03T15:05:40.788594+0000 (oldest deadline 2024-05-03T15:06:04.288673+0000) 2024-05-03T15:06:41.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:41 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:41.113+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:48.892458+0000 front 2024-05-03T15:05:48.892380+0000 (oldest deadline 2024-05-03T15:06:14.792663+0000) 2024-05-03T15:06:41.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:41 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:41.113+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:58.894896+0000 front 2024-05-03T15:05:58.894950+0000 (oldest deadline 2024-05-03T15:06:21.194384+0000) 2024-05-03T15:06:41.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:41 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:41.113+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:05.099256+0000 front 2024-05-03T15:06:05.099353+0000 (oldest deadline 2024-05-03T15:06:29.799510+0000) 2024-05-03T15:06:41.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:41 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:41.113+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-05-03T15:06:11.501358+0000 front 2024-05-03T15:06:11.501226+0000 (oldest deadline 2024-05-03T15:06:37.401492+0000) 2024-05-03T15:06:41.901 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:41 smithi149 podman[70127]: 2024-05-03 15:06:41.647469302 +0000 UTC m=+5.769856681 container died cb0438d840fbcbf0adefec5bd1b60e5b2fafeaaaafa8180daa81114b00111217 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-03T15:06:41.901 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:41 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:41.671+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:06:41.901 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:41 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:41.671+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:37.896657+0000 front 2024-05-03T15:05:37.896645+0000 (oldest deadline 2024-05-03T15:06:03.796605+0000) 2024-05-03T15:06:41.901 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:41 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:41.671+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.098837+0000 front 2024-05-03T15:05:50.098910+0000 (oldest deadline 2024-05-03T15:06:11.198967+0000) 2024-05-03T15:06:41.901 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:41 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:41.671+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:05.801228+0000 front 2024-05-03T15:06:05.801255+0000 (oldest deadline 2024-05-03T15:06:31.101050+0000) 2024-05-03T15:06:41.901 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:41 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:41.671+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-05-03T15:06:15.002844+0000 front 2024-05-03T15:06:15.002992+0000 (oldest deadline 2024-05-03T15:06:40.902980+0000) 2024-05-03T15:06:42.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:41 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:41.840+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.538644+0000 front 2024-05-03T15:05:32.538760+0000 (oldest deadline 2024-05-03T15:05:58.438503+0000) 2024-05-03T15:06:42.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:41 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:41.840+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:42.440172+0000 front 2024-05-03T15:05:42.440196+0000 (oldest deadline 2024-05-03T15:06:05.940332+0000) 2024-05-03T15:06:42.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:41 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:41.840+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:49.441284+0000 front 2024-05-03T15:05:49.441413+0000 (oldest deadline 2024-05-03T15:06:14.041739+0000) 2024-05-03T15:06:42.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:41 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:41.840+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:55.743016+0000 front 2024-05-03T15:05:55.743076+0000 (oldest deadline 2024-05-03T15:06:21.642629+0000) 2024-05-03T15:06:42.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:41 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:41.840+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:07.443866+0000 front 2024-05-03T15:06:07.443897+0000 (oldest deadline 2024-05-03T15:06:29.744048+0000) 2024-05-03T15:06:42.124 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:41 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:41.840+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-05-03T15:06:14.845949+0000 front 2024-05-03T15:06:14.845978+0000 (oldest deadline 2024-05-03T15:06:39.545914+0000) 2024-05-03T15:06:42.172 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:42 smithi149 podman[70127]: 2024-05-03 15:06:42.008300632 +0000 UTC m=+6.130688017 container cleanup cb0438d840fbcbf0adefec5bd1b60e5b2fafeaaaafa8180daa81114b00111217 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-05-03T15:06:42.172 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:42 smithi149 bash[70127]: ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8 2024-05-03T15:06:42.586 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:42 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:42.119+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.184568+0000 front 2024-05-03T15:05:32.184943+0000 (oldest deadline 2024-05-03T15:05:55.084762+0000) 2024-05-03T15:06:42.586 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:42 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:42.119+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.788481+0000 front 2024-05-03T15:05:40.788594+0000 (oldest deadline 2024-05-03T15:06:04.288673+0000) 2024-05-03T15:06:42.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:42 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:42.119+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:48.892458+0000 front 2024-05-03T15:05:48.892380+0000 (oldest deadline 2024-05-03T15:06:14.792663+0000) 2024-05-03T15:06:42.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:42 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:42.119+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:58.894896+0000 front 2024-05-03T15:05:58.894950+0000 (oldest deadline 2024-05-03T15:06:21.194384+0000) 2024-05-03T15:06:42.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:42 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:42.119+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:05.099256+0000 front 2024-05-03T15:06:05.099353+0000 (oldest deadline 2024-05-03T15:06:29.799510+0000) 2024-05-03T15:06:42.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:42 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:42.119+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-05-03T15:06:11.501358+0000 front 2024-05-03T15:06:11.501226+0000 (oldest deadline 2024-05-03T15:06:37.401492+0000) 2024-05-03T15:06:42.670 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:42 smithi149 podman[70144]: 2024-05-03 15:06:42.309695834 +0000 UTC m=+0.661226154 container remove cb0438d840fbcbf0adefec5bd1b60e5b2fafeaaaafa8180daa81114b00111217 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-05-03T15:06:42.922 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:42 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:42.693+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:06:42.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:42 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:42.693+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:37.896657+0000 front 2024-05-03T15:05:37.896645+0000 (oldest deadline 2024-05-03T15:06:03.796605+0000) 2024-05-03T15:06:42.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:42 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:42.693+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.098837+0000 front 2024-05-03T15:05:50.098910+0000 (oldest deadline 2024-05-03T15:06:11.198967+0000) 2024-05-03T15:06:42.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:42 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:42.693+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:05.801228+0000 front 2024-05-03T15:06:05.801255+0000 (oldest deadline 2024-05-03T15:06:31.101050+0000) 2024-05-03T15:06:42.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:42 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:42.693+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-05-03T15:06:15.002844+0000 front 2024-05-03T15:06:15.002992+0000 (oldest deadline 2024-05-03T15:06:40.902980+0000) 2024-05-03T15:06:42.923 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:42 smithi149 podman[70232]: 2024-05-03 15:06:42.573144681 +0000 UTC m=+0.018117654 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T15:06:42.923 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:42 smithi149 podman[70232]: 2024-05-03 15:06:42.693859861 +0000 UTC m=+0.138832826 container create 0c3824d1d7a192f22cb24794a53656928fee9be6c66464d98980e33b9533c8c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, GIT_CLEAN=True, ceph=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-03T15:06:42.924 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:42 smithi149 podman[70232]: 2024-05-03 15:06:42.879464935 +0000 UTC m=+0.324437892 container init 0c3824d1d7a192f22cb24794a53656928fee9be6c66464d98980e33b9533c8c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-05-03T15:06:42.924 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:42 smithi149 podman[70232]: 2024-05-03 15:06:42.88254905 +0000 UTC m=+0.327522011 container start 0c3824d1d7a192f22cb24794a53656928fee9be6c66464d98980e33b9533c8c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-03T15:06:43.186 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:43 smithi149 podman[70232]: 2024-05-03 15:06:43.013202758 +0000 UTC m=+0.458175728 container attach 0c3824d1d7a192f22cb24794a53656928fee9be6c66464d98980e33b9533c8c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-03T15:06:43.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:42 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:42.874+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.538644+0000 front 2024-05-03T15:05:32.538760+0000 (oldest deadline 2024-05-03T15:05:58.438503+0000) 2024-05-03T15:06:43.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:42 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:42.874+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:42.440172+0000 front 2024-05-03T15:05:42.440196+0000 (oldest deadline 2024-05-03T15:06:05.940332+0000) 2024-05-03T15:06:43.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:42 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:42.874+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:49.441284+0000 front 2024-05-03T15:05:49.441413+0000 (oldest deadline 2024-05-03T15:06:14.041739+0000) 2024-05-03T15:06:43.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:42 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:42.874+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:55.743016+0000 front 2024-05-03T15:05:55.743076+0000 (oldest deadline 2024-05-03T15:06:21.642629+0000) 2024-05-03T15:06:43.375 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:42 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:42.874+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:07.443866+0000 front 2024-05-03T15:06:07.443897+0000 (oldest deadline 2024-05-03T15:06:29.744048+0000) 2024-05-03T15:06:43.375 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:42 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:42.874+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-05-03T15:06:14.845949+0000 front 2024-05-03T15:06:14.845978+0000 (oldest deadline 2024-05-03T15:06:39.545914+0000) 2024-05-03T15:06:43.461 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 03 15:06:43 smithi149 podman[70232]: 2024-05-03 15:06:43.186613826 +0000 UTC m=+0.631586778 container died 0c3824d1d7a192f22cb24794a53656928fee9be6c66464d98980e33b9533c8c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-8-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-05-03T15:06:43.586 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:43 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:43.127+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.184568+0000 front 2024-05-03T15:05:32.184943+0000 (oldest deadline 2024-05-03T15:05:55.084762+0000) 2024-05-03T15:06:43.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:43 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:43.127+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.788481+0000 front 2024-05-03T15:05:40.788594+0000 (oldest deadline 2024-05-03T15:06:04.288673+0000) 2024-05-03T15:06:43.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:43 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:43.127+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:48.892458+0000 front 2024-05-03T15:05:48.892380+0000 (oldest deadline 2024-05-03T15:06:14.792663+0000) 2024-05-03T15:06:43.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:43 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:43.127+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:58.894896+0000 front 2024-05-03T15:05:58.894950+0000 (oldest deadline 2024-05-03T15:06:21.194384+0000) 2024-05-03T15:06:43.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:43 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:43.127+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:05.099256+0000 front 2024-05-03T15:06:05.099353+0000 (oldest deadline 2024-05-03T15:06:29.799510+0000) 2024-05-03T15:06:43.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:43 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:43.127+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-05-03T15:06:11.501358+0000 front 2024-05-03T15:06:11.501226+0000 (oldest deadline 2024-05-03T15:06:37.401492+0000) 2024-05-03T15:06:43.821 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-285d64b2-095a-11ef-bc97-c7b262605968@osd.8.service' 2024-05-03T15:06:43.860 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T15:06:43.860 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-05-03T15:06:43.860 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-05-03T15:06:43.861 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-285d64b2-095a-11ef-bc97-c7b262605968@osd.9 2024-05-03T15:06:44.172 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:43 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:43.728+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:06:44.172 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:43 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:43.728+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:37.896657+0000 front 2024-05-03T15:05:37.896645+0000 (oldest deadline 2024-05-03T15:06:03.796605+0000) 2024-05-03T15:06:44.172 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:43 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:43.728+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.098837+0000 front 2024-05-03T15:05:50.098910+0000 (oldest deadline 2024-05-03T15:06:11.198967+0000) 2024-05-03T15:06:44.173 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:43 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:43.728+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:05.801228+0000 front 2024-05-03T15:06:05.801255+0000 (oldest deadline 2024-05-03T15:06:31.101050+0000) 2024-05-03T15:06:44.173 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:43 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:43.728+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-05-03T15:06:15.002844+0000 front 2024-05-03T15:06:15.002992+0000 (oldest deadline 2024-05-03T15:06:40.902980+0000) 2024-05-03T15:06:44.336 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:43 smithi060 systemd[1]: Stopping Ceph osd.9 for 285d64b2-095a-11ef-bc97-c7b262605968... 2024-05-03T15:06:44.336 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:44 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:44.153+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.184568+0000 front 2024-05-03T15:05:32.184943+0000 (oldest deadline 2024-05-03T15:05:55.084762+0000) 2024-05-03T15:06:44.337 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:44 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:44.153+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.788481+0000 front 2024-05-03T15:05:40.788594+0000 (oldest deadline 2024-05-03T15:06:04.288673+0000) 2024-05-03T15:06:44.337 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:44 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:44.153+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:48.892458+0000 front 2024-05-03T15:05:48.892380+0000 (oldest deadline 2024-05-03T15:06:14.792663+0000) 2024-05-03T15:06:44.337 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:44 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:44.153+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:58.894896+0000 front 2024-05-03T15:05:58.894950+0000 (oldest deadline 2024-05-03T15:06:21.194384+0000) 2024-05-03T15:06:44.337 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:44 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:44.153+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:05.099256+0000 front 2024-05-03T15:06:05.099353+0000 (oldest deadline 2024-05-03T15:06:29.799510+0000) 2024-05-03T15:06:44.337 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:44 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:44.153+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-05-03T15:06:11.501358+0000 front 2024-05-03T15:06:11.501226+0000 (oldest deadline 2024-05-03T15:06:37.401492+0000) 2024-05-03T15:06:44.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:43 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:43.911+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.538644+0000 front 2024-05-03T15:05:32.538760+0000 (oldest deadline 2024-05-03T15:05:58.438503+0000) 2024-05-03T15:06:44.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:43 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:43.911+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:42.440172+0000 front 2024-05-03T15:05:42.440196+0000 (oldest deadline 2024-05-03T15:06:05.940332+0000) 2024-05-03T15:06:44.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:43 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:43.911+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:49.441284+0000 front 2024-05-03T15:05:49.441413+0000 (oldest deadline 2024-05-03T15:06:14.041739+0000) 2024-05-03T15:06:44.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:43 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:43.911+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:55.743016+0000 front 2024-05-03T15:05:55.743076+0000 (oldest deadline 2024-05-03T15:06:21.642629+0000) 2024-05-03T15:06:44.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:43 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:43.911+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:07.443866+0000 front 2024-05-03T15:06:07.443897+0000 (oldest deadline 2024-05-03T15:06:29.744048+0000) 2024-05-03T15:06:44.375 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:43 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:43.911+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-05-03T15:06:14.845949+0000 front 2024-05-03T15:06:14.845978+0000 (oldest deadline 2024-05-03T15:06:39.545914+0000) 2024-05-03T15:06:44.836 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:44 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:44.463+0000 7f2365800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T15:06:44.836 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:44 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:44.463+0000 7f2365800700 -1 osd.9 93 *** Got signal Terminated *** 2024-05-03T15:06:44.837 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:44 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:44.463+0000 7f2365800700 -1 osd.9 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-03T15:06:45.172 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:44 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:44.694+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:06:45.172 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:44 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:44.694+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:37.896657+0000 front 2024-05-03T15:05:37.896645+0000 (oldest deadline 2024-05-03T15:06:03.796605+0000) 2024-05-03T15:06:45.173 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:44 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:44.694+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.098837+0000 front 2024-05-03T15:05:50.098910+0000 (oldest deadline 2024-05-03T15:06:11.198967+0000) 2024-05-03T15:06:45.173 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:44 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:44.694+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:05.801228+0000 front 2024-05-03T15:06:05.801255+0000 (oldest deadline 2024-05-03T15:06:31.101050+0000) 2024-05-03T15:06:45.173 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:44 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:44.694+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-05-03T15:06:15.002844+0000 front 2024-05-03T15:06:15.002992+0000 (oldest deadline 2024-05-03T15:06:40.902980+0000) 2024-05-03T15:06:45.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:44 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:44.936+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.538644+0000 front 2024-05-03T15:05:32.538760+0000 (oldest deadline 2024-05-03T15:05:58.438503+0000) 2024-05-03T15:06:45.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:44 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:44.936+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:42.440172+0000 front 2024-05-03T15:05:42.440196+0000 (oldest deadline 2024-05-03T15:06:05.940332+0000) 2024-05-03T15:06:45.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:44 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:44.936+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:49.441284+0000 front 2024-05-03T15:05:49.441413+0000 (oldest deadline 2024-05-03T15:06:14.041739+0000) 2024-05-03T15:06:45.375 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:44 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:44.936+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:55.743016+0000 front 2024-05-03T15:05:55.743076+0000 (oldest deadline 2024-05-03T15:06:21.642629+0000) 2024-05-03T15:06:45.375 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:44 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:44.936+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:07.443866+0000 front 2024-05-03T15:06:07.443897+0000 (oldest deadline 2024-05-03T15:06:29.744048+0000) 2024-05-03T15:06:45.375 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:44 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:44.936+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-05-03T15:06:14.845949+0000 front 2024-05-03T15:06:14.845978+0000 (oldest deadline 2024-05-03T15:06:39.545914+0000) 2024-05-03T15:06:45.586 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:45 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:45.139+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.184568+0000 front 2024-05-03T15:05:32.184943+0000 (oldest deadline 2024-05-03T15:05:55.084762+0000) 2024-05-03T15:06:45.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:45 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:45.139+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.788481+0000 front 2024-05-03T15:05:40.788594+0000 (oldest deadline 2024-05-03T15:06:04.288673+0000) 2024-05-03T15:06:45.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:45 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:45.139+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:48.892458+0000 front 2024-05-03T15:05:48.892380+0000 (oldest deadline 2024-05-03T15:06:14.792663+0000) 2024-05-03T15:06:45.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:45 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:45.139+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:58.894896+0000 front 2024-05-03T15:05:58.894950+0000 (oldest deadline 2024-05-03T15:06:21.194384+0000) 2024-05-03T15:06:45.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:45 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:45.139+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:05.099256+0000 front 2024-05-03T15:06:05.099353+0000 (oldest deadline 2024-05-03T15:06:29.799510+0000) 2024-05-03T15:06:45.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:45 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:45.139+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-05-03T15:06:11.501358+0000 front 2024-05-03T15:06:11.501226+0000 (oldest deadline 2024-05-03T15:06:37.401492+0000) 2024-05-03T15:06:45.922 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:45 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:45.651+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:06:45.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:45 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:45.651+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:37.896657+0000 front 2024-05-03T15:05:37.896645+0000 (oldest deadline 2024-05-03T15:06:03.796605+0000) 2024-05-03T15:06:45.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:45 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:45.651+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.098837+0000 front 2024-05-03T15:05:50.098910+0000 (oldest deadline 2024-05-03T15:06:11.198967+0000) 2024-05-03T15:06:45.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:45 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:45.651+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:05.801228+0000 front 2024-05-03T15:06:05.801255+0000 (oldest deadline 2024-05-03T15:06:31.101050+0000) 2024-05-03T15:06:45.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:45 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:45.651+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-05-03T15:06:15.002844+0000 front 2024-05-03T15:06:15.002992+0000 (oldest deadline 2024-05-03T15:06:40.902980+0000) 2024-05-03T15:06:45.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:45 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:45.651+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-05-03T15:06:20.903903+0000 front 2024-05-03T15:06:20.903851+0000 (oldest deadline 2024-05-03T15:06:45.603530+0000) 2024-05-03T15:06:46.369 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:45 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:45.938+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.538644+0000 front 2024-05-03T15:05:32.538760+0000 (oldest deadline 2024-05-03T15:05:58.438503+0000) 2024-05-03T15:06:46.369 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:45 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:45.938+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:42.440172+0000 front 2024-05-03T15:05:42.440196+0000 (oldest deadline 2024-05-03T15:06:05.940332+0000) 2024-05-03T15:06:46.369 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:45 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:45.938+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:49.441284+0000 front 2024-05-03T15:05:49.441413+0000 (oldest deadline 2024-05-03T15:06:14.041739+0000) 2024-05-03T15:06:46.369 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:45 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:45.938+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:55.743016+0000 front 2024-05-03T15:05:55.743076+0000 (oldest deadline 2024-05-03T15:06:21.642629+0000) 2024-05-03T15:06:46.369 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:45 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:45.938+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:07.443866+0000 front 2024-05-03T15:06:07.443897+0000 (oldest deadline 2024-05-03T15:06:29.744048+0000) 2024-05-03T15:06:46.369 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:45 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:45.938+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-05-03T15:06:14.845949+0000 front 2024-05-03T15:06:14.845978+0000 (oldest deadline 2024-05-03T15:06:39.545914+0000) 2024-05-03T15:06:46.586 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:46 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:46.098+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.184568+0000 front 2024-05-03T15:05:32.184943+0000 (oldest deadline 2024-05-03T15:05:55.084762+0000) 2024-05-03T15:06:46.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:46 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:46.098+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.788481+0000 front 2024-05-03T15:05:40.788594+0000 (oldest deadline 2024-05-03T15:06:04.288673+0000) 2024-05-03T15:06:46.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:46 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:46.098+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:48.892458+0000 front 2024-05-03T15:05:48.892380+0000 (oldest deadline 2024-05-03T15:06:14.792663+0000) 2024-05-03T15:06:46.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:46 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:46.098+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:58.894896+0000 front 2024-05-03T15:05:58.894950+0000 (oldest deadline 2024-05-03T15:06:21.194384+0000) 2024-05-03T15:06:46.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:46 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:46.098+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:05.099256+0000 front 2024-05-03T15:06:05.099353+0000 (oldest deadline 2024-05-03T15:06:29.799510+0000) 2024-05-03T15:06:46.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:46 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:46.098+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-05-03T15:06:11.501358+0000 front 2024-05-03T15:06:11.501226+0000 (oldest deadline 2024-05-03T15:06:37.401492+0000) 2024-05-03T15:06:46.922 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:46 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:46.622+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:06:46.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:46 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:46.622+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:37.896657+0000 front 2024-05-03T15:05:37.896645+0000 (oldest deadline 2024-05-03T15:06:03.796605+0000) 2024-05-03T15:06:46.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:46 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:46.622+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.098837+0000 front 2024-05-03T15:05:50.098910+0000 (oldest deadline 2024-05-03T15:06:11.198967+0000) 2024-05-03T15:06:46.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:46 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:46.622+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:05.801228+0000 front 2024-05-03T15:06:05.801255+0000 (oldest deadline 2024-05-03T15:06:31.101050+0000) 2024-05-03T15:06:46.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:46 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:46.622+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-05-03T15:06:15.002844+0000 front 2024-05-03T15:06:15.002992+0000 (oldest deadline 2024-05-03T15:06:40.902980+0000) 2024-05-03T15:06:46.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:46 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:46.622+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-05-03T15:06:20.903903+0000 front 2024-05-03T15:06:20.903851+0000 (oldest deadline 2024-05-03T15:06:45.603530+0000) 2024-05-03T15:06:47.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:46 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:46.893+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.538644+0000 front 2024-05-03T15:05:32.538760+0000 (oldest deadline 2024-05-03T15:05:58.438503+0000) 2024-05-03T15:06:47.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:46 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:46.893+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:42.440172+0000 front 2024-05-03T15:05:42.440196+0000 (oldest deadline 2024-05-03T15:06:05.940332+0000) 2024-05-03T15:06:47.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:46 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:46.893+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:49.441284+0000 front 2024-05-03T15:05:49.441413+0000 (oldest deadline 2024-05-03T15:06:14.041739+0000) 2024-05-03T15:06:47.375 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:46 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:46.893+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:55.743016+0000 front 2024-05-03T15:05:55.743076+0000 (oldest deadline 2024-05-03T15:06:21.642629+0000) 2024-05-03T15:06:47.375 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:46 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:46.893+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:07.443866+0000 front 2024-05-03T15:06:07.443897+0000 (oldest deadline 2024-05-03T15:06:29.744048+0000) 2024-05-03T15:06:47.375 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:46 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:46.893+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-05-03T15:06:14.845949+0000 front 2024-05-03T15:06:14.845978+0000 (oldest deadline 2024-05-03T15:06:39.545914+0000) 2024-05-03T15:06:47.586 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:47 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:47.092+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.184568+0000 front 2024-05-03T15:05:32.184943+0000 (oldest deadline 2024-05-03T15:05:55.084762+0000) 2024-05-03T15:06:47.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:47 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:47.092+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.788481+0000 front 2024-05-03T15:05:40.788594+0000 (oldest deadline 2024-05-03T15:06:04.288673+0000) 2024-05-03T15:06:47.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:47 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:47.092+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:48.892458+0000 front 2024-05-03T15:05:48.892380+0000 (oldest deadline 2024-05-03T15:06:14.792663+0000) 2024-05-03T15:06:47.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:47 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:47.092+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:58.894896+0000 front 2024-05-03T15:05:58.894950+0000 (oldest deadline 2024-05-03T15:06:21.194384+0000) 2024-05-03T15:06:47.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:47 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:47.092+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:05.099256+0000 front 2024-05-03T15:06:05.099353+0000 (oldest deadline 2024-05-03T15:06:29.799510+0000) 2024-05-03T15:06:47.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:47 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:47.093+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-05-03T15:06:11.501358+0000 front 2024-05-03T15:06:11.501226+0000 (oldest deadline 2024-05-03T15:06:37.401492+0000) 2024-05-03T15:06:47.922 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:47 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:47.667+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:06:47.922 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:47 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:47.667+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:37.896657+0000 front 2024-05-03T15:05:37.896645+0000 (oldest deadline 2024-05-03T15:06:03.796605+0000) 2024-05-03T15:06:47.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:47 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:47.667+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.098837+0000 front 2024-05-03T15:05:50.098910+0000 (oldest deadline 2024-05-03T15:06:11.198967+0000) 2024-05-03T15:06:47.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:47 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:47.667+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:05.801228+0000 front 2024-05-03T15:06:05.801255+0000 (oldest deadline 2024-05-03T15:06:31.101050+0000) 2024-05-03T15:06:47.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:47 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:47.667+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-05-03T15:06:15.002844+0000 front 2024-05-03T15:06:15.002992+0000 (oldest deadline 2024-05-03T15:06:40.902980+0000) 2024-05-03T15:06:47.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:47 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:47.667+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-05-03T15:06:20.903903+0000 front 2024-05-03T15:06:20.903851+0000 (oldest deadline 2024-05-03T15:06:45.603530+0000) 2024-05-03T15:06:48.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:47 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:47.937+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.538644+0000 front 2024-05-03T15:05:32.538760+0000 (oldest deadline 2024-05-03T15:05:58.438503+0000) 2024-05-03T15:06:48.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:47 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:47.937+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:42.440172+0000 front 2024-05-03T15:05:42.440196+0000 (oldest deadline 2024-05-03T15:06:05.940332+0000) 2024-05-03T15:06:48.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:47 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:47.937+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:49.441284+0000 front 2024-05-03T15:05:49.441413+0000 (oldest deadline 2024-05-03T15:06:14.041739+0000) 2024-05-03T15:06:48.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:47 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:47.937+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:55.743016+0000 front 2024-05-03T15:05:55.743076+0000 (oldest deadline 2024-05-03T15:06:21.642629+0000) 2024-05-03T15:06:48.375 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:47 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:47.937+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:07.443866+0000 front 2024-05-03T15:06:07.443897+0000 (oldest deadline 2024-05-03T15:06:29.744048+0000) 2024-05-03T15:06:48.375 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:47 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:47.937+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-05-03T15:06:14.845949+0000 front 2024-05-03T15:06:14.845978+0000 (oldest deadline 2024-05-03T15:06:39.545914+0000) 2024-05-03T15:06:48.375 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:47 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:47.937+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-05-03T15:06:23.047158+0000 front 2024-05-03T15:06:23.047001+0000 (oldest deadline 2024-05-03T15:06:47.146805+0000) 2024-05-03T15:06:48.586 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:48 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:48.123+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.184568+0000 front 2024-05-03T15:05:32.184943+0000 (oldest deadline 2024-05-03T15:05:55.084762+0000) 2024-05-03T15:06:48.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:48 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:48.123+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.788481+0000 front 2024-05-03T15:05:40.788594+0000 (oldest deadline 2024-05-03T15:06:04.288673+0000) 2024-05-03T15:06:48.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:48 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:48.123+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:48.892458+0000 front 2024-05-03T15:05:48.892380+0000 (oldest deadline 2024-05-03T15:06:14.792663+0000) 2024-05-03T15:06:48.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:48 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:48.123+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:58.894896+0000 front 2024-05-03T15:05:58.894950+0000 (oldest deadline 2024-05-03T15:06:21.194384+0000) 2024-05-03T15:06:48.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:48 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:48.123+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:05.099256+0000 front 2024-05-03T15:06:05.099353+0000 (oldest deadline 2024-05-03T15:06:29.799510+0000) 2024-05-03T15:06:48.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:48 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:48.123+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-05-03T15:06:11.501358+0000 front 2024-05-03T15:06:11.501226+0000 (oldest deadline 2024-05-03T15:06:37.401492+0000) 2024-05-03T15:06:48.922 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:48 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:48.640+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:06:48.922 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:48 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:48.640+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:37.896657+0000 front 2024-05-03T15:05:37.896645+0000 (oldest deadline 2024-05-03T15:06:03.796605+0000) 2024-05-03T15:06:48.922 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:48 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:48.640+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.098837+0000 front 2024-05-03T15:05:50.098910+0000 (oldest deadline 2024-05-03T15:06:11.198967+0000) 2024-05-03T15:06:48.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:48 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:48.640+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:05.801228+0000 front 2024-05-03T15:06:05.801255+0000 (oldest deadline 2024-05-03T15:06:31.101050+0000) 2024-05-03T15:06:48.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:48 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:48.640+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-05-03T15:06:15.002844+0000 front 2024-05-03T15:06:15.002992+0000 (oldest deadline 2024-05-03T15:06:40.902980+0000) 2024-05-03T15:06:48.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:48 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:48.640+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-05-03T15:06:20.903903+0000 front 2024-05-03T15:06:20.903851+0000 (oldest deadline 2024-05-03T15:06:45.603530+0000) 2024-05-03T15:06:49.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:48 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:48.891+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.538644+0000 front 2024-05-03T15:05:32.538760+0000 (oldest deadline 2024-05-03T15:05:58.438503+0000) 2024-05-03T15:06:49.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:48 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:48.891+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:42.440172+0000 front 2024-05-03T15:05:42.440196+0000 (oldest deadline 2024-05-03T15:06:05.940332+0000) 2024-05-03T15:06:49.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:48 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:48.891+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:49.441284+0000 front 2024-05-03T15:05:49.441413+0000 (oldest deadline 2024-05-03T15:06:14.041739+0000) 2024-05-03T15:06:49.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:48 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:48.891+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:55.743016+0000 front 2024-05-03T15:05:55.743076+0000 (oldest deadline 2024-05-03T15:06:21.642629+0000) 2024-05-03T15:06:49.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:48 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:48.891+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:07.443866+0000 front 2024-05-03T15:06:07.443897+0000 (oldest deadline 2024-05-03T15:06:29.744048+0000) 2024-05-03T15:06:49.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:48 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:48.891+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-05-03T15:06:14.845949+0000 front 2024-05-03T15:06:14.845978+0000 (oldest deadline 2024-05-03T15:06:39.545914+0000) 2024-05-03T15:06:49.375 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:48 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:48.891+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-05-03T15:06:23.047158+0000 front 2024-05-03T15:06:23.047001+0000 (oldest deadline 2024-05-03T15:06:47.146805+0000) 2024-05-03T15:06:49.586 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:49 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:49.146+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.184568+0000 front 2024-05-03T15:05:32.184943+0000 (oldest deadline 2024-05-03T15:05:55.084762+0000) 2024-05-03T15:06:49.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:49 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:49.146+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:40.788481+0000 front 2024-05-03T15:05:40.788594+0000 (oldest deadline 2024-05-03T15:06:04.288673+0000) 2024-05-03T15:06:49.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:49 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:49.146+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:48.892458+0000 front 2024-05-03T15:05:48.892380+0000 (oldest deadline 2024-05-03T15:06:14.792663+0000) 2024-05-03T15:06:49.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:49 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:49.146+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:58.894896+0000 front 2024-05-03T15:05:58.894950+0000 (oldest deadline 2024-05-03T15:06:21.194384+0000) 2024-05-03T15:06:49.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:49 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:49.146+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:05.099256+0000 front 2024-05-03T15:06:05.099353+0000 (oldest deadline 2024-05-03T15:06:29.799510+0000) 2024-05-03T15:06:49.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:49 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9[60952]: 2024-05-03T15:06:49.146+0000 7f2361000700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-05-03T15:06:11.501358+0000 front 2024-05-03T15:06:11.501226+0000 (oldest deadline 2024-05-03T15:06:37.401492+0000) 2024-05-03T15:06:49.922 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:49 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:49.641+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:06:49.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:49 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:49.641+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:37.896657+0000 front 2024-05-03T15:05:37.896645+0000 (oldest deadline 2024-05-03T15:06:03.796605+0000) 2024-05-03T15:06:49.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:49 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:49.641+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.098837+0000 front 2024-05-03T15:05:50.098910+0000 (oldest deadline 2024-05-03T15:06:11.198967+0000) 2024-05-03T15:06:49.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:49 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:49.641+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:05.801228+0000 front 2024-05-03T15:06:05.801255+0000 (oldest deadline 2024-05-03T15:06:31.101050+0000) 2024-05-03T15:06:49.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:49 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:49.641+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-05-03T15:06:15.002844+0000 front 2024-05-03T15:06:15.002992+0000 (oldest deadline 2024-05-03T15:06:40.902980+0000) 2024-05-03T15:06:49.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:49 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:49.641+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-05-03T15:06:20.903903+0000 front 2024-05-03T15:06:20.903851+0000 (oldest deadline 2024-05-03T15:06:45.603530+0000) 2024-05-03T15:06:50.319 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:49 smithi060 podman[91034]: 2024-05-03 15:06:49.878349849 +0000 UTC m=+5.570974590 container died c4c65d055eddd63f1436df973c627e031b71547db4ddfe6243a2fc96fcabec8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T15:06:50.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:49 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:49.887+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.538644+0000 front 2024-05-03T15:05:32.538760+0000 (oldest deadline 2024-05-03T15:05:58.438503+0000) 2024-05-03T15:06:50.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:49 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:49.887+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:42.440172+0000 front 2024-05-03T15:05:42.440196+0000 (oldest deadline 2024-05-03T15:06:05.940332+0000) 2024-05-03T15:06:50.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:49 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:49.887+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:49.441284+0000 front 2024-05-03T15:05:49.441413+0000 (oldest deadline 2024-05-03T15:06:14.041739+0000) 2024-05-03T15:06:50.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:49 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:49.887+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:55.743016+0000 front 2024-05-03T15:05:55.743076+0000 (oldest deadline 2024-05-03T15:06:21.642629+0000) 2024-05-03T15:06:50.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:49 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:49.887+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:07.443866+0000 front 2024-05-03T15:06:07.443897+0000 (oldest deadline 2024-05-03T15:06:29.744048+0000) 2024-05-03T15:06:50.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:49 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:49.887+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-05-03T15:06:14.845949+0000 front 2024-05-03T15:06:14.845978+0000 (oldest deadline 2024-05-03T15:06:39.545914+0000) 2024-05-03T15:06:50.375 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:49 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:49.887+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-05-03T15:06:23.047158+0000 front 2024-05-03T15:06:23.047001+0000 (oldest deadline 2024-05-03T15:06:47.146805+0000) 2024-05-03T15:06:50.586 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:50 smithi060 podman[91034]: 2024-05-03 15:06:50.437925706 +0000 UTC m=+6.130550507 container cleanup c4c65d055eddd63f1436df973c627e031b71547db4ddfe6243a2fc96fcabec8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-05-03T15:06:50.587 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:50 smithi060 bash[91034]: ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9 2024-05-03T15:06:50.922 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:50 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:50.671+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:06:50.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:50 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:50.671+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:37.896657+0000 front 2024-05-03T15:05:37.896645+0000 (oldest deadline 2024-05-03T15:06:03.796605+0000) 2024-05-03T15:06:50.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:50 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:50.671+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.098837+0000 front 2024-05-03T15:05:50.098910+0000 (oldest deadline 2024-05-03T15:06:11.198967+0000) 2024-05-03T15:06:50.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:50 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:50.671+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:05.801228+0000 front 2024-05-03T15:06:05.801255+0000 (oldest deadline 2024-05-03T15:06:31.101050+0000) 2024-05-03T15:06:50.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:50 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:50.671+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-05-03T15:06:15.002844+0000 front 2024-05-03T15:06:15.002992+0000 (oldest deadline 2024-05-03T15:06:40.902980+0000) 2024-05-03T15:06:50.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:50 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:50.671+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-05-03T15:06:20.903903+0000 front 2024-05-03T15:06:20.903851+0000 (oldest deadline 2024-05-03T15:06:45.603530+0000) 2024-05-03T15:06:51.086 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:50 smithi060 podman[91054]: 2024-05-03 15:06:50.697275883 +0000 UTC m=+0.797121407 container remove c4c65d055eddd63f1436df973c627e031b71547db4ddfe6243a2fc96fcabec8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=) 2024-05-03T15:06:51.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:50 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:50.935+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.538644+0000 front 2024-05-03T15:05:32.538760+0000 (oldest deadline 2024-05-03T15:05:58.438503+0000) 2024-05-03T15:06:51.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:50 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:50.935+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:42.440172+0000 front 2024-05-03T15:05:42.440196+0000 (oldest deadline 2024-05-03T15:06:05.940332+0000) 2024-05-03T15:06:51.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:50 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:50.935+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:49.441284+0000 front 2024-05-03T15:05:49.441413+0000 (oldest deadline 2024-05-03T15:06:14.041739+0000) 2024-05-03T15:06:51.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:50 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:50.935+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:55.743016+0000 front 2024-05-03T15:05:55.743076+0000 (oldest deadline 2024-05-03T15:06:21.642629+0000) 2024-05-03T15:06:51.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:50 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:50.935+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:07.443866+0000 front 2024-05-03T15:06:07.443897+0000 (oldest deadline 2024-05-03T15:06:29.744048+0000) 2024-05-03T15:06:51.375 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:50 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:50.935+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-05-03T15:06:14.845949+0000 front 2024-05-03T15:06:14.845978+0000 (oldest deadline 2024-05-03T15:06:39.545914+0000) 2024-05-03T15:06:51.375 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:50 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:50.935+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-05-03T15:06:23.047158+0000 front 2024-05-03T15:06:23.047001+0000 (oldest deadline 2024-05-03T15:06:47.146805+0000) 2024-05-03T15:06:51.836 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:51 smithi060 podman[91140]: 2024-05-03 15:06:51.414061521 +0000 UTC m=+0.037661280 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T15:06:51.837 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:51 smithi060 podman[91140]: 2024-05-03 15:06:51.539989728 +0000 UTC m=+0.163589533 container create aff4b95627560951fbd23226dce727209baea4ab985bdc1aef47ff7ff15d4076 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-05-03T15:06:52.172 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:51 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:51.687+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:06:52.172 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:51 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:51.687+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:37.896657+0000 front 2024-05-03T15:05:37.896645+0000 (oldest deadline 2024-05-03T15:06:03.796605+0000) 2024-05-03T15:06:52.173 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:51 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:51.687+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.098837+0000 front 2024-05-03T15:05:50.098910+0000 (oldest deadline 2024-05-03T15:06:11.198967+0000) 2024-05-03T15:06:52.173 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:51 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:51.687+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:05.801228+0000 front 2024-05-03T15:06:05.801255+0000 (oldest deadline 2024-05-03T15:06:31.101050+0000) 2024-05-03T15:06:52.173 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:51 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:51.687+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-05-03T15:06:15.002844+0000 front 2024-05-03T15:06:15.002992+0000 (oldest deadline 2024-05-03T15:06:40.902980+0000) 2024-05-03T15:06:52.173 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:51 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:51.687+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-05-03T15:06:20.903903+0000 front 2024-05-03T15:06:20.903851+0000 (oldest deadline 2024-05-03T15:06:45.603530+0000) 2024-05-03T15:06:52.336 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:51 smithi060 podman[91140]: 2024-05-03 15:06:51.951894607 +0000 UTC m=+0.575494398 container init aff4b95627560951fbd23226dce727209baea4ab985bdc1aef47ff7ff15d4076 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9-deactivate, ceph=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-03T15:06:52.337 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:51 smithi060 podman[91140]: 2024-05-03 15:06:51.964133737 +0000 UTC m=+0.587733579 container start aff4b95627560951fbd23226dce727209baea4ab985bdc1aef47ff7ff15d4076 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9-deactivate, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430) 2024-05-03T15:06:52.337 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:52 smithi060 podman[91140]: 2024-05-03 15:06:52.050506975 +0000 UTC m=+0.674106813 container attach aff4b95627560951fbd23226dce727209baea4ab985bdc1aef47ff7ff15d4076 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T15:06:52.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:51 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:51.981+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.538644+0000 front 2024-05-03T15:05:32.538760+0000 (oldest deadline 2024-05-03T15:05:58.438503+0000) 2024-05-03T15:06:52.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:51 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:51.981+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:42.440172+0000 front 2024-05-03T15:05:42.440196+0000 (oldest deadline 2024-05-03T15:06:05.940332+0000) 2024-05-03T15:06:52.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:51 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:51.981+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:49.441284+0000 front 2024-05-03T15:05:49.441413+0000 (oldest deadline 2024-05-03T15:06:14.041739+0000) 2024-05-03T15:06:52.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:51 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:51.981+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:55.743016+0000 front 2024-05-03T15:05:55.743076+0000 (oldest deadline 2024-05-03T15:06:21.642629+0000) 2024-05-03T15:06:52.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:51 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:51.981+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:07.443866+0000 front 2024-05-03T15:06:07.443897+0000 (oldest deadline 2024-05-03T15:06:29.744048+0000) 2024-05-03T15:06:52.375 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:51 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:51.981+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-05-03T15:06:14.845949+0000 front 2024-05-03T15:06:14.845978+0000 (oldest deadline 2024-05-03T15:06:39.545914+0000) 2024-05-03T15:06:52.375 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:51 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:51.981+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-05-03T15:06:23.047158+0000 front 2024-05-03T15:06:23.047001+0000 (oldest deadline 2024-05-03T15:06:47.146805+0000) 2024-05-03T15:06:52.717 INFO:journalctl@ceph.osd.9.smithi060.stdout:May 03 15:06:52 smithi060 podman[91140]: 2024-05-03 15:06:52.424519523 +0000 UTC m=+1.048119334 container died aff4b95627560951fbd23226dce727209baea4ab985bdc1aef47ff7ff15d4076 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-9-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-05-03T15:06:52.922 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:52 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:52.640+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:06:52.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:52 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:52.640+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:37.896657+0000 front 2024-05-03T15:05:37.896645+0000 (oldest deadline 2024-05-03T15:06:03.796605+0000) 2024-05-03T15:06:52.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:52 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:52.640+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.098837+0000 front 2024-05-03T15:05:50.098910+0000 (oldest deadline 2024-05-03T15:06:11.198967+0000) 2024-05-03T15:06:52.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:52 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:52.640+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:05.801228+0000 front 2024-05-03T15:06:05.801255+0000 (oldest deadline 2024-05-03T15:06:31.101050+0000) 2024-05-03T15:06:52.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:52 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:52.640+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-05-03T15:06:15.002844+0000 front 2024-05-03T15:06:15.002992+0000 (oldest deadline 2024-05-03T15:06:40.902980+0000) 2024-05-03T15:06:52.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:52 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:52.640+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-05-03T15:06:20.903903+0000 front 2024-05-03T15:06:20.903851+0000 (oldest deadline 2024-05-03T15:06:45.603530+0000) 2024-05-03T15:06:53.157 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-285d64b2-095a-11ef-bc97-c7b262605968@osd.9.service' 2024-05-03T15:06:53.265 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T15:06:53.266 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-05-03T15:06:53.266 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-05-03T15:06:53.266 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-285d64b2-095a-11ef-bc97-c7b262605968@osd.10 2024-05-03T15:06:53.292 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:52 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:52.933+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.538644+0000 front 2024-05-03T15:05:32.538760+0000 (oldest deadline 2024-05-03T15:05:58.438503+0000) 2024-05-03T15:06:53.292 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:52 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:52.933+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:42.440172+0000 front 2024-05-03T15:05:42.440196+0000 (oldest deadline 2024-05-03T15:06:05.940332+0000) 2024-05-03T15:06:53.292 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:52 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:52.933+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:49.441284+0000 front 2024-05-03T15:05:49.441413+0000 (oldest deadline 2024-05-03T15:06:14.041739+0000) 2024-05-03T15:06:53.293 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:52 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:52.933+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:55.743016+0000 front 2024-05-03T15:05:55.743076+0000 (oldest deadline 2024-05-03T15:06:21.642629+0000) 2024-05-03T15:06:53.293 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:52 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:52.933+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:07.443866+0000 front 2024-05-03T15:06:07.443897+0000 (oldest deadline 2024-05-03T15:06:29.744048+0000) 2024-05-03T15:06:53.293 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:52 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:52.933+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-05-03T15:06:14.845949+0000 front 2024-05-03T15:06:14.845978+0000 (oldest deadline 2024-05-03T15:06:39.545914+0000) 2024-05-03T15:06:53.293 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:52 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:52.933+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-05-03T15:06:23.047158+0000 front 2024-05-03T15:06:23.047001+0000 (oldest deadline 2024-05-03T15:06:47.146805+0000) 2024-05-03T15:06:53.561 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:53 smithi090 systemd[1]: Stopping Ceph osd.10 for 285d64b2-095a-11ef-bc97-c7b262605968... 2024-05-03T15:06:53.873 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:53 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:53.560+0000 7f7bed471700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T15:06:53.874 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:53 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:53.560+0000 7f7bed471700 -1 osd.10 93 *** Got signal Terminated *** 2024-05-03T15:06:53.874 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:53 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:53.560+0000 7f7bed471700 -1 osd.10 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-03T15:06:53.922 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:53 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:53.632+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:06:53.922 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:53 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:53.632+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:37.896657+0000 front 2024-05-03T15:05:37.896645+0000 (oldest deadline 2024-05-03T15:06:03.796605+0000) 2024-05-03T15:06:53.922 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:53 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:53.632+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.098837+0000 front 2024-05-03T15:05:50.098910+0000 (oldest deadline 2024-05-03T15:06:11.198967+0000) 2024-05-03T15:06:53.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:53 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:53.632+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:05.801228+0000 front 2024-05-03T15:06:05.801255+0000 (oldest deadline 2024-05-03T15:06:31.101050+0000) 2024-05-03T15:06:53.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:53 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:53.632+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-05-03T15:06:15.002844+0000 front 2024-05-03T15:06:15.002992+0000 (oldest deadline 2024-05-03T15:06:40.902980+0000) 2024-05-03T15:06:53.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:53 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:53.632+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-05-03T15:06:20.903903+0000 front 2024-05-03T15:06:20.903851+0000 (oldest deadline 2024-05-03T15:06:45.603530+0000) 2024-05-03T15:06:54.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:53 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:53.886+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.538644+0000 front 2024-05-03T15:05:32.538760+0000 (oldest deadline 2024-05-03T15:05:58.438503+0000) 2024-05-03T15:06:54.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:53 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:53.886+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:42.440172+0000 front 2024-05-03T15:05:42.440196+0000 (oldest deadline 2024-05-03T15:06:05.940332+0000) 2024-05-03T15:06:54.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:53 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:53.886+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:49.441284+0000 front 2024-05-03T15:05:49.441413+0000 (oldest deadline 2024-05-03T15:06:14.041739+0000) 2024-05-03T15:06:54.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:53 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:53.886+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:55.743016+0000 front 2024-05-03T15:05:55.743076+0000 (oldest deadline 2024-05-03T15:06:21.642629+0000) 2024-05-03T15:06:54.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:53 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:53.886+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:07.443866+0000 front 2024-05-03T15:06:07.443897+0000 (oldest deadline 2024-05-03T15:06:29.744048+0000) 2024-05-03T15:06:54.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:53 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:53.886+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-05-03T15:06:14.845949+0000 front 2024-05-03T15:06:14.845978+0000 (oldest deadline 2024-05-03T15:06:39.545914+0000) 2024-05-03T15:06:54.375 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:53 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:53.886+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-05-03T15:06:23.047158+0000 front 2024-05-03T15:06:23.047001+0000 (oldest deadline 2024-05-03T15:06:47.146805+0000) 2024-05-03T15:06:54.922 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:54 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:54.612+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:06:54.922 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:54 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:54.612+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:37.896657+0000 front 2024-05-03T15:05:37.896645+0000 (oldest deadline 2024-05-03T15:06:03.796605+0000) 2024-05-03T15:06:54.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:54 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:54.612+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.098837+0000 front 2024-05-03T15:05:50.098910+0000 (oldest deadline 2024-05-03T15:06:11.198967+0000) 2024-05-03T15:06:54.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:54 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:54.612+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:05.801228+0000 front 2024-05-03T15:06:05.801255+0000 (oldest deadline 2024-05-03T15:06:31.101050+0000) 2024-05-03T15:06:54.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:54 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:54.612+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-05-03T15:06:15.002844+0000 front 2024-05-03T15:06:15.002992+0000 (oldest deadline 2024-05-03T15:06:40.902980+0000) 2024-05-03T15:06:54.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:54 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:54.612+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-05-03T15:06:20.903903+0000 front 2024-05-03T15:06:20.903851+0000 (oldest deadline 2024-05-03T15:06:45.603530+0000) 2024-05-03T15:06:54.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:54 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:54.612+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6820 osd.7 since back 2024-05-03T15:06:31.404718+0000 front 2024-05-03T15:06:31.404806+0000 (oldest deadline 2024-05-03T15:06:53.704567+0000) 2024-05-03T15:06:55.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:54 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:54.936+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.538644+0000 front 2024-05-03T15:05:32.538760+0000 (oldest deadline 2024-05-03T15:05:58.438503+0000) 2024-05-03T15:06:55.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:54 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:54.936+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:42.440172+0000 front 2024-05-03T15:05:42.440196+0000 (oldest deadline 2024-05-03T15:06:05.940332+0000) 2024-05-03T15:06:55.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:54 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:54.936+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:49.441284+0000 front 2024-05-03T15:05:49.441413+0000 (oldest deadline 2024-05-03T15:06:14.041739+0000) 2024-05-03T15:06:55.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:54 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:54.936+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:55.743016+0000 front 2024-05-03T15:05:55.743076+0000 (oldest deadline 2024-05-03T15:06:21.642629+0000) 2024-05-03T15:06:55.375 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:54 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:54.936+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:07.443866+0000 front 2024-05-03T15:06:07.443897+0000 (oldest deadline 2024-05-03T15:06:29.744048+0000) 2024-05-03T15:06:55.375 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:54 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:54.936+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-05-03T15:06:14.845949+0000 front 2024-05-03T15:06:14.845978+0000 (oldest deadline 2024-05-03T15:06:39.545914+0000) 2024-05-03T15:06:55.375 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:54 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:54.936+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-05-03T15:06:23.047158+0000 front 2024-05-03T15:06:23.047001+0000 (oldest deadline 2024-05-03T15:06:47.146805+0000) 2024-05-03T15:06:55.922 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:55 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:55.621+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:06:55.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:55 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:55.621+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:37.896657+0000 front 2024-05-03T15:05:37.896645+0000 (oldest deadline 2024-05-03T15:06:03.796605+0000) 2024-05-03T15:06:55.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:55 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:55.621+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.098837+0000 front 2024-05-03T15:05:50.098910+0000 (oldest deadline 2024-05-03T15:06:11.198967+0000) 2024-05-03T15:06:55.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:55 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:55.621+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:05.801228+0000 front 2024-05-03T15:06:05.801255+0000 (oldest deadline 2024-05-03T15:06:31.101050+0000) 2024-05-03T15:06:55.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:55 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:55.621+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-05-03T15:06:15.002844+0000 front 2024-05-03T15:06:15.002992+0000 (oldest deadline 2024-05-03T15:06:40.902980+0000) 2024-05-03T15:06:55.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:55 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:55.621+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-05-03T15:06:20.903903+0000 front 2024-05-03T15:06:20.903851+0000 (oldest deadline 2024-05-03T15:06:45.603530+0000) 2024-05-03T15:06:55.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:55 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:55.621+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6820 osd.7 since back 2024-05-03T15:06:31.404718+0000 front 2024-05-03T15:06:31.404806+0000 (oldest deadline 2024-05-03T15:06:53.704567+0000) 2024-05-03T15:06:56.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:55 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:55.925+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.538644+0000 front 2024-05-03T15:05:32.538760+0000 (oldest deadline 2024-05-03T15:05:58.438503+0000) 2024-05-03T15:06:56.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:55 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:55.925+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:42.440172+0000 front 2024-05-03T15:05:42.440196+0000 (oldest deadline 2024-05-03T15:06:05.940332+0000) 2024-05-03T15:06:56.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:55 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:55.925+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:49.441284+0000 front 2024-05-03T15:05:49.441413+0000 (oldest deadline 2024-05-03T15:06:14.041739+0000) 2024-05-03T15:06:56.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:55 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:55.925+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:55.743016+0000 front 2024-05-03T15:05:55.743076+0000 (oldest deadline 2024-05-03T15:06:21.642629+0000) 2024-05-03T15:06:56.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:55 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:55.925+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:07.443866+0000 front 2024-05-03T15:06:07.443897+0000 (oldest deadline 2024-05-03T15:06:29.744048+0000) 2024-05-03T15:06:56.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:55 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:55.925+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-05-03T15:06:14.845949+0000 front 2024-05-03T15:06:14.845978+0000 (oldest deadline 2024-05-03T15:06:39.545914+0000) 2024-05-03T15:06:56.375 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:55 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:55.925+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-05-03T15:06:23.047158+0000 front 2024-05-03T15:06:23.047001+0000 (oldest deadline 2024-05-03T15:06:47.146805+0000) 2024-05-03T15:06:56.922 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:56 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:56.605+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:06:56.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:56 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:56.605+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:37.896657+0000 front 2024-05-03T15:05:37.896645+0000 (oldest deadline 2024-05-03T15:06:03.796605+0000) 2024-05-03T15:06:56.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:56 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:56.605+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.098837+0000 front 2024-05-03T15:05:50.098910+0000 (oldest deadline 2024-05-03T15:06:11.198967+0000) 2024-05-03T15:06:56.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:56 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:56.605+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:05.801228+0000 front 2024-05-03T15:06:05.801255+0000 (oldest deadline 2024-05-03T15:06:31.101050+0000) 2024-05-03T15:06:56.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:56 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:56.605+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-05-03T15:06:15.002844+0000 front 2024-05-03T15:06:15.002992+0000 (oldest deadline 2024-05-03T15:06:40.902980+0000) 2024-05-03T15:06:56.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:56 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:56.605+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-05-03T15:06:20.903903+0000 front 2024-05-03T15:06:20.903851+0000 (oldest deadline 2024-05-03T15:06:45.603530+0000) 2024-05-03T15:06:56.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:56 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:56.605+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6820 osd.7 since back 2024-05-03T15:06:31.404718+0000 front 2024-05-03T15:06:31.404806+0000 (oldest deadline 2024-05-03T15:06:53.704567+0000) 2024-05-03T15:06:57.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:56 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:56.886+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.538644+0000 front 2024-05-03T15:05:32.538760+0000 (oldest deadline 2024-05-03T15:05:58.438503+0000) 2024-05-03T15:06:57.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:56 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:56.886+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:42.440172+0000 front 2024-05-03T15:05:42.440196+0000 (oldest deadline 2024-05-03T15:06:05.940332+0000) 2024-05-03T15:06:57.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:56 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:56.886+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:49.441284+0000 front 2024-05-03T15:05:49.441413+0000 (oldest deadline 2024-05-03T15:06:14.041739+0000) 2024-05-03T15:06:57.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:56 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:56.886+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:55.743016+0000 front 2024-05-03T15:05:55.743076+0000 (oldest deadline 2024-05-03T15:06:21.642629+0000) 2024-05-03T15:06:57.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:56 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:56.886+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:07.443866+0000 front 2024-05-03T15:06:07.443897+0000 (oldest deadline 2024-05-03T15:06:29.744048+0000) 2024-05-03T15:06:57.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:56 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:56.886+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-05-03T15:06:14.845949+0000 front 2024-05-03T15:06:14.845978+0000 (oldest deadline 2024-05-03T15:06:39.545914+0000) 2024-05-03T15:06:57.375 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:56 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:56.886+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-05-03T15:06:23.047158+0000 front 2024-05-03T15:06:23.047001+0000 (oldest deadline 2024-05-03T15:06:47.146805+0000) 2024-05-03T15:06:57.922 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:57 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:57.604+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:06:57.922 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:57 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:57.604+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:37.896657+0000 front 2024-05-03T15:05:37.896645+0000 (oldest deadline 2024-05-03T15:06:03.796605+0000) 2024-05-03T15:06:57.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:57 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:57.604+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.098837+0000 front 2024-05-03T15:05:50.098910+0000 (oldest deadline 2024-05-03T15:06:11.198967+0000) 2024-05-03T15:06:57.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:57 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:57.604+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:05.801228+0000 front 2024-05-03T15:06:05.801255+0000 (oldest deadline 2024-05-03T15:06:31.101050+0000) 2024-05-03T15:06:57.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:57 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:57.604+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-05-03T15:06:15.002844+0000 front 2024-05-03T15:06:15.002992+0000 (oldest deadline 2024-05-03T15:06:40.902980+0000) 2024-05-03T15:06:57.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:57 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:57.604+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-05-03T15:06:20.903903+0000 front 2024-05-03T15:06:20.903851+0000 (oldest deadline 2024-05-03T15:06:45.603530+0000) 2024-05-03T15:06:57.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:57 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:57.604+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6820 osd.7 since back 2024-05-03T15:06:31.404718+0000 front 2024-05-03T15:06:31.404806+0000 (oldest deadline 2024-05-03T15:06:53.704567+0000) 2024-05-03T15:06:58.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:57 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:57.919+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:32.538644+0000 front 2024-05-03T15:05:32.538760+0000 (oldest deadline 2024-05-03T15:05:58.438503+0000) 2024-05-03T15:06:58.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:57 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:57.919+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:42.440172+0000 front 2024-05-03T15:05:42.440196+0000 (oldest deadline 2024-05-03T15:06:05.940332+0000) 2024-05-03T15:06:58.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:57 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:57.919+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:49.441284+0000 front 2024-05-03T15:05:49.441413+0000 (oldest deadline 2024-05-03T15:06:14.041739+0000) 2024-05-03T15:06:58.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:57 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:57.919+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-05-03T15:05:55.743016+0000 front 2024-05-03T15:05:55.743076+0000 (oldest deadline 2024-05-03T15:06:21.642629+0000) 2024-05-03T15:06:58.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:57 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:57.919+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:07.443866+0000 front 2024-05-03T15:06:07.443897+0000 (oldest deadline 2024-05-03T15:06:29.744048+0000) 2024-05-03T15:06:58.374 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:57 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:57.919+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-05-03T15:06:14.845949+0000 front 2024-05-03T15:06:14.845978+0000 (oldest deadline 2024-05-03T15:06:39.545914+0000) 2024-05-03T15:06:58.375 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:57 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10[58506]: 2024-05-03T15:06:57.919+0000 7f7be9a65700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-05-03T15:06:23.047158+0000 front 2024-05-03T15:06:23.047001+0000 (oldest deadline 2024-05-03T15:06:47.146805+0000) 2024-05-03T15:06:58.922 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:58 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:58.602+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:06:58.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:58 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:58.602+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:37.896657+0000 front 2024-05-03T15:05:37.896645+0000 (oldest deadline 2024-05-03T15:06:03.796605+0000) 2024-05-03T15:06:58.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:58 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:58.602+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.098837+0000 front 2024-05-03T15:05:50.098910+0000 (oldest deadline 2024-05-03T15:06:11.198967+0000) 2024-05-03T15:06:58.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:58 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:58.602+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:05.801228+0000 front 2024-05-03T15:06:05.801255+0000 (oldest deadline 2024-05-03T15:06:31.101050+0000) 2024-05-03T15:06:58.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:58 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:58.602+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-05-03T15:06:15.002844+0000 front 2024-05-03T15:06:15.002992+0000 (oldest deadline 2024-05-03T15:06:40.902980+0000) 2024-05-03T15:06:58.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:58 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:58.602+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-05-03T15:06:20.903903+0000 front 2024-05-03T15:06:20.903851+0000 (oldest deadline 2024-05-03T15:06:45.603530+0000) 2024-05-03T15:06:58.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:58 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:58.602+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6820 osd.7 since back 2024-05-03T15:06:31.404718+0000 front 2024-05-03T15:06:31.404806+0000 (oldest deadline 2024-05-03T15:06:53.704567+0000) 2024-05-03T15:06:59.345 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:59 smithi090 podman[69528]: 2024-05-03 15:06:59.018797986 +0000 UTC m=+5.564521252 container died 95eea62665208d718566c105af49e4d5cd7e12a5da07399ac92650aa2181391a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-03T15:06:59.624 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:59 smithi090 podman[69528]: 2024-05-03 15:06:59.344994623 +0000 UTC m=+5.890717888 container cleanup 95eea62665208d718566c105af49e4d5cd7e12a5da07399ac92650aa2181391a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-05-03T15:06:59.624 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:59 smithi090 bash[69528]: ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10 2024-05-03T15:06:59.624 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:59 smithi090 podman[69545]: 2024-05-03 15:06:59.538226553 +0000 UTC m=+0.513180350 container remove 95eea62665208d718566c105af49e4d5cd7e12a5da07399ac92650aa2181391a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-03T15:06:59.922 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:59 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:59.592+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:06:59.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:59 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:59.592+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:37.896657+0000 front 2024-05-03T15:05:37.896645+0000 (oldest deadline 2024-05-03T15:06:03.796605+0000) 2024-05-03T15:06:59.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:59 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:59.592+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.098837+0000 front 2024-05-03T15:05:50.098910+0000 (oldest deadline 2024-05-03T15:06:11.198967+0000) 2024-05-03T15:06:59.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:59 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:59.592+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:05.801228+0000 front 2024-05-03T15:06:05.801255+0000 (oldest deadline 2024-05-03T15:06:31.101050+0000) 2024-05-03T15:06:59.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:59 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:59.592+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-05-03T15:06:15.002844+0000 front 2024-05-03T15:06:15.002992+0000 (oldest deadline 2024-05-03T15:06:40.902980+0000) 2024-05-03T15:06:59.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:59 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:59.592+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-05-03T15:06:20.903903+0000 front 2024-05-03T15:06:20.903851+0000 (oldest deadline 2024-05-03T15:06:45.603530+0000) 2024-05-03T15:06:59.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:06:59 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:06:59.592+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6820 osd.7 since back 2024-05-03T15:06:31.404718+0000 front 2024-05-03T15:06:31.404806+0000 (oldest deadline 2024-05-03T15:06:53.704567+0000) 2024-05-03T15:07:00.227 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:06:59 smithi090 podman[69636]: 2024-05-03 15:06:59.863519622 +0000 UTC m=+0.017956780 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T15:07:00.227 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:07:00 smithi090 podman[69636]: 2024-05-03 15:07:00.015449733 +0000 UTC m=+0.169886856 container create c3afe66d19fc4240f0919aed396161b9dc4e0909c82263abc8abb3bdaeaec1a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-03T15:07:00.528 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:07:00 smithi090 podman[69636]: 2024-05-03 15:07:00.226871548 +0000 UTC m=+0.381308680 container init c3afe66d19fc4240f0919aed396161b9dc4e0909c82263abc8abb3bdaeaec1a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10-deactivate, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-05-03T15:07:00.528 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:07:00 smithi090 podman[69636]: 2024-05-03 15:07:00.2302136 +0000 UTC m=+0.384650727 container start c3afe66d19fc4240f0919aed396161b9dc4e0909c82263abc8abb3bdaeaec1a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T15:07:00.528 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:07:00 smithi090 podman[69636]: 2024-05-03 15:07:00.310170841 +0000 UTC m=+0.464607977 container attach c3afe66d19fc4240f0919aed396161b9dc4e0909c82263abc8abb3bdaeaec1a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T15:07:00.838 INFO:journalctl@ceph.osd.10.smithi090.stdout:May 03 15:07:00 smithi090 podman[69636]: 2024-05-03 15:07:00.528136547 +0000 UTC m=+0.682573680 container died c3afe66d19fc4240f0919aed396161b9dc4e0909c82263abc8abb3bdaeaec1a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-05-03T15:07:00.922 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:07:00 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:07:00.543+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:07:00.922 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:07:00 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:07:00.543+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:37.896657+0000 front 2024-05-03T15:05:37.896645+0000 (oldest deadline 2024-05-03T15:06:03.796605+0000) 2024-05-03T15:07:00.922 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:07:00 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:07:00.543+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.098837+0000 front 2024-05-03T15:05:50.098910+0000 (oldest deadline 2024-05-03T15:06:11.198967+0000) 2024-05-03T15:07:00.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:07:00 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:07:00.543+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:05.801228+0000 front 2024-05-03T15:06:05.801255+0000 (oldest deadline 2024-05-03T15:06:31.101050+0000) 2024-05-03T15:07:00.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:07:00 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:07:00.543+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-05-03T15:06:15.002844+0000 front 2024-05-03T15:06:15.002992+0000 (oldest deadline 2024-05-03T15:06:40.902980+0000) 2024-05-03T15:07:00.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:07:00 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:07:00.543+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-05-03T15:06:20.903903+0000 front 2024-05-03T15:06:20.903851+0000 (oldest deadline 2024-05-03T15:06:45.603530+0000) 2024-05-03T15:07:00.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:07:00 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:07:00.543+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6820 osd.7 since back 2024-05-03T15:06:31.404718+0000 front 2024-05-03T15:06:31.404806+0000 (oldest deadline 2024-05-03T15:06:53.704567+0000) 2024-05-03T15:07:01.123 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-285d64b2-095a-11ef-bc97-c7b262605968@osd.10.service' 2024-05-03T15:07:01.200 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T15:07:01.201 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-05-03T15:07:01.201 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-05-03T15:07:01.201 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-285d64b2-095a-11ef-bc97-c7b262605968@osd.11 2024-05-03T15:07:01.503 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:07:01 smithi149 systemd[1]: Stopping Ceph osd.11 for 285d64b2-095a-11ef-bc97-c7b262605968... 2024-05-03T15:07:01.922 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:07:01 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:07:01.501+0000 7f0a1921b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T15:07:01.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:07:01 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:07:01.501+0000 7f0a1921b700 -1 osd.11 93 *** Got signal Terminated *** 2024-05-03T15:07:01.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:07:01 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:07:01.501+0000 7f0a1921b700 -1 osd.11 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-03T15:07:01.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:07:01 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:07:01.555+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:07:01.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:07:01 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:07:01.555+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:37.896657+0000 front 2024-05-03T15:05:37.896645+0000 (oldest deadline 2024-05-03T15:06:03.796605+0000) 2024-05-03T15:07:01.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:07:01 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:07:01.555+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.098837+0000 front 2024-05-03T15:05:50.098910+0000 (oldest deadline 2024-05-03T15:06:11.198967+0000) 2024-05-03T15:07:01.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:07:01 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:07:01.555+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:05.801228+0000 front 2024-05-03T15:06:05.801255+0000 (oldest deadline 2024-05-03T15:06:31.101050+0000) 2024-05-03T15:07:01.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:07:01 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:07:01.555+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-05-03T15:06:15.002844+0000 front 2024-05-03T15:06:15.002992+0000 (oldest deadline 2024-05-03T15:06:40.902980+0000) 2024-05-03T15:07:01.924 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:07:01 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:07:01.555+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-05-03T15:06:20.903903+0000 front 2024-05-03T15:06:20.903851+0000 (oldest deadline 2024-05-03T15:06:45.603530+0000) 2024-05-03T15:07:01.924 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:07:01 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:07:01.555+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6820 osd.7 since back 2024-05-03T15:06:31.404718+0000 front 2024-05-03T15:06:31.404806+0000 (oldest deadline 2024-05-03T15:06:53.704567+0000) 2024-05-03T15:07:02.922 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:07:02 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:07:02.531+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:07:02.922 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:07:02 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:07:02.531+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:37.896657+0000 front 2024-05-03T15:05:37.896645+0000 (oldest deadline 2024-05-03T15:06:03.796605+0000) 2024-05-03T15:07:02.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:07:02 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:07:02.531+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.098837+0000 front 2024-05-03T15:05:50.098910+0000 (oldest deadline 2024-05-03T15:06:11.198967+0000) 2024-05-03T15:07:02.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:07:02 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:07:02.531+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:05.801228+0000 front 2024-05-03T15:06:05.801255+0000 (oldest deadline 2024-05-03T15:06:31.101050+0000) 2024-05-03T15:07:02.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:07:02 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:07:02.531+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-05-03T15:06:15.002844+0000 front 2024-05-03T15:06:15.002992+0000 (oldest deadline 2024-05-03T15:06:40.902980+0000) 2024-05-03T15:07:02.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:07:02 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:07:02.531+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-05-03T15:06:20.903903+0000 front 2024-05-03T15:06:20.903851+0000 (oldest deadline 2024-05-03T15:06:45.603530+0000) 2024-05-03T15:07:02.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:07:02 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:07:02.532+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6820 osd.7 since back 2024-05-03T15:06:31.404718+0000 front 2024-05-03T15:06:31.404806+0000 (oldest deadline 2024-05-03T15:06:53.704567+0000) 2024-05-03T15:07:02.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:07:02 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:07:02.532+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6820 osd.8 since back 2024-05-03T15:06:37.205657+0000 front 2024-05-03T15:06:37.205627+0000 (oldest deadline 2024-05-03T15:07:01.806212+0000) 2024-05-03T15:07:03.922 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:07:03 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:07:03.553+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:07:03.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:07:03 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:07:03.553+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:37.896657+0000 front 2024-05-03T15:05:37.896645+0000 (oldest deadline 2024-05-03T15:06:03.796605+0000) 2024-05-03T15:07:03.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:07:03 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:07:03.553+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.098837+0000 front 2024-05-03T15:05:50.098910+0000 (oldest deadline 2024-05-03T15:06:11.198967+0000) 2024-05-03T15:07:03.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:07:03 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:07:03.553+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:05.801228+0000 front 2024-05-03T15:06:05.801255+0000 (oldest deadline 2024-05-03T15:06:31.101050+0000) 2024-05-03T15:07:03.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:07:03 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:07:03.553+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-05-03T15:06:15.002844+0000 front 2024-05-03T15:06:15.002992+0000 (oldest deadline 2024-05-03T15:06:40.902980+0000) 2024-05-03T15:07:03.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:07:03 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:07:03.553+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-05-03T15:06:20.903903+0000 front 2024-05-03T15:06:20.903851+0000 (oldest deadline 2024-05-03T15:06:45.603530+0000) 2024-05-03T15:07:03.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:07:03 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:07:03.553+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6820 osd.7 since back 2024-05-03T15:06:31.404718+0000 front 2024-05-03T15:06:31.404806+0000 (oldest deadline 2024-05-03T15:06:53.704567+0000) 2024-05-03T15:07:03.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:07:03 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:07:03.553+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6820 osd.8 since back 2024-05-03T15:06:37.205657+0000 front 2024-05-03T15:06:37.205627+0000 (oldest deadline 2024-05-03T15:07:01.806212+0000) 2024-05-03T15:07:04.922 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:07:04 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:07:04.570+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:07:04.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:07:04 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:07:04.570+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:37.896657+0000 front 2024-05-03T15:05:37.896645+0000 (oldest deadline 2024-05-03T15:06:03.796605+0000) 2024-05-03T15:07:04.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:07:04 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:07:04.570+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.098837+0000 front 2024-05-03T15:05:50.098910+0000 (oldest deadline 2024-05-03T15:06:11.198967+0000) 2024-05-03T15:07:04.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:07:04 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:07:04.570+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:05.801228+0000 front 2024-05-03T15:06:05.801255+0000 (oldest deadline 2024-05-03T15:06:31.101050+0000) 2024-05-03T15:07:04.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:07:04 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:07:04.570+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-05-03T15:06:15.002844+0000 front 2024-05-03T15:06:15.002992+0000 (oldest deadline 2024-05-03T15:06:40.902980+0000) 2024-05-03T15:07:04.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:07:04 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:07:04.570+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-05-03T15:06:20.903903+0000 front 2024-05-03T15:06:20.903851+0000 (oldest deadline 2024-05-03T15:06:45.603530+0000) 2024-05-03T15:07:04.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:07:04 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:07:04.570+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6820 osd.7 since back 2024-05-03T15:06:31.404718+0000 front 2024-05-03T15:06:31.404806+0000 (oldest deadline 2024-05-03T15:06:53.704567+0000) 2024-05-03T15:07:04.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:07:04 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:07:04.570+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6820 osd.8 since back 2024-05-03T15:06:37.205657+0000 front 2024-05-03T15:06:37.205627+0000 (oldest deadline 2024-05-03T15:07:01.806212+0000) 2024-05-03T15:07:05.922 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:07:05 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:07:05.591+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-05-03T15:05:31.495344+0000 front 2024-05-03T15:05:31.495545+0000 (oldest deadline 2024-05-03T15:05:55.595384+0000) 2024-05-03T15:07:05.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:07:05 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:07:05.591+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-05-03T15:05:37.896657+0000 front 2024-05-03T15:05:37.896645+0000 (oldest deadline 2024-05-03T15:06:03.796605+0000) 2024-05-03T15:07:05.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:07:05 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:07:05.591+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-03T15:05:50.098837+0000 front 2024-05-03T15:05:50.098910+0000 (oldest deadline 2024-05-03T15:06:11.198967+0000) 2024-05-03T15:07:05.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:07:05 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:07:05.591+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6812 osd.4 since back 2024-05-03T15:06:05.801228+0000 front 2024-05-03T15:06:05.801255+0000 (oldest deadline 2024-05-03T15:06:31.101050+0000) 2024-05-03T15:07:05.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:07:05 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:07:05.591+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-05-03T15:06:15.002844+0000 front 2024-05-03T15:06:15.002992+0000 (oldest deadline 2024-05-03T15:06:40.902980+0000) 2024-05-03T15:07:05.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:07:05 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:07:05.591+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-05-03T15:06:20.903903+0000 front 2024-05-03T15:06:20.903851+0000 (oldest deadline 2024-05-03T15:06:45.603530+0000) 2024-05-03T15:07:05.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:07:05 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:07:05.592+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.90:6820 osd.7 since back 2024-05-03T15:06:31.404718+0000 front 2024-05-03T15:06:31.404806+0000 (oldest deadline 2024-05-03T15:06:53.704567+0000) 2024-05-03T15:07:05.923 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:07:05 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11[59231]: 2024-05-03T15:07:05.592+0000 7f0a1580f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.149:6820 osd.8 since back 2024-05-03T15:06:37.205657+0000 front 2024-05-03T15:06:37.205627+0000 (oldest deadline 2024-05-03T15:07:01.806212+0000) 2024-05-03T15:07:07.228 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:07:06 smithi149 podman[70351]: 2024-05-03 15:07:06.950179364 +0000 UTC m=+5.561172212 container died fe8dc32ddc6e306a539c6f0a4f05f8ba5c84f32b0beadc14cea3238eae326aa8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=) 2024-05-03T15:07:07.562 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:07:07 smithi149 podman[70351]: 2024-05-03 15:07:07.319205236 +0000 UTC m=+5.930198086 container cleanup fe8dc32ddc6e306a539c6f0a4f05f8ba5c84f32b0beadc14cea3238eae326aa8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, ceph=True) 2024-05-03T15:07:07.562 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:07:07 smithi149 bash[70351]: ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11 2024-05-03T15:07:07.871 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:07:07 smithi149 podman[70368]: 2024-05-03 15:07:07.561917526 +0000 UTC m=+0.613439600 container remove fe8dc32ddc6e306a539c6f0a4f05f8ba5c84f32b0beadc14cea3238eae326aa8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240430) 2024-05-03T15:07:08.172 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:07:07 smithi149 podman[70460]: 2024-05-03 15:07:07.871276802 +0000 UTC m=+0.102434891 container create 041f4166bd8067ad1e8d9459eb546e726735ce098bf3eedcccdc71e049be66d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-03T15:07:08.172 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:07:07 smithi149 podman[70460]: 2024-05-03 15:07:07.786704057 +0000 UTC m=+0.017862155 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T15:07:08.173 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:07:08 smithi149 podman[70460]: 2024-05-03 15:07:08.090360972 +0000 UTC m=+0.321519061 container init 041f4166bd8067ad1e8d9459eb546e726735ce098bf3eedcccdc71e049be66d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux ) 2024-05-03T15:07:08.173 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:07:08 smithi149 podman[70460]: 2024-05-03 15:07:08.093691919 +0000 UTC m=+0.324850009 container start 041f4166bd8067ad1e8d9459eb546e726735ce098bf3eedcccdc71e049be66d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, GIT_CLEAN=True) 2024-05-03T15:07:08.672 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:07:08 smithi149 podman[70460]: 2024-05-03 15:07:08.198527866 +0000 UTC m=+0.429685965 container attach 041f4166bd8067ad1e8d9459eb546e726735ce098bf3eedcccdc71e049be66d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11-deactivate, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-03T15:07:08.672 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 03 15:07:08 smithi149 podman[70460]: 2024-05-03 15:07:08.390461922 +0000 UTC m=+0.621620003 container died 041f4166bd8067ad1e8d9459eb546e726735ce098bf3eedcccdc71e049be66d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-03T15:07:08.950 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-285d64b2-095a-11ef-bc97-c7b262605968@osd.11.service' 2024-05-03T15:07:08.988 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T15:07:08.988 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-05-03T15:07:08.988 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-05-03T15:07:08.989 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-285d64b2-095a-11ef-bc97-c7b262605968@mds.a 2024-05-03T15:07:09.337 INFO:journalctl@ceph.mds.a.smithi060.stdout:May 03 15:07:09 smithi060 systemd[1]: Stopping Ceph mds.a for 285d64b2-095a-11ef-bc97-c7b262605968... 2024-05-03T15:07:09.836 INFO:journalctl@ceph.mds.a.smithi060.stdout:May 03 15:07:09 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-a[68311]: 2024-05-03T15:07:09.569+0000 7f3188e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T15:07:09.837 INFO:journalctl@ceph.mds.a.smithi060.stdout:May 03 15:07:09 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-a[68311]: 2024-05-03T15:07:09.569+0000 7f3188e00700 -1 mds.a *** got signal Terminated *** 2024-05-03T15:07:13.511 INFO:journalctl@ceph.mds.a.smithi060.stdout:May 03 15:07:13 smithi060 podman[91256]: 2024-05-03 15:07:13.205701256 +0000 UTC m=+3.757015238 container died 4750ab8adb520a22152554b4c05a81b2e9c0ad385e86a86df3acd42deda7fcc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-a, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-05-03T15:07:13.807 INFO:journalctl@ceph.mds.a.smithi060.stdout:May 03 15:07:13 smithi060 podman[91256]: 2024-05-03 15:07:13.630268973 +0000 UTC m=+4.181582952 container cleanup 4750ab8adb520a22152554b4c05a81b2e9c0ad385e86a86df3acd42deda7fcc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-a, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, ceph=True) 2024-05-03T15:07:13.807 INFO:journalctl@ceph.mds.a.smithi060.stdout:May 03 15:07:13 smithi060 bash[91256]: ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-a 2024-05-03T15:07:14.086 INFO:journalctl@ceph.mds.a.smithi060.stdout:May 03 15:07:13 smithi060 podman[91270]: 2024-05-03 15:07:13.806165389 +0000 UTC m=+0.583560963 container remove 4750ab8adb520a22152554b4c05a81b2e9c0ad385e86a86df3acd42deda7fcc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-a, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T15:07:14.351 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-285d64b2-095a-11ef-bc97-c7b262605968@mds.a.service' 2024-05-03T15:07:14.464 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T15:07:14.465 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-05-03T15:07:14.465 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-05-03T15:07:14.465 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-285d64b2-095a-11ef-bc97-c7b262605968@mds.d 2024-05-03T15:07:15.086 INFO:journalctl@ceph.mds.d.smithi060.stdout:May 03 15:07:14 smithi060 systemd[1]: Stopping Ceph mds.d for 285d64b2-095a-11ef-bc97-c7b262605968... 2024-05-03T15:07:15.587 INFO:journalctl@ceph.mds.d.smithi060.stdout:May 03 15:07:15 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-d[67024]: 2024-05-03T15:07:15.098+0000 7f2c04a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T15:07:15.587 INFO:journalctl@ceph.mds.d.smithi060.stdout:May 03 15:07:15 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-d[67024]: 2024-05-03T15:07:15.098+0000 7f2c04a00700 -1 mds.d *** got signal Terminated *** 2024-05-03T15:07:17.586 INFO:journalctl@ceph.mds.d.smithi060.stdout:May 03 15:07:17 smithi060 podman[91376]: 2024-05-03 15:07:17.314461012 +0000 UTC m=+2.349552775 container died 5ffd9e3041e2afb6067d25aac658497919b80e3ed21bfcc631e778538d99f6d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-05-03T15:07:17.969 INFO:journalctl@ceph.mds.d.smithi060.stdout:May 03 15:07:17 smithi060 podman[91376]: 2024-05-03 15:07:17.759082566 +0000 UTC m=+2.794174322 container cleanup 5ffd9e3041e2afb6067d25aac658497919b80e3ed21bfcc631e778538d99f6d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T15:07:17.969 INFO:journalctl@ceph.mds.d.smithi060.stdout:May 03 15:07:17 smithi060 bash[91376]: ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-d 2024-05-03T15:07:18.336 INFO:journalctl@ceph.mds.d.smithi060.stdout:May 03 15:07:17 smithi060 podman[91390]: 2024-05-03 15:07:17.968233016 +0000 UTC m=+0.638758791 container remove 5ffd9e3041e2afb6067d25aac658497919b80e3ed21bfcc631e778538d99f6d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-d, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-03T15:07:18.430 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-285d64b2-095a-11ef-bc97-c7b262605968@mds.d.service' 2024-05-03T15:07:18.544 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T15:07:18.544 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-05-03T15:07:18.545 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-05-03T15:07:18.545 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-285d64b2-095a-11ef-bc97-c7b262605968@mds.g 2024-05-03T15:07:19.086 INFO:journalctl@ceph.mds.g.smithi060.stdout:May 03 15:07:18 smithi060 systemd[1]: Stopping Ceph mds.g for 285d64b2-095a-11ef-bc97-c7b262605968... 2024-05-03T15:07:19.586 INFO:journalctl@ceph.mds.g.smithi060.stdout:May 03 15:07:19 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-g[67852]: 2024-05-03T15:07:19.108+0000 7fb7c2200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T15:07:19.586 INFO:journalctl@ceph.mds.g.smithi060.stdout:May 03 15:07:19 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-g[67852]: 2024-05-03T15:07:19.108+0000 7fb7c2200700 -1 mds.g *** got signal Terminated *** 2024-05-03T15:07:21.443 INFO:journalctl@ceph.mds.g.smithi060.stdout:May 03 15:07:21 smithi060 podman[91498]: 2024-05-03 15:07:21.186366041 +0000 UTC m=+2.209992570 container died 596ec49ce14cdc814b880abb845f35f6de9bceb8d2fd194180ab565d2ad65279 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-g, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux ) 2024-05-03T15:07:21.773 INFO:journalctl@ceph.mds.g.smithi060.stdout:May 03 15:07:21 smithi060 podman[91498]: 2024-05-03 15:07:21.537752379 +0000 UTC m=+2.561378899 container cleanup 596ec49ce14cdc814b880abb845f35f6de9bceb8d2fd194180ab565d2ad65279 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-g, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-03T15:07:21.773 INFO:journalctl@ceph.mds.g.smithi060.stdout:May 03 15:07:21 smithi060 bash[91498]: ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-g 2024-05-03T15:07:22.086 INFO:journalctl@ceph.mds.g.smithi060.stdout:May 03 15:07:21 smithi060 podman[91512]: 2024-05-03 15:07:21.772060462 +0000 UTC m=+0.566442902 container remove 596ec49ce14cdc814b880abb845f35f6de9bceb8d2fd194180ab565d2ad65279 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-g, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-03T15:07:22.339 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-285d64b2-095a-11ef-bc97-c7b262605968@mds.g.service' 2024-05-03T15:07:22.456 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T15:07:22.456 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-05-03T15:07:22.457 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-05-03T15:07:22.457 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-285d64b2-095a-11ef-bc97-c7b262605968@mds.j 2024-05-03T15:07:22.836 INFO:journalctl@ceph.mds.j.smithi060.stdout:May 03 15:07:22 smithi060 systemd[1]: Stopping Ceph mds.j for 285d64b2-095a-11ef-bc97-c7b262605968... 2024-05-03T15:07:23.336 INFO:journalctl@ceph.mds.j.smithi060.stdout:May 03 15:07:23 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-j[66611]: 2024-05-03T15:07:23.039+0000 7f858b400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T15:07:23.336 INFO:journalctl@ceph.mds.j.smithi060.stdout:May 03 15:07:23 smithi060 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-j[66611]: 2024-05-03T15:07:23.039+0000 7f858b400700 -1 mds.j *** got signal Terminated *** 2024-05-03T15:07:24.586 INFO:journalctl@ceph.mds.j.smithi060.stdout:May 03 15:07:24 smithi060 podman[91618]: 2024-05-03 15:07:24.329586951 +0000 UTC m=+1.417908286 container died 6e88f6f17e7d7ac51bbf65716ec006ececd573e3590fe0e7f39b28f2027bc2a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-j, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-05-03T15:07:24.911 INFO:journalctl@ceph.mds.j.smithi060.stdout:May 03 15:07:24 smithi060 podman[91618]: 2024-05-03 15:07:24.749397581 +0000 UTC m=+1.837718909 container cleanup 6e88f6f17e7d7ac51bbf65716ec006ececd573e3590fe0e7f39b28f2027bc2a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-j, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-03T15:07:24.911 INFO:journalctl@ceph.mds.j.smithi060.stdout:May 03 15:07:24 smithi060 bash[91618]: ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-j 2024-05-03T15:07:25.260 INFO:journalctl@ceph.mds.j.smithi060.stdout:May 03 15:07:24 smithi060 podman[91641]: 2024-05-03 15:07:24.90904828 +0000 UTC m=+0.561994251 container remove 6e88f6f17e7d7ac51bbf65716ec006ececd573e3590fe0e7f39b28f2027bc2a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-j, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True) 2024-05-03T15:07:25.286 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-285d64b2-095a-11ef-bc97-c7b262605968@mds.j.service' 2024-05-03T15:07:25.375 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T15:07:25.375 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-05-03T15:07:25.375 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-05-03T15:07:25.376 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-285d64b2-095a-11ef-bc97-c7b262605968@mds.b 2024-05-03T15:07:25.661 INFO:journalctl@ceph.mds.b.smithi090.stdout:May 03 15:07:25 smithi090 systemd[1]: Stopping Ceph mds.b for 285d64b2-095a-11ef-bc97-c7b262605968... 2024-05-03T15:07:25.925 INFO:journalctl@ceph.mds.b.smithi090.stdout:May 03 15:07:25 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-b[63731]: 2024-05-03T15:07:25.661+0000 7f201ba0a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T15:07:25.926 INFO:journalctl@ceph.mds.b.smithi090.stdout:May 03 15:07:25 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-b[63731]: 2024-05-03T15:07:25.661+0000 7f201ba0a700 -1 mds.b *** got signal Terminated *** 2024-05-03T15:07:29.388 INFO:journalctl@ceph.mds.b.smithi090.stdout:May 03 15:07:29 smithi090 podman[69740]: 2024-05-03 15:07:29.132088532 +0000 UTC m=+3.568633170 container died 0f91cd7392434763593d410402dcdff4871dc7077384f9ecb84a811defba4fec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.build-date=20240430) 2024-05-03T15:07:29.780 INFO:journalctl@ceph.mds.b.smithi090.stdout:May 03 15:07:29 smithi090 podman[69740]: 2024-05-03 15:07:29.48678083 +0000 UTC m=+3.923325508 container cleanup 0f91cd7392434763593d410402dcdff4871dc7077384f9ecb84a811defba4fec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-05-03T15:07:29.780 INFO:journalctl@ceph.mds.b.smithi090.stdout:May 03 15:07:29 smithi090 bash[69740]: ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-b 2024-05-03T15:07:29.919 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-285d64b2-095a-11ef-bc97-c7b262605968@mds.b.service' 2024-05-03T15:07:29.953 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T15:07:29.953 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-05-03T15:07:29.954 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-05-03T15:07:29.954 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-285d64b2-095a-11ef-bc97-c7b262605968@mds.e 2024-05-03T15:07:30.374 INFO:journalctl@ceph.mds.e.smithi090.stdout:May 03 15:07:30 smithi090 systemd[1]: Stopping Ceph mds.e for 285d64b2-095a-11ef-bc97-c7b262605968... 2024-05-03T15:07:30.374 INFO:journalctl@ceph.mds.e.smithi090.stdout:May 03 15:07:30 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-e[63100]: 2024-05-03T15:07:30.265+0000 7f339bd5f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T15:07:30.374 INFO:journalctl@ceph.mds.e.smithi090.stdout:May 03 15:07:30 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-e[63100]: 2024-05-03T15:07:30.265+0000 7f339bd5f700 -1 mds.e *** got signal Terminated *** 2024-05-03T15:07:33.443 INFO:journalctl@ceph.mds.e.smithi090.stdout:May 03 15:07:33 smithi090 podman[69874]: 2024-05-03 15:07:33.143737639 +0000 UTC m=+2.973230784 container died 5daafecc9028ebd3edb91faf819fa78f54abbd8bd8b2e390499e183bd6d277c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-e, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T15:07:33.443 INFO:journalctl@ceph.mds.e.smithi090.stdout:May 03 15:07:33 smithi090 podman[69874]: 2024-05-03 15:07:33.442897083 +0000 UTC m=+3.272390216 container cleanup 5daafecc9028ebd3edb91faf819fa78f54abbd8bd8b2e390499e183bd6d277c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-e, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-05-03T15:07:33.779 INFO:journalctl@ceph.mds.e.smithi090.stdout:May 03 15:07:33 smithi090 bash[69874]: ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-e 2024-05-03T15:07:33.780 INFO:journalctl@ceph.mds.e.smithi090.stdout:May 03 15:07:33 smithi090 podman[69888]: 2024-05-03 15:07:33.669316062 +0000 UTC m=+0.524345090 container remove 5daafecc9028ebd3edb91faf819fa78f54abbd8bd8b2e390499e183bd6d277c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-e, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-03T15:07:33.790 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-285d64b2-095a-11ef-bc97-c7b262605968@mds.e.service' 2024-05-03T15:07:33.823 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T15:07:33.823 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-05-03T15:07:33.824 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-05-03T15:07:33.824 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-285d64b2-095a-11ef-bc97-c7b262605968@mds.h 2024-05-03T15:07:34.123 INFO:journalctl@ceph.mds.h.smithi090.stdout:May 03 15:07:33 smithi090 systemd[1]: Stopping Ceph mds.h for 285d64b2-095a-11ef-bc97-c7b262605968... 2024-05-03T15:07:34.624 INFO:journalctl@ceph.mds.h.smithi090.stdout:May 03 15:07:34 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-h[63425]: 2024-05-03T15:07:34.137+0000 7f0b59480700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T15:07:34.624 INFO:journalctl@ceph.mds.h.smithi090.stdout:May 03 15:07:34 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-h[63425]: 2024-05-03T15:07:34.137+0000 7f0b59480700 -1 mds.h *** got signal Terminated *** 2024-05-03T15:07:37.374 INFO:journalctl@ceph.mds.h.smithi090.stdout:May 03 15:07:37 smithi090 podman[69995]: 2024-05-03 15:07:37.131620751 +0000 UTC m=+3.097019750 container died 025a6e74398801d3634c2e5d9e9064ec58beb13240790993b2581dd71a93ea67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-h, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-03T15:07:37.751 INFO:journalctl@ceph.mds.h.smithi090.stdout:May 03 15:07:37 smithi090 podman[69995]: 2024-05-03 15:07:37.482080766 +0000 UTC m=+3.447479772 container cleanup 025a6e74398801d3634c2e5d9e9064ec58beb13240790993b2581dd71a93ea67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-05-03T15:07:37.752 INFO:journalctl@ceph.mds.h.smithi090.stdout:May 03 15:07:37 smithi090 bash[69995]: ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-h 2024-05-03T15:07:37.904 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-285d64b2-095a-11ef-bc97-c7b262605968@mds.h.service' 2024-05-03T15:07:37.936 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T15:07:37.936 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-05-03T15:07:37.936 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-05-03T15:07:37.936 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-285d64b2-095a-11ef-bc97-c7b262605968@mds.k 2024-05-03T15:07:38.272 INFO:journalctl@ceph.mds.k.smithi090.stdout:May 03 15:07:38 smithi090 systemd[1]: Stopping Ceph mds.k for 285d64b2-095a-11ef-bc97-c7b262605968... 2024-05-03T15:07:38.624 INFO:journalctl@ceph.mds.k.smithi090.stdout:May 03 15:07:38 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-k[62785]: 2024-05-03T15:07:38.270+0000 7f4778f7e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T15:07:38.624 INFO:journalctl@ceph.mds.k.smithi090.stdout:May 03 15:07:38 smithi090 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-k[62785]: 2024-05-03T15:07:38.270+0000 7f4778f7e700 -1 mds.k *** got signal Terminated *** 2024-05-03T15:07:41.415 INFO:journalctl@ceph.mds.k.smithi090.stdout:May 03 15:07:41 smithi090 podman[70117]: 2024-05-03 15:07:41.139437983 +0000 UTC m=+2.985040412 container died 4b2334b62b44c11dc84b7449582b73ba1ad90fa080c3428e9e1a110eebbf7613 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-k, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-03T15:07:41.666 INFO:journalctl@ceph.mds.k.smithi090.stdout:May 03 15:07:41 smithi090 podman[70117]: 2024-05-03 15:07:41.489902888 +0000 UTC m=+3.335505325 container cleanup 4b2334b62b44c11dc84b7449582b73ba1ad90fa080c3428e9e1a110eebbf7613 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-k, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T15:07:41.666 INFO:journalctl@ceph.mds.k.smithi090.stdout:May 03 15:07:41 smithi090 bash[70117]: ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-k 2024-05-03T15:07:41.905 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-285d64b2-095a-11ef-bc97-c7b262605968@mds.k.service' 2024-05-03T15:07:41.925 INFO:journalctl@ceph.mds.k.smithi090.stdout:May 03 15:07:41 smithi090 podman[70132]: 2024-05-03 15:07:41.666065606 +0000 UTC m=+0.518632149 container remove 4b2334b62b44c11dc84b7449582b73ba1ad90fa080c3428e9e1a110eebbf7613 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-k, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430) 2024-05-03T15:07:41.925 INFO:journalctl@ceph.mds.k.smithi090.stdout:May 03 15:07:41 smithi090 systemd[1]: ceph-285d64b2-095a-11ef-bc97-c7b262605968@mds.k.service: Deactivated successfully. 2024-05-03T15:07:41.925 INFO:journalctl@ceph.mds.k.smithi090.stdout:May 03 15:07:41 smithi090 systemd[1]: Stopped Ceph mds.k for 285d64b2-095a-11ef-bc97-c7b262605968. 2024-05-03T15:07:41.925 INFO:journalctl@ceph.mds.k.smithi090.stdout:May 03 15:07:41 smithi090 systemd[1]: ceph-285d64b2-095a-11ef-bc97-c7b262605968@mds.k.service: Consumed 2.573s CPU time. 2024-05-03T15:07:41.935 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T15:07:41.935 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-05-03T15:07:41.935 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-05-03T15:07:41.935 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-285d64b2-095a-11ef-bc97-c7b262605968@mds.c 2024-05-03T15:07:42.301 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 03 15:07:41 smithi149 systemd[1]: Stopping Ceph mds.c for 285d64b2-095a-11ef-bc97-c7b262605968... 2024-05-03T15:07:42.561 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 03 15:07:42 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-c[64220]: 2024-05-03T15:07:42.300+0000 7f4cd4da7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T15:07:42.561 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 03 15:07:42 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-c[64220]: 2024-05-03T15:07:42.300+0000 7f4cd4da7700 -1 mds.c *** got signal Terminated *** 2024-05-03T15:07:45.422 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 03 15:07:45 smithi149 podman[70566]: 2024-05-03 15:07:45.137694846 +0000 UTC m=+3.016337167 container died 45b18e5ebe10a70f66c319db9da6a824a4f5ca851bf651c9f1c1f183a5c7a3b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-c, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T15:07:45.802 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 03 15:07:45 smithi149 podman[70566]: 2024-05-03 15:07:45.542780623 +0000 UTC m=+3.421422933 container cleanup 45b18e5ebe10a70f66c319db9da6a824a4f5ca851bf651c9f1c1f183a5c7a3b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-c, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-05-03T15:07:45.802 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 03 15:07:45 smithi149 bash[70566]: ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-c 2024-05-03T15:07:45.969 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-285d64b2-095a-11ef-bc97-c7b262605968@mds.c.service' 2024-05-03T15:07:46.004 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T15:07:46.004 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-05-03T15:07:46.004 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-05-03T15:07:46.004 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-285d64b2-095a-11ef-bc97-c7b262605968@mds.f 2024-05-03T15:07:46.314 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 03 15:07:46 smithi149 systemd[1]: Stopping Ceph mds.f for 285d64b2-095a-11ef-bc97-c7b262605968... 2024-05-03T15:07:46.315 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 03 15:07:46 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-f[63919]: 2024-05-03T15:07:46.313+0000 7f17d88c9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T15:07:46.315 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 03 15:07:46 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-f[63919]: 2024-05-03T15:07:46.313+0000 7f17d88c9700 -1 mds.f *** got signal Terminated *** 2024-05-03T15:07:49.389 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 03 15:07:49 smithi149 podman[70699]: 2024-05-03 15:07:49.132244843 +0000 UTC m=+2.915690840 container died 937f42513b12102ff78784b5a177d7189f4ada6a0348d2713f9f4a38c8a244d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-f, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T15:07:49.672 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 03 15:07:49 smithi149 podman[70699]: 2024-05-03 15:07:49.463089633 +0000 UTC m=+3.246535628 container cleanup 937f42513b12102ff78784b5a177d7189f4ada6a0348d2713f9f4a38c8a244d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-f, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True) 2024-05-03T15:07:49.672 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 03 15:07:49 smithi149 bash[70699]: ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-f 2024-05-03T15:07:49.887 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-285d64b2-095a-11ef-bc97-c7b262605968@mds.f.service' 2024-05-03T15:07:49.919 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T15:07:49.919 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-05-03T15:07:49.920 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-05-03T15:07:49.920 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-285d64b2-095a-11ef-bc97-c7b262605968@mds.i 2024-05-03T15:07:50.422 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 03 15:07:49 smithi149 systemd[1]: Stopping Ceph mds.i for 285d64b2-095a-11ef-bc97-c7b262605968... 2024-05-03T15:07:50.422 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 03 15:07:50 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-i[64526]: 2024-05-03T15:07:50.215+0000 7fd4252b1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T15:07:50.423 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 03 15:07:50 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-i[64526]: 2024-05-03T15:07:50.215+0000 7fd4252b1700 -1 mds.i *** got signal Terminated *** 2024-05-03T15:07:53.401 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 03 15:07:53 smithi149 podman[70822]: 2024-05-03 15:07:53.137026451 +0000 UTC m=+3.007306787 container died 2fa92e0610c58974fcf3976efeb9d6a9289fce2def8d1b8d34140d29efc13006 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-i, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-03T15:07:53.672 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 03 15:07:53 smithi149 podman[70822]: 2024-05-03 15:07:53.508847581 +0000 UTC m=+3.379127914 container cleanup 2fa92e0610c58974fcf3976efeb9d6a9289fce2def8d1b8d34140d29efc13006 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-i, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-03T15:07:53.673 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 03 15:07:53 smithi149 bash[70822]: ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-i 2024-05-03T15:07:53.928 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-285d64b2-095a-11ef-bc97-c7b262605968@mds.i.service' 2024-05-03T15:07:53.959 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T15:07:53.959 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-05-03T15:07:53.959 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-05-03T15:07:53.960 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-285d64b2-095a-11ef-bc97-c7b262605968@mds.l 2024-05-03T15:07:54.422 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 03 15:07:54 smithi149 systemd[1]: Stopping Ceph mds.l for 285d64b2-095a-11ef-bc97-c7b262605968... 2024-05-03T15:07:54.422 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 03 15:07:54 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-l[63595]: 2024-05-03T15:07:54.261+0000 7f4d1e183700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T15:07:54.423 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 03 15:07:54 smithi149 ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-l[63595]: 2024-05-03T15:07:54.261+0000 7f4d1e183700 -1 mds.l *** got signal Terminated *** 2024-05-03T15:07:57.406 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 03 15:07:57 smithi149 podman[70944]: 2024-05-03 15:07:57.132023746 +0000 UTC m=+2.960193924 container died 9c403211bfc412d732fbb3308df962f8583628549f26d91e7ac17a99462f0ce6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-l, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T15:07:57.665 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 03 15:07:57 smithi149 podman[70944]: 2024-05-03 15:07:57.504243604 +0000 UTC m=+3.332413792 container cleanup 9c403211bfc412d732fbb3308df962f8583628549f26d91e7ac17a99462f0ce6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-l, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-05-03T15:07:57.666 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 03 15:07:57 smithi149 bash[70944]: ceph-285d64b2-095a-11ef-bc97-c7b262605968-mds-l 2024-05-03T15:07:57.892 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-285d64b2-095a-11ef-bc97-c7b262605968@mds.l.service' 2024-05-03T15:07:57.922 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T15:07:57.923 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-05-03T15:07:57.923 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 285d64b2-095a-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-03T15:07:58.261 INFO:teuthology.orchestra.run.smithi060.stdout:Deleting cluster with fsid: 285d64b2-095a-11ef-bc97-c7b262605968 2024-05-03T15:08:03.333 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 285d64b2-095a-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-03T15:08:03.522 INFO:teuthology.orchestra.run.smithi090.stdout:Deleting cluster with fsid: 285d64b2-095a-11ef-bc97-c7b262605968 2024-05-03T15:08:05.981 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 285d64b2-095a-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-03T15:08:06.171 INFO:teuthology.orchestra.run.smithi149.stdout:Deleting cluster with fsid: 285d64b2-095a-11ef-bc97-c7b262605968 2024-05-03T15:08:08.512 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-03T15:08:08.609 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-03T15:08:08.642 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-03T15:08:08.667 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-03T15:08:08.670 DEBUG:teuthology.misc:Transferring archived files from smithi060:/var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687343/remote/smithi060/crash 2024-05-03T15:08:08.671 DEBUG:teuthology.orchestra.run.smithi060:> sudo tar c -f - -C /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/crash -- . 2024-05-03T15:08:08.782 INFO:teuthology.orchestra.run.smithi060.stderr:tar: /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-03T15:08:08.782 INFO:teuthology.orchestra.run.smithi060.stderr:tar: Error is not recoverable: exiting now 2024-05-03T15:08:08.787 DEBUG:teuthology.misc:Transferring archived files from smithi090:/var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687343/remote/smithi090/crash 2024-05-03T15:08:08.788 DEBUG:teuthology.orchestra.run.smithi090:> sudo tar c -f - -C /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/crash -- . 2024-05-03T15:08:08.864 INFO:teuthology.orchestra.run.smithi090.stderr:tar: /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-03T15:08:08.864 INFO:teuthology.orchestra.run.smithi090.stderr:tar: Error is not recoverable: exiting now 2024-05-03T15:08:08.866 DEBUG:teuthology.misc:Transferring archived files from smithi149:/var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687343/remote/smithi149/crash 2024-05-03T15:08:08.867 DEBUG:teuthology.orchestra.run.smithi149:> sudo tar c -f - -C /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/crash -- . 2024-05-03T15:08:08.907 INFO:teuthology.orchestra.run.smithi149.stderr:tar: /var/lib/ceph/285d64b2-095a-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-03T15:08:08.908 INFO:teuthology.orchestra.run.smithi149.stderr:tar: Error is not recoverable: exiting now 2024-05-03T15:08:08.909 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-03T15:08:08.909 DEBUG:teuthology.orchestra.run.smithi060:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-05-03T15:08:09.021 INFO:tasks.cephadm:Compressing logs... 2024-05-03T15:08:09.021 DEBUG:teuthology.orchestra.run.smithi060:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-03T15:08:09.025 DEBUG:teuthology.orchestra.run.smithi090:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-03T15:08:09.028 DEBUG:teuthology.orchestra.run.smithi149:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-03T15:08:09.062 INFO:teuthology.orchestra.run.smithi149.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph.log 2024-05-03T15:08:09.063 INFO:teuthology.orchestra.run.smithi149.stderr:: No such file or directory 2024-05-03T15:08:09.063 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-osd.2.log 2024-05-03T15:08:09.063 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-mds.f.log 2024-05-03T15:08:09.063 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph.audit.log 2024-05-03T15:08:09.063 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-osd.5.log 2024-05-03T15:08:09.063 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-mds.c.log 2024-05-03T15:08:09.064 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph.audit.log: /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-mon.c.log 2024-05-03T15:08:09.064 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-mgr.z.log 2024-05-03T15:08:09.064 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-mds.i.log 2024-05-03T15:08:09.064 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-mgr.z.log: 89.0% -- replaced with /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph.log.gz 2024-05-03T15:08:09.064 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph.cephadm.log 2024-05-03T15:08:09.064 INFO:teuthology.orchestra.run.smithi149.stderr: 90.5% -- replaced with /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph.audit.log.gz 2024-05-03T15:08:09.064 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-osd.11.log 2024-05-03T15:08:09.065 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph.cephadm.log: 82.5% -- replaced with /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph.cephadm.log.gz 2024-05-03T15:08:09.065 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-osd.8.log 2024-05-03T15:08:09.069 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-osd.11.log: 93.3% -- replaced with /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-mgr.z.log.gz 2024-05-03T15:08:09.069 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-volume.log 2024-05-03T15:08:09.070 INFO:teuthology.orchestra.run.smithi090.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph.log 2024-05-03T15:08:09.070 INFO:teuthology.orchestra.run.smithi090.stderr:: No such file or directory 2024-05-03T15:08:09.070 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph.audit.log 2024-05-03T15:08:09.071 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-osd.7.log 2024-05-03T15:08:09.071 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-mds.b.log 2024-05-03T15:08:09.071 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-osd.4.log 2024-05-03T15:08:09.071 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-osd.1.log 2024-05-03T15:08:09.071 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-mgr.y.log 2024-05-03T15:08:09.071 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-mds.h.log 2024-05-03T15:08:09.072 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-mgr.y.log: 89.0%gzip -5 --verbose -- /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-osd.10.log 2024-05-03T15:08:09.072 INFO:teuthology.orchestra.run.smithi090.stderr: -- replaced with /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph.log.gz 2024-05-03T15:08:09.072 INFO:teuthology.orchestra.run.smithi090.stderr: 90.4% -- replaced with /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph.audit.log.gz 2024-05-03T15:08:09.072 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph.cephadm.log 2024-05-03T15:08:09.072 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-mds.e.log 2024-05-03T15:08:09.072 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph.cephadm.log: 82.6% -- replaced with /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph.cephadm.log.gz 2024-05-03T15:08:09.072 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-volume.log 2024-05-03T15:08:09.072 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-mds.e.log: 93.2% -- replaced with /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-mgr.y.log.gz 2024-05-03T15:08:09.072 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-mon.b.log 2024-05-03T15:08:09.074 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-mds.l.log 2024-05-03T15:08:09.077 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-mds.k.log 2024-05-03T15:08:09.079 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-volume.log.gz 2024-05-03T15:08:09.080 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-03T15:08:09.083 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-03T15:08:09.086 INFO:teuthology.orchestra.run.smithi060.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-03T15:08:09.086 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph.log 2024-05-03T15:08:09.088 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph.audit.log 2024-05-03T15:08:09.089 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-mds.g.log 2024-05-03T15:08:09.090 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-mds.j.log 2024-05-03T15:08:09.091 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-mds.g.log: gzip -5 88.9% --verbose -- replaced with /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph.log.gz -- 2024-05-03T15:08:09.092 INFO:teuthology.orchestra.run.smithi060.stderr: /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-osd.3.log 2024-05-03T15:08:09.093 INFO:teuthology.orchestra.run.smithi060.stderr: 90.3% -- replaced with /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph.audit.log.gz 2024-05-03T15:08:09.093 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-osd.0.log 2024-05-03T15:08:09.094 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-mgr.x.log 2024-05-03T15:08:09.096 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-mon.a.log 2024-05-03T15:08:09.097 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-osd.6.log 2024-05-03T15:08:09.099 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph.cephadm.log 2024-05-03T15:08:09.100 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 91.0% -- replaced with /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-volume.log.gz 2024-05-03T15:08:09.101 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-osd.6.log: 94.5% -- replaced with /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-mds.j.log.gz 2024-05-03T15:08:09.101 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-03T15:08:09.102 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph.cephadm.log: 82.8% -- replaced with /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph.cephadm.log.gz 2024-05-03T15:08:09.102 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-mds.a.log 2024-05-03T15:08:09.104 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-volume.log 2024-05-03T15:08:09.106 INFO:teuthology.orchestra.run.smithi149.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-03T15:08:09.107 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-mds.a.log: /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-mds.d.log 2024-05-03T15:08:09.112 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-osd.9.log 2024-05-03T15:08:09.112 INFO:teuthology.orchestra.run.smithi149.stderr: 94.2% -- replaced with /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-mds.c.log.gz 2024-05-03T15:08:09.115 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-03T15:08:09.117 INFO:teuthology.orchestra.run.smithi090.stderr: 94.1% -- replaced with /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-mds.b.log.gz 2024-05-03T15:08:09.118 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-05-03T15:08:09.118 INFO:teuthology.orchestra.run.smithi149.stderr: 94.1% -- replaced with /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-mds.i.log.gz 2024-05-03T15:08:09.120 INFO:teuthology.orchestra.run.smithi090.stderr: 94.2% -- replaced with /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-mds.h.log.gz 2024-05-03T15:08:09.129 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/ceph-client.admin.log: 94.4% -- replaced with /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-mds.d.log.gz 2024-05-03T15:08:09.130 INFO:teuthology.orchestra.run.smithi060.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-05-03T15:08:09.136 INFO:teuthology.orchestra.run.smithi060.stderr: 90.9% -- replaced with /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-volume.log.gz 2024-05-03T15:08:09.144 INFO:teuthology.orchestra.run.smithi060.stderr: 93.6% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-03T15:08:09.155 INFO:teuthology.orchestra.run.smithi149.stderr: 94.5% -- replaced with /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-mds.f.log.gz 2024-05-03T15:08:09.158 INFO:teuthology.orchestra.run.smithi090.stderr: 94.7% -- replaced with /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-mds.k.log.gz 2024-05-03T15:08:09.158 INFO:teuthology.orchestra.run.smithi090.stderr: 94.6% -- replaced with /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-mds.e.log.gz 2024-05-03T15:08:09.173 INFO:teuthology.orchestra.run.smithi149.stderr: 94.6% -- replaced with /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-mds.l.log.gz 2024-05-03T15:08:09.195 INFO:teuthology.orchestra.run.smithi060.stderr: 94.6% -- replaced with /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-mds.g.log.gz 2024-05-03T15:08:09.354 INFO:teuthology.orchestra.run.smithi060.stderr: 93.5% -- replaced with /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-mds.a.log.gz 2024-05-03T15:08:09.476 INFO:teuthology.orchestra.run.smithi060.stderr: 90.4% -- replaced with /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-mgr.x.log.gz 2024-05-03T15:08:09.655 INFO:teuthology.orchestra.run.smithi149.stderr: 92.0% -- replaced with /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-mon.c.log.gz 2024-05-03T15:08:09.673 INFO:teuthology.orchestra.run.smithi090.stderr: 92.0% -- replaced with /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-mon.b.log.gz 2024-05-03T15:08:09.780 INFO:teuthology.orchestra.run.smithi149.stderr: 92.8% -- replaced with /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-osd.11.log.gz 2024-05-03T15:08:10.019 INFO:teuthology.orchestra.run.smithi090.stderr: 92.3% -- replaced with /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-osd.7.log.gz 2024-05-03T15:08:10.079 INFO:teuthology.orchestra.run.smithi060.stderr: 92.4% -- replaced with /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-osd.0.log.gz 2024-05-03T15:08:10.080 INFO:teuthology.orchestra.run.smithi090.stderr: 92.4% -- replaced with /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-osd.1.log.gz 2024-05-03T15:08:10.133 INFO:teuthology.orchestra.run.smithi060.stderr: 92.6% -- replaced with /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-osd.3.log.gz 2024-05-03T15:08:10.228 INFO:teuthology.orchestra.run.smithi090.stderr: 92.3% -- replaced with /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-osd.10.log.gz 2024-05-03T15:08:10.264 INFO:teuthology.orchestra.run.smithi149.stderr: 93.0% -- replaced with /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-osd.8.log.gz 2024-05-03T15:08:10.446 INFO:teuthology.orchestra.run.smithi090.stderr: 92.9% -- replaced with /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-osd.4.log.gz 2024-05-03T15:08:10.448 INFO:teuthology.orchestra.run.smithi090.stderr: 2024-05-03T15:08:10.448 INFO:teuthology.orchestra.run.smithi090.stderr:real 0m1.406s 2024-05-03T15:08:10.448 INFO:teuthology.orchestra.run.smithi090.stderr:user 0m5.248s 2024-05-03T15:08:10.448 INFO:teuthology.orchestra.run.smithi090.stderr:sys 0m0.213s 2024-05-03T15:08:10.453 INFO:teuthology.orchestra.run.smithi060.stderr: 91.9% -- replaced with /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-osd.9.log.gz 2024-05-03T15:08:10.589 INFO:teuthology.orchestra.run.smithi149.stderr: 92.8% -- replaced with /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-osd.5.log.gz 2024-05-03T15:08:10.607 INFO:teuthology.orchestra.run.smithi060.stderr: 90.3% -- replaced with /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-mon.a.log.gz 2024-05-03T15:08:10.645 INFO:teuthology.orchestra.run.smithi149.stderr: 92.8% -- replaced with /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-osd.2.log.gz 2024-05-03T15:08:10.647 INFO:teuthology.orchestra.run.smithi149.stderr: 2024-05-03T15:08:10.647 INFO:teuthology.orchestra.run.smithi149.stderr:real 0m1.605s 2024-05-03T15:08:10.647 INFO:teuthology.orchestra.run.smithi149.stderr:user 0m5.736s 2024-05-03T15:08:10.647 INFO:teuthology.orchestra.run.smithi149.stderr:sys 0m0.230s 2024-05-03T15:08:10.741 INFO:teuthology.orchestra.run.smithi060.stderr: 92.8% -- replaced with /var/log/ceph/285d64b2-095a-11ef-bc97-c7b262605968/ceph-osd.6.log.gz 2024-05-03T15:08:10.744 INFO:teuthology.orchestra.run.smithi060.stderr: 2024-05-03T15:08:10.744 INFO:teuthology.orchestra.run.smithi060.stderr:real 0m1.685s 2024-05-03T15:08:10.744 INFO:teuthology.orchestra.run.smithi060.stderr:user 0m6.775s 2024-05-03T15:08:10.744 INFO:teuthology.orchestra.run.smithi060.stderr:sys 0m0.574s 2024-05-03T15:08:10.745 INFO:tasks.cephadm:Archiving logs... 2024-05-03T15:08:10.745 DEBUG:teuthology.misc:Transferring archived files from smithi060:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687343/remote/smithi060/log 2024-05-03T15:08:10.746 DEBUG:teuthology.orchestra.run.smithi060:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-03T15:08:11.726 DEBUG:teuthology.misc:Transferring archived files from smithi090:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687343/remote/smithi090/log 2024-05-03T15:08:11.727 DEBUG:teuthology.orchestra.run.smithi090:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-03T15:08:12.545 DEBUG:teuthology.misc:Transferring archived files from smithi149:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687343/remote/smithi149/log 2024-05-03T15:08:12.546 DEBUG:teuthology.orchestra.run.smithi149:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-03T15:08:13.453 INFO:tasks.cephadm:Removing cluster... 2024-05-03T15:08:13.453 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 285d64b2-095a-11ef-bc97-c7b262605968 --force 2024-05-03T15:08:13.767 INFO:teuthology.orchestra.run.smithi060.stdout:Deleting cluster with fsid: 285d64b2-095a-11ef-bc97-c7b262605968 2024-05-03T15:08:14.406 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 285d64b2-095a-11ef-bc97-c7b262605968 --force 2024-05-03T15:08:14.612 INFO:teuthology.orchestra.run.smithi090.stdout:Deleting cluster with fsid: 285d64b2-095a-11ef-bc97-c7b262605968 2024-05-03T15:08:14.877 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 285d64b2-095a-11ef-bc97-c7b262605968 --force 2024-05-03T15:08:15.058 INFO:teuthology.orchestra.run.smithi149.stdout:Deleting cluster with fsid: 285d64b2-095a-11ef-bc97-c7b262605968 2024-05-03T15:08:15.317 INFO:tasks.cephadm:Removing cephadm ... 2024-05-03T15:08:15.317 DEBUG:teuthology.orchestra.run.smithi060:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-03T15:08:15.380 DEBUG:teuthology.orchestra.run.smithi090:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-03T15:08:15.395 DEBUG:teuthology.orchestra.run.smithi149:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-03T15:08:15.411 INFO:tasks.cephadm:Teardown complete 2024-05-03T15:08:15.411 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-03T15:08:15.421 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-03T15:08:15.421 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-03T15:08:15.428 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-03T15:08:15.438 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-03T15:08:15.591 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-03T15:08:15.602 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-03T15:08:15.602 DEBUG:teuthology.orchestra.run.smithi060:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-03T15:08:15.611 DEBUG:teuthology.orchestra.run.smithi090:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-03T15:08:15.613 DEBUG:teuthology.orchestra.run.smithi149:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-03T15:08:15.624 INFO:teuthology.orchestra.run.smithi090.stderr:bash: line 1: ntpq: command not found 2024-05-03T15:08:15.627 INFO:teuthology.orchestra.run.smithi149.stderr:bash: line 1: ntpq: command not found 2024-05-03T15:08:15.628 INFO:teuthology.orchestra.run.smithi090.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-03T15:08:15.628 INFO:teuthology.orchestra.run.smithi090.stdout:=============================================================================== 2024-05-03T15:08:15.628 INFO:teuthology.orchestra.run.smithi090.stdout:^* hv01.front.sepia.ceph.com 3 6 377 49 +269us[ +271us] +/- 58ms 2024-05-03T15:08:15.628 INFO:teuthology.orchestra.run.smithi090.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 45 -2606us[-2606us] +/- 66ms 2024-05-03T15:08:15.628 INFO:teuthology.orchestra.run.smithi090.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 49 +2964us[+2964us] +/- 75ms 2024-05-03T15:08:15.629 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T15:08:15.631 INFO:teuthology.orchestra.run.smithi149.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-03T15:08:15.631 INFO:teuthology.orchestra.run.smithi149.stdout:=============================================================================== 2024-05-03T15:08:15.631 INFO:teuthology.orchestra.run.smithi149.stdout:^* hv01.front.sepia.ceph.com 3 6 377 47 +137us[ +131us] +/- 58ms 2024-05-03T15:08:15.631 INFO:teuthology.orchestra.run.smithi149.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 45 -2719us[-2719us] +/- 66ms 2024-05-03T15:08:15.631 INFO:teuthology.orchestra.run.smithi149.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 48 +2833us[+2827us] +/- 75ms 2024-05-03T15:08:15.631 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T15:08:15.661 INFO:teuthology.orchestra.run.smithi060.stderr:bash: line 1: ntpq: command not found 2024-05-03T15:08:15.670 INFO:teuthology.orchestra.run.smithi060.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-03T15:08:15.671 INFO:teuthology.orchestra.run.smithi060.stdout:=============================================================================== 2024-05-03T15:08:15.671 INFO:teuthology.orchestra.run.smithi060.stdout:^* hv01.front.sepia.ceph.com 3 6 377 46 +197us[ +188us] +/- 58ms 2024-05-03T15:08:15.671 INFO:teuthology.orchestra.run.smithi060.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 46 -2666us[-2666us] +/- 66ms 2024-05-03T15:08:15.671 INFO:teuthology.orchestra.run.smithi060.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 47 +2904us[+2895us] +/- 75ms 2024-05-03T15:08:15.671 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T15:08:15.672 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-03T15:08:15.683 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-03T15:08:15.683 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-03T15:08:15.693 DEBUG:teuthology.orchestra.run.smithi060:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-03T15:08:15.916 DEBUG:teuthology.orchestra.run.smithi090:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-03T15:08:15.981 DEBUG:teuthology.orchestra.run.smithi149:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-03T15:08:16.047 DEBUG:teuthology.orchestra.run.smithi060:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-03T15:08:16.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T15:08:16.146 DEBUG:teuthology.orchestra.run.smithi090:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-03T15:08:16.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T15:08:16.173 DEBUG:teuthology.orchestra.run.smithi149:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-03T15:08:16.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T15:08:16.200 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-03T15:08:16.214 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-03T15:08:16.234 INFO:teuthology.task.internal:Duration was 2531.008866 seconds 2024-05-03T15:08:16.235 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-03T15:08:16.244 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-03T15:08:16.244 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-03T15:08:16.252 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-03T15:08:16.254 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-03T15:08:16.288 INFO:teuthology.orchestra.run.smithi090.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-03T15:08:16.290 INFO:teuthology.orchestra.run.smithi149.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-03T15:08:16.402 INFO:teuthology.orchestra.run.smithi060.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-03T15:08:16.712 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-03T15:08:16.712 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi060.front.sepia.ceph.com 2024-05-03T15:08:16.713 DEBUG:teuthology.orchestra.run.smithi060:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-03T15:08:16.791 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi090.front.sepia.ceph.com 2024-05-03T15:08:16.791 DEBUG:teuthology.orchestra.run.smithi090:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-03T15:08:16.819 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi149.front.sepia.ceph.com 2024-05-03T15:08:16.820 DEBUG:teuthology.orchestra.run.smithi149:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-03T15:08:16.848 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-03T15:08:16.848 DEBUG:teuthology.orchestra.run.smithi060:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-03T15:08:16.856 DEBUG:teuthology.orchestra.run.smithi090:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-03T15:08:16.862 DEBUG:teuthology.orchestra.run.smithi149:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-03T15:08:16.946 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-03T15:08:16.946 DEBUG:teuthology.orchestra.run.smithi060:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-03T15:08:16.951 DEBUG:teuthology.orchestra.run.smithi090:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-03T15:08:16.954 DEBUG:teuthology.orchestra.run.smithi149:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-03T15:08:17.649 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-03T15:08:17.660 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-03T15:08:17.660 DEBUG:teuthology.orchestra.run.smithi060:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-03T15:08:17.747 DEBUG:teuthology.orchestra.run.smithi090:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-03T15:08:17.771 DEBUG:teuthology.orchestra.run.smithi149:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-03T15:08:17.797 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-03T15:08:17.808 DEBUG:teuthology.orchestra.run.smithi060:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-03T15:08:17.816 DEBUG:teuthology.orchestra.run.smithi090:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-03T15:08:17.818 DEBUG:teuthology.orchestra.run.smithi149:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-03T15:08:17.838 INFO:teuthology.orchestra.run.smithi090.stdout:kernel.core_pattern = core 2024-05-03T15:08:17.860 INFO:teuthology.orchestra.run.smithi149.stdout:kernel.core_pattern = core 2024-05-03T15:08:17.918 INFO:teuthology.orchestra.run.smithi060.stdout:kernel.core_pattern = core 2024-05-03T15:08:17.964 DEBUG:teuthology.orchestra.run.smithi060:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-03T15:08:18.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T15:08:18.016 DEBUG:teuthology.orchestra.run.smithi090:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-03T15:08:18.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T15:08:18.030 DEBUG:teuthology.orchestra.run.smithi149:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-03T15:08:18.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T15:08:18.045 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-03T15:08:18.056 INFO:teuthology.task.internal:Transferring archived files... 2024-05-03T15:08:18.057 DEBUG:teuthology.misc:Transferring archived files from smithi060:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687343/remote/smithi060 2024-05-03T15:08:18.057 DEBUG:teuthology.orchestra.run.smithi060:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-03T15:08:18.177 DEBUG:teuthology.misc:Transferring archived files from smithi090:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687343/remote/smithi090 2024-05-03T15:08:18.177 DEBUG:teuthology.orchestra.run.smithi090:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-03T15:08:18.211 DEBUG:teuthology.misc:Transferring archived files from smithi149:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687343/remote/smithi149 2024-05-03T15:08:18.211 DEBUG:teuthology.orchestra.run.smithi149:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-03T15:08:18.247 INFO:teuthology.task.internal:Removing archive directory... 2024-05-03T15:08:18.248 DEBUG:teuthology.orchestra.run.smithi060:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-03T15:08:18.255 DEBUG:teuthology.orchestra.run.smithi090:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-03T15:08:18.258 DEBUG:teuthology.orchestra.run.smithi149:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-03T15:08:18.308 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-03T15:08:18.321 INFO:teuthology.task.internal:Not uploading archives. 2024-05-03T15:08:18.322 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-03T15:08:18.333 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-03T15:08:18.334 DEBUG:teuthology.orchestra.run.smithi060:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-03T15:08:18.341 DEBUG:teuthology.orchestra.run.smithi090:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-03T15:08:18.343 DEBUG:teuthology.orchestra.run.smithi149:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-03T15:08:18.355 INFO:teuthology.orchestra.run.smithi090.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 3 15:08 /home/ubuntu/cephtest 2024-05-03T15:08:18.358 INFO:teuthology.orchestra.run.smithi149.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 3 15:08 /home/ubuntu/cephtest 2024-05-03T15:08:18.394 INFO:teuthology.orchestra.run.smithi060.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 3 15:08 /home/ubuntu/cephtest 2024-05-03T15:08:18.398 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-03T15:08:18.409 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-03T15:08:18.431 INFO:teuthology.nuke:Checking targets against current locks 2024-05-03T15:08:18.454 DEBUG:teuthology.nuke:shortname: smithi060 2024-05-03T15:08:18.455 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-03T15:08:18.479 DEBUG:teuthology.nuke:shortname: smithi090 2024-05-03T15:08:18.479 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-03T15:08:18.484 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi060.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687343', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-03 14:16:48.122341', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN60ohXqyQ4xTGpuJWX9HxonnjrmxRBmiqHg5iN26cwI/KCpNJ5Sc1kSgjcB2p3dpSe7oz9UzLlYPp5bTS+Am+E='} 2024-05-03T15:08:18.503 DEBUG:teuthology.nuke:shortname: smithi149 2024-05-03T15:08:18.503 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-03T15:08:18.508 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi090.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687343', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-03 14:16:48.120985', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOlmViT/vmNVrEbE6Z+ZbKbEiQbLdqSvx1HK3HuBrCj8yAimQ9rfBovQ4tx6cxlcm+K1iqPhg3qtIQz6dLl8CRo='} 2024-05-03T15:08:18.536 INFO:teuthology.orchestra.console.smithi060:Power off 2024-05-03T15:08:18.536 DEBUG:teuthology.orchestra.console.smithi060:pexpect command: ipmitool -H smithi060.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-03T15:08:18.550 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi149.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687343', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-03 14:16:48.119511', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB+gfbFMrK4+ZHV2UxCa21jLg5KDjBEEOh1D2EqBf8EtG7hwlZl2V6EHmyJ7kJDQVC1ELMmTGpXlQKZHTKZbu98='} 2024-05-03T15:08:18.561 DEBUG:teuthology.orchestra.console.smithi060:power off output: Chassis Power Control: Down/Off 2024-05-03T15:08:18.561 DEBUG:teuthology.orchestra.console.smithi060:pexpect command: ipmitool -H smithi060.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-03T15:08:18.582 INFO:teuthology.orchestra.console.smithi090:Power off 2024-05-03T15:08:18.582 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-03T15:08:18.597 DEBUG:teuthology.orchestra.console.smithi060:check power output: Chassis Power is on 2024-05-03T15:08:18.606 DEBUG:teuthology.orchestra.console.smithi090:power off output: Chassis Power Control: Down/Off 2024-05-03T15:08:18.606 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-03T15:08:18.627 INFO:teuthology.orchestra.console.smithi149:Power off 2024-05-03T15:08:18.627 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-03T15:08:18.637 DEBUG:teuthology.orchestra.console.smithi090:check power output: Chassis Power is on 2024-05-03T15:08:18.651 DEBUG:teuthology.orchestra.console.smithi149:power off output: Chassis Power Control: Down/Off 2024-05-03T15:08:18.651 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-03T15:08:18.674 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is on 2024-05-03T15:08:22.597 DEBUG:teuthology.orchestra.console.smithi060:pexpect command: ipmitool -H smithi060.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-03T15:08:22.638 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-03T15:08:22.676 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-03T15:08:22.709 DEBUG:teuthology.orchestra.console.smithi060:check power output: Chassis Power is on 2024-05-03T15:08:22.750 DEBUG:teuthology.orchestra.console.smithi090:check power output: Chassis Power is on 2024-05-03T15:08:22.786 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is on 2024-05-03T15:08:26.714 DEBUG:teuthology.orchestra.console.smithi060:pexpect command: ipmitool -H smithi060.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-03T15:08:26.750 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-03T15:08:26.787 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-03T15:08:26.825 DEBUG:teuthology.orchestra.console.smithi060:check power output: Chassis Power is off 2024-05-03T15:08:26.860 DEBUG:teuthology.orchestra.console.smithi090:check power output: Chassis Power is off 2024-05-03T15:08:26.898 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is off 2024-05-03T15:08:26.926 INFO:teuthology.orchestra.console.smithi060:Power off completed 2024-05-03T15:08:26.961 INFO:teuthology.orchestra.console.smithi090:Power off completed 2024-05-03T15:08:26.999 INFO:teuthology.orchestra.console.smithi149:Power off completed 2024-05-03T15:08:27.104 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/suites/pjd}} duration: 2531.008866071701 failure_reason: 'Command failed on smithi060 with status 110: "sudo TESTDIR=/home/ubuntu/cephtest bash -c ''ceph fs subvolumegroup pin cephfs qa random 0.10''"' flavor: default owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=d7925e8176b54e8c95f9433176f35efe status: fail success: false 2024-05-03T15:08:27.105 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-03T15:08:27.190 INFO:teuthology.run:FAIL